Consulting Benchmark Retrieval Tool Readiness Checklist

Role: Lead AI Automation Engineer in a global management consulting practice.

Context

• Consultancy: {{consultancy_name}}
• Client Industry: {{client_industry}}
• Data Endpoints: {{retrieval_tool_endpoints}}
• User Clearance Level: {{user_clearance_level}}
• Per-Call Token Budget: {{max_token_budget_per_call}}
• Fallback Service: {{fallback_search_service}}

Task

Produce an operational readiness checklist to ensure that an automated research agent accurately selects, parametrizes, and parses data retrieval tools across proprietary benchmarks and market research repositories without exceeding token limits or returning ungrounded metrics.

Method

1. Analyze function definition descriptions for {{retrieval_tool_endpoints}} to ensure semantic clarity for the model's tool selection.
2. Validate that function parameters enforce industry categorization standards relevant to {{client_industry}}.
3. Verify role-based access filtering matches {{user_clearance_level}} across all data extraction endpoints.
4. Assess agent payload truncation rules to prevent responses from exceeding {{max_token_budget_per_call}}.
5. Confirm that JSON output parsing includes automated schema validation before returning metrics to consultants.
6. Test agent failover paths to {{fallback_search_service}} when primary data tools return empty sets or HTTP 5xx codes.
7. Establish accuracy benchmarks for structured outputs like financial ratios, valuation multiples, and operational metrics.

Constraints

• MUST restrict tool access strictly to schemas authorized under {{user_clearance_level}}.
• MUST NOT permit recursive tool calling loops exceeding three sequential invocations.
• All checklist items MUST have an assigned validation method (e.g., automated test, manual inspection).
• Deliverable must remain concise and directly actionable for consulting tech teams.

Output format

Provide the response in the following order:

1. Execution Profile (concise overview table including {{consultancy_name}} and {{client_industry}} boundaries).
2. Readiness Checklist (10-15 checklist items grouped into: Tool Definition, Parameter Safety, Execution Limits, and Fallback Handling).
3. Failure Response Protocol (4-5 concrete steps when tool calls fail in production).

Self-review

• Check that token limit enforcement is directly verified against {{max_token_budget_per_call}}.
• Confirm the fallback mechanism to {{fallback_search_service}} is comprehensively checked.
• Verify every checklist item uses markdown checkbox notation (- [ ]).