Omnichannel Support Agent Quality Assurance Checklist

Role: Principal Conversational AI QA Director specializing in enterprise retail and consumer e-commerce customer support systems.

Context

• Retail brand: {{retail_brand_name}}
• Deployment touchpoints: {{support_channels}}
• Integrated backend stack: {{agent_integration_stack}}
• Evaluation batch size: {{evaluation_sample_size}}
• Policy and handoff triggers: {{escalation_threshold_rules}}
• Voice and style parameters: {{brand_tone_guidelines}}

Task

Generate a comprehensive, actionable evaluation checklist to audit {{retail_brand_name}} conversational AI agents across accuracy, transaction handling, guardrail adherence, and human handover reliability across {{support_channels}}.

Method

1. Analyze {{support_channels}} and {{agent_integration_stack}} to establish integration test points for order lookup, returns processing, and account management.
2. Review {{escalation_threshold_rules}} to construct verification checkpoints for sentiment decline, policy edge cases, and high-value customer escalation.
3. Map {{brand_tone_guidelines}} into explicit linguistic scoring criteria, assessing empathy, concise product explanations, and brand consistency.
4. Design transaction verification checks ensuring the agent queries real-time inventory and shipping APIs without hallucinating delivery dates or refund amounts.
5. Formulate security and PII compliance audit items covering credit card masking, address validation, and authentication verification.
6. Structure multi-turn failure recovery checks assessing how gracefully the agent handles ambiguity, slang, and sudden topic shifts.
7. Establish deterministic pass/fail metrics and remediation actions for each audit line item based on a {{evaluation_sample_size}} transcript sample.

Constraints

• Every checklist item MUST include a verification procedure, pass/fail threshold, and severity rating (Critical, High, Medium).
• The checklist MUST NOT include generic software testing items unrelated to conversational retail agent performance.
• You MUST explicitly reference {{escalation_threshold_rules}} and {{brand_tone_guidelines}} in the evaluation criteria.
• Remediation triggers must identify specific owner roles across engineering, product, or customer experience.

Output format

• Section 1: Audit Scope & Architecture Summary (max 150 words)
• Section 2: Core Capability Evaluation Checklist (table with Columns: Category, Checkpoint, Verification Method, Pass Criteria, Severity)
• Section 3: Guardrails & Policy Adherence Checklist (ordered list with sub-bullets for failure thresholds)
• Section 4: Audit Scoring Matrix & Escalation Protocol (max 200 words)

Self-review

• Are all 6 contextual variables explicitly integrated into the verification steps?
• Are pass/fail criteria measurable rather than subjective?
• Does the checklist cover both conversational UX and backend transactional integrity?