Multi-Modal Agent Visual Verification Protocol

Role: Lead Multi-Modal AI Evaluation Engineer designing autonomous visual QA, inspection, and prompt-refinement loops.

Context

• Multi-Modal Runtime: {{visual_agent_runtime}}
• Target Inspection Criteria: {{inspection_criteria}}
• Inbound Generation Payload: {{tool_call_payload}}
• Pass/Fail Quality Thresholds: {{pass_fail_thresholds}}
• Automated Remediation Workflow: {{anomaly_mitigation_action}}
• Vision Evaluation Backbone: {{evaluation_vision_model}}

Task

Develop an autonomous visual verification and prompt-correction brief that enables a vision agent to critically inspect diffusion model outputs against defined visual tokens and output actionable, differential prompt revisions upon quality failure.

Method

1. Deconstruct the original generation intent from {{tool_call_payload}} into verifiable compositional components.
2. Establish multi-point visual inspection checks using {{evaluation_vision_model}} focused directly on {{inspection_criteria}}.
3. Calibrate deterministic scoring algorithms aligned to {{pass_fail_thresholds}} covering composition, anatomy, lighting, and prompt adherence.
4. Design difference-detection routines that identify missing entities, spatial hallucinations, or unintended color shifts.
5. Formulate iterative prompt-patching logic that modifies positive weights, adds targeted negative prompts, or adjusts seed parameters.
6. Detail tool-dispatch triggers for {{anomaly_mitigation_action}} when an asset fails consecutive verification cycles.
7. Specify structured JSON response formats for the critic agent to communicate grading results back to the workflow controller.

Constraints

• The evaluation framework MUST return quantitative rubric scores across all predefined criteria.
• The critic agent MUST NOT accept assets falling below {{pass_fail_thresholds}} under any circumstance.
• Revised prompt payloads must isolate and repair only the identified visual defects without disrupting preserved elements.
• Inspection heuristics must be compatible with the inference constraints of {{visual_agent_runtime}}.

Output format

Deliver an engineering brief containing:

1. Visual Inspection Architecture & Rubric Matrix (Scoring table with 1-5 scale benchmarks)
2. Vision-Agent System Prompt Directive (Verbatim prompt template)
3. Remediation & Differential Prompt Modification Rules (Conditional adjustment trees)
4. Tool Output Schema (JSON structure for critique, scores, and updated prompt payload)

Self-review

• Confirm all context variables ({{visual_agent_runtime}}, {{inspection_criteria}}, {{tool_call_payload}}, {{pass_fail_thresholds}}, {{anomaly_mitigation_action}}, {{evaluation_vision_model}}) are logically leveraged.
• Verify that the differential prompt modification step explicitly prevents full re-roll degradation.
• Ensure the output contract contains concrete evaluation schema definitions.