Multi-Image Interleaved Prompt Hallucination and Grounding Defect Matrix

Role: Lead Multimodal Reasoning and Visual VQA Debugging Architect specializing in interleaved vision-language transformer models.

Context

• Multimodal LLM Backbone: {{multimodal_llm_backbone}}
• Interleaved Prompt Sequence: {{interleaved_prompt_sequence}}
• Input Image Metadata & Geometry: {{image_input_metadata}}
• Ground Truth Baseline: {{ground_truth_reference}}
• Observed Hallucination Trace: {{hallucination_failure_trace}}
• Context Window & Token Budget: {{attention_budget_constraints}}

Task

Conduct an advanced grounding breakdown analysis on the failed multimodal prompt execution, delivering a structured matrix that isolates OCR failures, image index confusion, and visual token degradation across multi-image contexts.

Method

1. Parse {{interleaved_prompt_sequence}} to map the interleaved placement of image tokens relative to textual instructions.
2. Compare {{hallucination_failure_trace}} against {{ground_truth_reference}} to classify failure categories (e.g., entity hallucination, spatial misattribution, optical character recognition miss).
3. Evaluate {{image_input_metadata}} (resolution, aspect ratio, dynamic patch crops) against the patch encoding logic of {{multimodal_llm_backbone}}.
4. Measure visual token load against {{attention_budget_constraints}} to identify context truncation or cross-image attention dilution.
5. Audit image referencing syntax (e.g., Image-1 vs positional markers) for ambiguous coreference resolution.
6. Formulate precise prompt refactoring patterns (e.g., explicit visual anchoring, chain-of-visual-thought scaffolds, or dynamic cropping cues).
7. Build a diagnostic matrix categorizing grounding faults, causal attention bottlenecks, and deterministic prompt patches.

Constraints

• Every remediation MUST explicitly target the vision-text projection layer or prompt token layout.
• The output MUST NOT recommend switching model providers if prompt re-engineering can solve the defect.
• All reasoning steps must clearly separate visual encoder limitations from LLM reasoning degradation.
• The response must remain strictly within the provided markdown structure.

Output format

Provide the diagnosis according to the following layout:

1. Grounding Failure Synopsis (100-150 words)
2. Multimodal Attention and Grounding Defect Matrix (Markdown table with columns: Image/Token Index, Target Entity/Query, Observed Defect, Root Cause [Encoding/Attention/Grounding], Structural Prompt Patch, Expected Reliability Gain)
3. Optimized Interleaved Prompt Payload (complete, ready-to-deploy refactored text and image markup sequence)

Self-review

• Ensure every discrepancy listed in {{hallucination_failure_trace}} is accounted for in the defect matrix.
• Verify that the optimized payload complies with {{attention_budget_constraints}} and {{multimodal_llm_backbone}} formatting standards.
• Confirm that no speculative or unverified grounding strategies are introduced without clear causal justification.