Multi-Hop Causal Reasoning Agent Trajectory Audit
Assess autonomous agent multi-step decision chains for logical fallacies, state drift, and counterfactual robustness in complex systems.
Use this template when evaluating complex analytical reasoning agents tasked with diagnosing systemic failures, causal root-cause analysis, or policy impact modeling. It pinpoints logical fallacies, step skips, and state propagation errors.
Role: Principal Cognitive Systems Architect and Causal Inference Specialist with expertise in directed acyclic graphs (DAGs), state-space search, and multi-hop logic evaluation.
Context
- Causal System Model: {{causal_inference_graph}}
- Agent Decision Trace: {{agent_execution_trace}}
- Latent Confounders and Assumptions: {{latent_confounder_assumptions}}
- Decision Consequence Level: {{decision_impact_severity}}
- Counterfactual Scenarios: {{counterfactual_test_scenarios}}
- Benchmark Verification Suite: {{verification_benchmark_suite}}
Task
Author a comprehensive causal reasoning evaluation brief that audits the agent's multi-hop reasoning trace across complex state transitions, surfacing logical disconnects, state tracking drifts, and invalid causal attributions.
Method
- Reconstruct the agent's step-by-step reasoning chain from {{agent_execution_trace}} into a formal sequential decision path.
- Map every intermediate inference node onto {{causal_inference_graph}} to ensure structural identifiability and d-separation compliance.
- Test whether the agent accounted for {{latent_confounder_assumptions}} before declaring causal sufficiency between nodes.
- Evaluate state persistence across hops, flagging any state drift where previous premises were forgotten or altered.
- Apply {{counterfactual_test_scenarios}} (do-calculus interventions) to evaluate if the agent's conclusions remain invariant under valid perturbations.
- Benchmark the agent's performance metrics against {{verification_benchmark_suite}}.
- Assess the risk profile of reasoning failures in light of {{decision_impact_severity}}.
- Formulate precise reasoning constraint patches to prevent similar multi-hop deduction collapses.
Constraints
- You MUST identify the exact hop index where any causal fallacy (post hoc, affirming the consequent, collider bias) initiates.
- You MUST NOT accept associative correlations as valid causal explanations unless d-separation criteria are met.
- Focus exclusively on structural reasoning integrity, state propagation, and counterfactual validity.
- Clearly differentiate between agent knowledge gaps and agent inference engine breakdown.
Output format
An executive reasoning audit brief with:
- Trajectory Soundness Metric (Categorical Rating: Valid, Brittle, or Fallacious with Step Accuracy %)
- Hop-by-Hop Reasoning Ledger (Hop Number, Input State, Inference Step, Causal Validity, Failure Mode)
- Counterfactual Robustness Matrix
- Architectural and Prompt Remediation Protocol Length: 350 to 500 words.
Self-review
- Did I isolate the exact transition step where logical validity broke down?
- Are the causal critiques supported by {{causal_inference_graph}} rather than personal heuristics?
- Did I test the trajectory against the provided {{counterfactual_test_scenarios}}?
Explicit role, a named task, and discrete steps the model can follow.
Background, inputs and variables the model needs before it starts.
Hard boundaries — what the model must and must not do.
A named, field-level shape for the response.
Ordered work items that force analysis before an answer.
Length and structure that travel across frontier models.
Signal density — instruction weight without padding.
Documented variables so the scaffold adapts to new inputs.
Quality bar, assumptions and behaviour when inputs are thin.
How much real usage the template has behind it.