Algorithmic Inference Robustness Evaluation Grid

Role: Staff AI Safety Research Scientist specializing in statistical algorithm validation and adversarial robustness.

Context

• Algorithm Architecture: {{algorithm_family}}
• Theoretical Distribution Bounds: {{data_distribution_assumptions}}
• Noise Vectors & Adversarial Transforms: {{perturbation_vectors}}
• Critical Error Tolerance: {{error_tolerance_bounds}}
• Target Compute Ceiling: {{compute_constraints}}
• Reference Baselines: {{baseline_benchmarks}}

Task

Synthesize a rigorous Robustness Evaluation Matrix that systematically tests {{algorithm_family}} against synthetic perturbations, out-of-distribution drifts, and adversarial edge cases to detect mathematical instability, precision collapse, or inference failure.

Method

1. Formalize the baseline mathematical properties of {{algorithm_family}} against {{data_distribution_assumptions}}.
2. Parameterize each perturbation vector in {{perturbation_vectors}} by magnitude, distribution shift type, and injection layer.
3. Calibrate stress thresholds using {{error_tolerance_bounds}} to define boundary lines between acceptable drift and systemic failure.
4. Cross-tabulate each perturbation vector against algorithmic sub-components (loss calculation, gradient step, output projection).
5. Map expected behavioral anomalies, such as floating-point underflow, vanishing gradients, or non-convergent oscillations.
6. Compare projected degradation against {{baseline_benchmarks}} to isolate regressions introduced by algorithmic changes.
7. Evaluate compute overhead for each stress-test scenario ensuring compliance with {{compute_constraints}}.
8. Formulate empirical verification rules to automate pass/fail decisions during continuous benchmarking runs.

Constraints

• MUST quantify error bounds using precise statistical metrics (e.g., KL-divergence, L-infinity norm, Wasserstein distance).
• MUST NOT accept qualitative stability assessments without accompanying numerical threshold targets.
• The evaluation grid MUST test both deterministic boundary violations and stochastic noise perturbations.
• Every stress condition must specify the exact mathematical mechanism causing degradation.

Output format

Provide the evaluation plan in the following structured format:

1. Perturbation Taxonomy: A summary of attack vectors, statistical profiles, and test objectives.
2. Robustness Matrix: A structured grid with columns for Stress Test ID, Perturbation Vector, Parameter Range, Metric Evaluated, Failure Threshold, and Expected Baseline Comparison.
3. Automated Triage Protocol: A 4-step decision hierarchy for isolating numerical instability versus conceptual algorithm defects.

Self-review

• Ensure every item in {{perturbation_vectors}} is mapped to a row in the Robustness Matrix.
• Verify error thresholds do not exceed tolerances defined in {{error_tolerance_bounds}}.
• Confirm metric definitions are mathematically unambiguous.