Metamorphic Test Suite Script for Bayesian Inference Pipelines

Role: Lead Machine Learning QA Architect specializing in Probabilistic Systems and Statistical Inference.

Context

• Inference algorithm under test: {{inference_algorithm}}
• Metamorphic relations to test: {{metamorphic_relations}}
• Input distribution specification: {{input_distribution_spec}}
• Prior distribution rules: {{prior_distribution_rules}}
• Statistical significance threshold: {{drift_significance_level}}
• Target runtime platform: {{runtime_platform_runtime}}

Task

Author an end-to-end metamorphic testing and statistical assertion script in {{runtime_platform_runtime}} to detect algorithmic regression, numerical drift, and sampling distribution collapse in complex probabilistic models.

Method

1. Parse the expected behavior of {{inference_algorithm}} alongside its baseline distribution constraints.
2. Generate synthetic test datasets conforming to {{input_distribution_spec}} using deterministic pseudo-random seeds.
3. Apply metamorphic transformations defined in {{metamorphic_relations}} (e.g., affine scaling, label permutations, coordinate shifts).
4. Execute inference across base and transformed datasets, recording posterior parameter estimates.
5. Implement two-sample statistical hypothesis tests (e.g., Kolmogorov-Smirnov, Wasserstein distance) calibrated to {{drift_significance_level}}.
6. Validate that posterior outputs respect constraints from {{prior_distribution_rules}}.
7. Implement failure diagnostics that output quantile-quantile comparisons and divergence metrics when tests reject the null hypothesis.

Constraints

• The script MUST be written as a self-contained, runnable test suite for {{runtime_platform_runtime}}.
• Statistical assertions MUST NOT use fixed threshold comparisons for stochastic outputs; two-sample statistical tests are required.
• Execution MUST emit standardized test output with significance p-values and effect sizes.
• Every metamorphic relation MUST have an explicit deterministic baseline and transformation function.

Output format

Provide a complete executable script containing:

1. Dataset Synthesis and Metamorphic Transformation Helpers
2. Probabilistic Oracle and Statistical Test Assertion Classes
3. Parametrized Test Suite Execution Runner
4. Divergence Metric Logging and Structured Report Exporter

Self-review

• Confirm {{inference_algorithm}} output distributions are validated using valid statistical hypothesis tests.
• Verify {{metamorphic_relations}} transformations preserve expected mathematical invariants.
• Ensure execution does not depend on non-deterministic unseeded random states.