Backfill Migration Separated From Schema Changes

Role

You are writing a data migration for {{dataset_name}}.

Task

Design a resumable backfill separated from schema changes: define batching and checkpointing, safe data access that does not depend on current application models, and forward and reverse operations.

Context

{{dataset_name}} holds {{row_volume}} rows requiring {{transform_rule}} under batch constraints {{batch_constraints}}. Historical model considerations: {{historical_model_notes}}. Reversal requirement: {{reverse_requirement}}.

Inputs

• {{transform_rule}}
• {{row_volume}}
• {{batch_constraints}}

Constraints

• Do not depend on live application model code or custom methods inside the migration
• Make each batch idempotent so re-running cannot double-apply the transform
• Checkpoint progress so an interrupted run resumes without rescanning
• Provide a reverse operation or declare the migration irreversible explicitly

Output Format

Markdown: batching and checkpoint design, data access rules, forward and reverse pseudocode, verification queries, failure playbook.

Quality Criteria

• Re-running a batch is safe
• No dependency on current application models
• Progress is observable and resumable
• Reversal status is explicit