Automated Grading Engine Code Review and Hardening Plan
Plan a comprehensive code quality, security, and fairness review for automated student code evaluation systems.
Use this template when evaluating or upgrading an autograding codebase used in computer science or STEM courses. It provides a structured strategy to audit sandbox isolation, feedback reliability, and grading correctness.
Role: Lead Educational Technology Systems Architect with expertise in secure code evaluation engines.
Context
- Academic Institution: {{institution_name}}
- Educational Context: {{target_course}}
- Evaluation System: {{grading_engine_codebase}}
- Isolation Technology: {{execution_sandbox}}
- Concurrency Scale: {{student_cohort_size}}
- Service Level Target: {{grading_turnaround_target}}
Task
Produce an actionable code review and hardening plan for {{grading_engine_codebase}} to verify sandbox security, resource limits, deterministic scoring, and informative student error feedback under a peak load of {{student_cohort_size}} submissions.
Method
- Analyze {{grading_engine_codebase}} architecture for unsafe execution vectors and privilege escalation risks.
- Review container orchestration and process isolation rules within {{execution_sandbox}}.
- Define inspection points for CPU, memory, and wall-clock execution limits to guarantee {{grading_turnaround_target}}.
- Design test suites to verify that non-deterministic student submissions (e.g., infinite loops, fork bombs) are safely contained.
- Audit test harness scripts to ensure student feedback does not leak solution keys or private test data.
- Specify peer review protocols for instructor-written unit tests supporting {{target_course}}.
- Create a regression test strategy for scoring edge cases (partial credit, syntax errors, memory leaks).
Constraints
- Focus on system resilience, pedagogical integrity, and sandbox security.
- MUST explicitly evaluate escape risks associated with {{execution_sandbox}}.
- MUST NOT recommend manual grading fallbacks as a primary resolution for engine defects.
- Ensure recommendations handle high concurrent bursts during deadline windows.
- Balance strict security controls with constructive diagnostic feedback for students.
Output format
Format the response strictly with the following headers:
- Code Review Scope & Threat Model Summary (overview of assessment targets)
- Security & Sandbox Review Criteria (bulleted inspection points)
- Test Harness & Feedback Validation Protocol (numbered action items)
- Implementation & Rollout Roadmap (timeline and reviewer assignments)
Self-review
- Ensure all variables are integrated into the context and body.
- Verify that sandbox security and student feedback mechanisms receive distinct review steps.
- Confirm the structure matches the specified 4-section output format.
Explicit role, a named task, and discrete steps the model can follow.
Background, inputs and variables the model needs before it starts.
Hard boundaries — what the model must and must not do.
A named, field-level shape for the response.
Ordered work items that force analysis before an answer.
Length and structure that travel across frontier models.
Signal density — instruction weight without padding.
Documented variables so the scaffold adapts to new inputs.
Quality bar, assumptions and behaviour when inputs are thin.
How much real usage the template has behind it.