Computer Science Auto-Grader Harness Review Spec

Role: Principal Instructional Technologist and Computer Science Course Architect.

Context

• Academic Unit: {{course_code}}
• Problem Focus: {{target_assignment_type}}
• Language Under Test: {{submission_language}}
• Isolation Framework: {{execution_sandbox_backend}}
• Integrity System: {{plagiarism_detection_tool}}
• Execution Limit: {{grading_timeout_seconds}} seconds

Task

Author a comprehensive code review specification to audit the automated assessment suite and grading harness, verifying test suite correctness, exploit resistance, and grading equity across cohort submissions.

Method

1. Map vulnerabilities where untrusted {{submission_language}} code could escape {{execution_sandbox_backend}} or tamper with grading fixtures.
2. Establish inspection rules for unit, property-based, and hidden edge-case tests tailored to {{target_assignment_type}}.
3. Define timeout handling standards ensuring execution halts at exactly {{grading_timeout_seconds}} without leaking zombie processes.
4. Create review standards for memory safety, thread leaks, and system-call filtering specific to {{submission_language}}.
5. Formulate score calculation audit steps, verifying partial-credit determinism and idempotent re-grading behavior.
6. Detail integration verification for automated hand-off to {{plagiarism_detection_tool}} without sanitization data loss.
7. Specify student feedback sanitization standards to prevent leakage of internal test harness code or private test inputs.

Constraints

• MUST treat all student submissions as untrusted and potentially malicious code.
• MUST NOT permit infinite loop vulnerability or unbounded memory consumption to crash the host runner.
• Specifications must account for grading variance caused by concurrency race conditions.
• Total document length must remain focused on operational test harness review criteria.

Output format

Format the specification with exact numbered sections:

• 1. Sandbox Isolation & Security Audit Spec (minimum 4 rigorous controls)
• 2. Test Suite Quality & Edge-Case Coverage Rules (minimum 5 criteria)
• 3. Resource Boundary & {{grading_timeout_seconds}}s Timeout Enforcement Spec
• 4. Output Sanitization & Feedback Privacy Protocols (3 criteria)
• 5. Plagiarism Pre-processing Inspection Gate (integration checklist for {{plagiarism_detection_tool}})

Self-review

• Ensure the spec specifically addresses the challenges of grading {{target_assignment_type}}.
• Check that isolation controls reflect the mechanics of {{execution_sandbox_backend}}.
• Verify all MUST/MUST NOT safety boundaries are concrete and enforceable.