Experiment Coding Stage 6

This skill transforms a frozen research plan into reproducible, auditable experiment code with strict data-leak prevention. Rather than writing a working notebook, the goal is to implement the upstream frozen plan line-by-line into code that is falsifiable, re-runnable, and fully auditable. The core priorities are: preserve the research question unchanged; prevent evaluation information from leaking into training; ensure reproducibility with fixed environment and seed; retain raw evidence for downstream result analysis to recalculate; and keep code clean only insofar as it serves these constraints.

What it does

• Establishes a minimal runnable project using the provided scaffold: pyproject.toml, uv.lock, configuration schemas, data/model/metric interfaces, and reproducibility helpers.
• Implements test-first validation with gold tests, property tests, metamorphic tests, and train-only-fit tests before writing preprocessing, training, or evaluation logic.
• Enforces holdout and cross-validation leakage prevention: splits before fitting, uses sklearn Pipeline for grouped preprocessing per fold, and blocks data leakage across train/test boundaries.
• Controls randomness systematically by separating fixed-seed reproducibility runs from multi-seed randomness estimation, covering all RNG sources (Python hash seed, NumPy, framework, CUDA, cuDNN, deterministic algorithms).
• Preserves raw run bundles using light.run_manifest.v3: config snapshots, code commit/hash, data SHA256, environment details, stdout/stderr, per-fold metrics, entity-level predictions, and test evidence for every matrix row × config × seed × attempt.
• Applies gating checkpoints via experiment_execution_contract.py, review_gate.py, seed_audit.py, and repro_gate.py to verify scope, frozen budget, failure-tree binding, reproducibility level, and remote authorization before handoff to result analysis.

How to use it

Invoke this skill at the experiment-coding stage after the research plan is frozen. You provide the frozen research plan, experiment matrix, pre-registration, failure-tree report, data lineage with split IDs and leakage evidence, and result-analysis consumption contract. The skill reads and hashes all frozen inputs, then implements the plan into code that passes test-first validation, data-leak checks, reproducibility gates, and produces verifiable raw-run bundles with full provenance for downstream analysis.