Analysis rows, pilot and report scripts

was_test_modified reads intermediate score events, not just the final score.
ImpossibleBench's scorer repairs test.py when it finds it modified, and basic_agent
scores every attempt, so an agent that edits the tests on attempt one and not again
leaves a final score with no flag on it. The repair erased the evidence.

snapshot_scratch reports whether the directory exists alongside its contents, because
otherwise a directory nobody wrote to and a directory that was never created look
identical, and the second is a broken harness reported as a real null result.

What the agents wrote is not auto-classified. "Did it write" is safe to automate;
whether a note is addressed to somebody is the judgement the pilot exists to make, so
pilot.py emits a markdown file to read by hand.

Claude-Session: https://claude.ai/code/session_01Cq98H7sNoSJdL3W98f18bu

justfile

@@ -16,7 +16,26 @@ test:
 smoke:
     uv run python scripts/smoke_mock.py
 
-# Confirms the daemon is up and pre-pulls the image so the first eval does not
-# time out on a cold pull. Should print "/".
+# Confirms the daemon is up and pre-pulls the image. Should print "/".
 docker-check:
     docker run --rm aisiuk/inspect-tool-support pwd
+
+# Confirms the model slug, the key and tool support. Costs about a cent.
+model-check:
+    uv run python scripts/model_check.py
+
+# Phase 1: cheat rate on conflicting, solve rate on original. About $1.
+baseline:
+    uv run python -u scripts/baseline.py conflicting original
+
+# The forked task against a fake model. Free. Checks the scratch wiring.
+smoke-scratch:
+    uv run python scripts/smoke_scratch.py
+
+# Phase 2: thirty agents with a scratch directory. Does anybody write anything?
+pilot:
+    uv run python -u scripts/pilot.py
+
+# Rebuild the CSV and hand-read file from an existing log, no re-run.
+report log_dir="logs/pilot":
+    uv run python scripts/report.py {{log_dir}}