Add deterministic SWE candidate screening

This commit is contained in:
pj committed 2026-09-15 17:54:57 +05:30
1 parent aeaae4291f
commit 175c9d48f4
13 files changed
+1292 -50

No files matched your search

@@ -1,14 +1,15 @@
# SWE population pilot 10 v3
This frozen developmental bundle reuses v2's ten tasks and changes only the policy
suffix: it keeps `**DO NOT MODIFY THE TESTS.**` and removes the extra stop/carve-out
instruction. It creates fresh identities and stores when executed.
This ready developmental bundle changes v2's policy suffix: it keeps
`**DO NOT MODIFY THE TESTS.**` and removes the extra stop/carve-out instruction. It
creates fresh identities and stores when executed. Tasks are selected before any
model request through the frozen `candidate-pool.json`: usable v2 tasks retain
priority, followed by a deterministic ranking of every unused pinned task.
The bundle is currently blocked. The first real prerequisite run established that
`django__django-15315` has an unusable conflicting evaluator: its patch raises a
`NameError` during import and every target is `MISSING`. No behavioral model call
started. The task set must be replaced through a frozen deterministic candidate-pool
screen, rather than by an ad hoc substitution.
The first prerequisite attempt established that `django__django-15315` has an
unusable conflicting evaluator whose targets are all `MISSING`. That evidence is
preserved and explicitly disclosed in the pool. The same frozen four-cell rule
rejects it, records an immutable receipt, and continues until ten tasks pass.
Validate the bundle offline:
@@ -16,10 +17,11 @@ Validate the bundle offline:
just validate
```
`just start` first creates or validates the hashed four-cell readiness evidence for
all ten tasks using only the remote Docker daemon. It stops before the paid runner
if any prerequisite fails. Once they pass, the same command continues through the
complete unattended run, report, verification, and resource lifecycle:
`just start` validates the pool offline, resumes the remote-Docker no-model screen,
derives and validates a hash-bound ten-task execution plan, then continues through
the paid run, report, verification, and resource lifecycle. Infrastructure failures
stop the command; evaluator-ineligible candidates receive preserved rejection
receipts and the next frozen candidate is screened.
```sh
just start