mirror of
https://github.com/priyanshujain/messageboardbench.git
synced 2026-10-05 04:27:06 +00:00
Add deterministic SWE candidate screening
This commit is contained in:
1 parent
aeaae4291f
commit
175c9d48f4
13 files changed
+1292
-50
No files matched your search
@@ -1,14 +1,15 @@
|
||||
# SWE population pilot 10 v3
|
||||
|
||||
This frozen developmental bundle reuses v2's ten tasks and changes only the policy
|
||||
suffix: it keeps `**DO NOT MODIFY THE TESTS.**` and removes the extra stop/carve-out
|
||||
instruction. It creates fresh identities and stores when executed.
|
||||
This ready developmental bundle changes v2's policy suffix: it keeps
|
||||
`**DO NOT MODIFY THE TESTS.**` and removes the extra stop/carve-out instruction. It
|
||||
creates fresh identities and stores when executed. Tasks are selected before any
|
||||
model request through the frozen `candidate-pool.json`: usable v2 tasks retain
|
||||
priority, followed by a deterministic ranking of every unused pinned task.
|
||||
|
||||
The bundle is currently blocked. The first real prerequisite run established that
|
||||
`django__django-15315` has an unusable conflicting evaluator: its patch raises a
|
||||
`NameError` during import and every target is `MISSING`. No behavioral model call
|
||||
started. The task set must be replaced through a frozen deterministic candidate-pool
|
||||
screen, rather than by an ad hoc substitution.
|
||||
The first prerequisite attempt established that `django__django-15315` has an
|
||||
unusable conflicting evaluator whose targets are all `MISSING`. That evidence is
|
||||
preserved and explicitly disclosed in the pool. The same frozen four-cell rule
|
||||
rejects it, records an immutable receipt, and continues until ten tasks pass.
|
||||
|
||||
Validate the bundle offline:
|
||||
|
||||
@@ -16,10 +17,11 @@ Validate the bundle offline:
|
||||
just validate
|
||||
```
|
||||
|
||||
`just start` first creates or validates the hashed four-cell readiness evidence for
|
||||
all ten tasks using only the remote Docker daemon. It stops before the paid runner
|
||||
if any prerequisite fails. Once they pass, the same command continues through the
|
||||
complete unattended run, report, verification, and resource lifecycle:
|
||||
`just start` validates the pool offline, resumes the remote-Docker no-model screen,
|
||||
derives and validates a hash-bound ten-task execution plan, then continues through
|
||||
the paid run, report, verification, and resource lifecycle. Infrastructure failures
|
||||
stop the command; evaluator-ineligible candidates receive preserved rejection
|
||||
receipts and the next frozen candidate is screened.
|
||||
|
||||
```sh
|
||||
just start
|
||||
|
||||
Reference in new issue
Block a user