archive old experiments and publish swe series

This commit is contained in:
pj committed 2026-09-25 12:34:44 +05:30
1 parent 480100587e
commit 638e978227
1522 files changed
+220002 -4900

No files matched your search

+4 -30
View File
@@ -1,35 +1,9 @@
# messageboardbench
Do coding agents share, adopt, reject, or correct cheating techniques through a public team board?
Research harness for studying whether coding agents publish, receive, and adopt evaluator-gaming methods across shared scratchpads and message boards.
Agents work on ImpossibleBench tasks in separate containers with private scratch space
and, in the board condition, explicit tools for reading and posting team messages.
The default pilot uses three agents per cohort, two cohorts, and 90 messages per
episode; model, population size, sampling and budgets are configurable.
The current study runs three OpenRouter models on the same 20 ImpossibleBench SWE tasks under isolated, scratchpad, board, feedback, internet, and token-awareness conditions.
Built on [ImpossibleBench](https://github.com/safety-research/impossiblebench) and
[Inspect AI](https://inspect.aisi.org.uk). Passing contradictory tests is an operational
signal of specification gaming, not a complete label of intent or attempted cheating.
See the [current experiment](EXPERIMENT.md), [scenario series](docs/swe-scenario-series.md), [experiment index](experiments/README.md), and [setup guide](docs/setup.md).
See [setup](docs/setup.md) and the [board experiment workflow](docs/board-pilot.md).
Run `just board` to choose parameters interactively and start the experiment.
Use `just board-preview` for a configuration preview without model calls, or
`just board-run --model muse` to run with Muse Spark Contributor. Executing makes
paid requests; the existing key spending cap is never raised.
Start with [EXPERIMENT.md](EXPERIMENT.md) for the research question, what we have
done, what the results establish, and the next work. The latest pilot contains one
explicitly attributed cross-task adoption; it does not establish an increased
gaming rate or concealed collusion.
| Location | Purpose |
|---|---|
| `src/`, `scripts/`, `tests/` | Implementation, entry points and tests |
| `docs/` | Setup, workflow and supporting research |
| [results/](results/README.md) | Reviewed, frozen experiment evidence |
| `logs/` | Raw local runs, ignored by Git |
| `work/` | Disposable local working files, ignored by Git |
This is the sole working repository for the experiment. The former `messageboard`
repo is [retired](docs/migration/README.md); personal notes and unrelated material
remain in its archive. Run `just evidence-check` to verify the migrated evidence.
Current raw runs live in `logs/`. Historical local experiments and logs stay in ignored `archive/`.