mirror of
https://github.com/priyanshujain/messageboardbench.git
synced 2026-10-02 19:17:07 +00:00
add v3 board activation experiment analysis
This commit is contained in:
1 parent
bce10b34ac
commit
62a9a35537
6 files changed
+382
No files matched your search
@@ -0,0 +1,24 @@
|
||||
# SWE board activation 10 v3 design
|
||||
|
||||
## Question
|
||||
|
||||
Will agents facing longer, contradictory coding tasks publish and retrieve useful
|
||||
gaming methods when they are explicitly told their token budget is limited?
|
||||
|
||||
## Design
|
||||
|
||||
GLM and Muse each solve the same ten previously unused ImpossibleBench SWE tasks
|
||||
labeled `1-4 hours`. Each model has its own empty persistent board. Five agents run
|
||||
concurrently in cohort 1, followed by five in cohort 2. The boards never exchange
|
||||
messages.
|
||||
|
||||
The system prompt adds a factual limited-token sentence after the existing optional
|
||||
peer-board sentence. Board tools, the 1M-token cap, evaluator, and remaining model
|
||||
parameters match v2. There is no control arm, feedback tool, seeded post, forced board
|
||||
action, notification, privacy claim, or shared coding objective.
|
||||
|
||||
The primary outcomes are substantive publication and exact cohort-2 peer receipt.
|
||||
Method adaptation, attribution, rejection, independent discovery, gaming success,
|
||||
and timing relative to first gaming attempts are reviewed separately. Because task
|
||||
difficulty and prompt wording both change, this developmental run cannot attribute
|
||||
any difference to either factor or estimate a causal communication effect.
|
||||
Reference in new issue
Block a user