pj/messageboardbench
1
0 Fork 0
mirror of https://github.com/priyanshujain/messageboardbench.git synced 2026-10-02 11:07:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

Keep communication diagnostics coding-relevant

Browse Source
This commit is contained in:
pj committed 2026-09-15 15:59:10 +05:30
1 parent 72d77018d8
commit b896955d3e
1 file changed
+7 -4
Download Patch File Download Diff File Expand all files Collapse all files
experiments/swe-population-pilot-10-v2/analysis.md
+7 -4
View File
@@ -159,7 +159,10 @@ peer exposure, only one board/team, severe differential limits, and six wholly
missing evaluator outcomes. missing evaluator outcomes.
Before another SWE causal run, fix environment completeness and fail-closed Before another SWE causal run, fix environment completeness and fail-closed
missing-target handling. Separately debug communication uptake on short neutral missing-target handling. Separately run a small communication-uptake diagnostic
tasks without Docker, scoring, contradiction, or feedback-channel competition. on coding tasks that retain the hard or contradictory evaluator pressure relevant
That diagnostic should manipulate board salience and compare models before we to the research question. It should vary board salience prospectively, compare
attribute non-use either to the prompt or to a stable model behavior. models, and separate spontaneous publication from receipt and adoption conditional
on exposure. A non-coding tool-compliance task could establish plumbing only; it
would not validate the target behavior. Keep organizer-feedback availability fixed
within the first prompt comparison, then test channel competition separately.