mirror of
https://github.com/priyanshujain/messageboardbench.git
synced 2026-10-02 11:07:07 +00:00
Repo skeleton, pinned environment, free smoke test
Public from commit one, so no key ever enters this history. compose.yaml is ImpossibleBench's, plus working_dir: /workspace. The image has no WORKDIR, so inspect resolves it to "/" and the task files land at the filesystem root among twenty-odd entries. This experiment turns on whether an agent notices a scratch directory, so that is a bad place to put one. ImpossibleBench installs with --no-deps to keep the swebench tree out; datasets is declared here instead because hf_dataset genuinely needs it. Verified: docker run prints "/", impossiblebench imports, and the real task against mockllm/model completes with a real score and tracebacks rooted at /workspace. Claude-Session: https://claude.ai/code/session_01Cq98H7sNoSJdL3W98f18bu
This commit is contained in:
commit
bf3bb0d144
12 files changed
+2495
No files matched your search
@@ -0,0 +1,22 @@
|
||||
# Copied from impossiblebench/src/impossiblebench/compose.yaml (commit 061dc3d),
|
||||
# with working_dir added.
|
||||
#
|
||||
# The image has no WORKDIR, so inspect resolves the container working directory to
|
||||
# "/" and the task's func.py and test.py land at the filesystem root, mixed in with
|
||||
# the twenty-odd entries a bare `ls` returns there. This experiment turns on whether
|
||||
# an agent notices and uses a scratch directory, so burying it in / is a bad idea.
|
||||
# /workspace holds exactly func.py, test.py and scratch/. Docker creates the
|
||||
# directory on start; it does not exist in the image.
|
||||
#
|
||||
# This is a deviation from the published benchmark's default environment and belongs
|
||||
# in the paper's limitations. It applies identically to both conditions, so it cannot
|
||||
# produce the effect.
|
||||
services:
|
||||
default:
|
||||
image: "aisiuk/inspect-tool-support"
|
||||
command: "tail -f /dev/null"
|
||||
working_dir: /workspace
|
||||
init: true
|
||||
network_mode: none
|
||||
stop_grace_period: 1s
|
||||
mem_limit: 1gb
|
||||
Reference in new issue
Block a user