experiment instrumentation: step budgets, arm labels, campaign runner (#72)

* feat(cli): add --max-steps for step-bounded runs

runner.Options.MaxSteps already worked but was unreachable from the command
line. A step budget is what makes two generators comparable: one making a
model call per step and one drawing from a PRNG are not comparable per second.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* feat(trace): record arm membership and host in meta.json

meta.json recorded the seed but not which picker ran, how it was configured,
what budget it was given, or which machine produced it. A directory of runs
cannot be attributed to an experiment cell without those, which makes any
factorial computed from such a directory unanalysable after the fact.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* feat(cli): add --arm and populate run meta from it

Model and instructions are recorded only when the LLM picker is the one that
will actually run, so a spec declaring generator = llm() that is run under the
seeded picker does not label its trace with a model it never called.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* feat(campaign): sweep seeds for one experiment cell

campaign.json lists the seeds a sweep intended to run and is written before
the first run, so a host that dropped runs shows up as missing seeds rather
than as a smaller sample. Seed 0 is rejected: sanderling test reads it as
"derive a seed from the clock", which is why conformance/gates.sh controls
nothing today.

Each run contributes one runs.jsonl line carrying steps to first violation by
origin step, the step that armed the failed obligation, so the survival
analysis never reopens a trace.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* fix(runner): no silent generator fallback, and llm on web

--generator llm against a spec declaring no generator = llm(...) logged a
warning and ran the seeded picker. For a comparison campaign that is silent
arm corruption: the run completes, the directory looks correct, and the wrong
policy drove it. It is now fatal.

pickSources also returned the V8 source for both action and extractor on web
before it looked at the generator, so the llm policy was unreachable there.
The two axes are now independent: the driver picks the extractor source, the
flag picks the action source, and llmSource composes with either because the
runner populates the candidate list and screenshot on every platform.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* fix(chrome): make the hierarchy dump agree with the web runtime

Three facts differed between the dump the goja host reads and the DOM the V8
host reads, so the two enumerated different candidates on one page.

scrollable was never emitted, and worker.go reads exactly that attribute while
targets.ts requires it for scrolls, so the goja host could not offer a single
web scroll. clickable tested el.onclick, which React assigns to its root
container for event delegation, making the whole viewport a tap target here and
in no other enumeration. Both now resolve through the selector sets in
pkg/spec/src/web-runtime.ts.

The dump also rooted at body while collectTargets walks querySelectorAll("*"),
so the goja host never saw html, where page-level scrolling lives. It now roots
at documentElement and skips the head subtree, which is all zero-bounds and
would otherwise carry script and title text into the trace.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* fix(conformance): give the gate reproducible seeds

SEED defaulted to 0 and sanderling test reads --seed 0 as "derive a seed from
the clock", so the tunable controlled nothing and a gate failure could not be
re-run. SEEDS now takes one explicit non-zero seed per run, recorded in the
results table so a failing row names its stream.

The five runs stay on five different streams: a gate that scored one path five
times would catch less than one that scores five.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* fix(chrome): emit editable as a plain boolean

editable was emitted as `isEditable || null`, and an absent field sends
internal/hierarchy into the native fallback, which reads any class name
containing "EditText" as an Android text widget. On web that is just a CSS
class, so a page styling a div with it was editable to the goja host and not to
the web runtime, and the model policy could be offered typing into a div.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* fix(spec): leave the head subtree out of the web target walk

collectTargets walked querySelectorAll("*") while the hierarchy dump skips head,
so the two hosts enumerated different element sets on every page with a <head>.
No candidate changes: builtinCandidates pushes only for targets acceptsTarget
admits, and head elements have no positive bounds, so the list the draw ranges
over is untouched. What changes is that targetIndex now means the same thing on
both hosts.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* test(chrome): compare the facts both hosts derive from one DOM

The existing parity harness hand-authors the facts on both sides, so it proves
that given identical facts both hosts select identical candidates, and says
nothing about the two code paths that derive those facts from a real page. Four
divergences lived in that blind spot and it passed throughout.

This drives one real page and compares clickable, enabled, editable, scrollable
and positiveBounds element by element, plus the element sets themselves, which
is what catches a host that omits html or includes head. Reverting any of the
four fixes makes it fail naming the element and the fact.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* chore(make): run the browser packages one at a time

Both launch Chrome and launching two at once has failed with "Launch: context
canceled".

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* style: remove every em-dash and en-dash

Eighteen occurrences across fourteen files. Each sentence was repunctuated to
suit what the dash was doing rather than swapped for a hyphen, which produces
comma splices. The minus sign in folio-web's ledger is a minus sign and stays.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* fix(chrome): honor the caller context in Launch

Launch and clearState ran against d.tabCtx, so a target that accepts the
connection and never answers wedged the process past its own --duration and
through SIGTERM, needing SIGKILL. Unattended that is a campaign worker lost for
the rest of the sweep with no diagnostic.

The browser is still allocated against d.tabCtx first, because chromedp starts
Chrome under whichever context calls Run first and allocating under a caller
deadline would kill the browser when Launch returns. Everything after
allocation goes through runCtx.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* fix(sidecarassets): publish the extracted jar through a rename

Extract wrote a 96 MB jar with a plain WriteFile into a temp path every
sanderling process on the host shares. On a cold host several concurrent
workers all miss the checksum and all write the same path, and O_TRUNC lets one
spawn a JVM against another's half-written archive. A fresh experiment host is
exactly a cold host.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* feat(campaign): kill a run that outlives --run-timeout

A wedged run holds its worker for the rest of the sweep, and on an unattended
host nothing else will send it a signal. Defaults to three times --duration and
must exceed it. A killed run is recorded as timed_out rather than as a generic
failure, so the analysis can tell a lost cell from a real crash.

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

* style(test): gofmt browser_test.go

Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX

CLAUDE.md

@@ -17,7 +17,7 @@
 
 ## PR Rules
 
-  - Simple PR title, few-line description. Never write a wall of text — nobody reads it.
+  - Simple PR title, few-line description. Never write a wall of text. Nobody reads it.
   - Everything lowercase in PR titles, descriptions, and comments.
 
 ### PR Description Rules

Makefile

@@ -110,9 +110,11 @@ web-test:
 	cd replay-ui && bun install --frozen-lockfile && bun test
 
 # Drives small web fixtures and the chrome driver through real headless Chrome.
-# Kept out of `test` because it needs a Chrome binary on PATH.
+# Kept out of `test` because it needs a Chrome binary on PATH. -p 1 runs the two
+# packages one after the other: both launch Chrome, and launching two at once
+# has failed with "Launch: context canceled".
 test-browser:
-	$(GO) test -tags browser ./test/browser/... ./internal/driver/chrome/...
+	$(GO) test -p 1 -tags browser ./test/browser/... ./internal/driver/chrome/...
 
 # Runs the withcompanion-tagged tests (asset embedding, extraction, checksum
 # reuse) against the real companion and runner bundles. Kept out of `test`

cmd/internal-tools/campaign/campaign.go

@@ -0,0 +1,227 @@
+package main
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"strconv"
+	"strings"
+	"sync"
+	"time"
+)
+
+// commandExecutor runs one sanderling invocation and returns its exit code.
+// A non-nil error means the process could not be run at all, which is a
+// different failure from a run that started and exited non-zero.
+type commandExecutor func(ctx context.Context, binary string, arguments []string, output io.Writer) (int, error)
+
+func executeCommand(ctx context.Context, binary string, arguments []string, output io.Writer) (int, error) {
+	command := exec.CommandContext(ctx, binary, arguments...)
+	command.Stdout = output
+	command.Stderr = output
+	err := command.Run()
+	if err == nil {
+		return 0, nil
+	}
+	var exitError *exec.ExitError
+	if errors.As(err, &exitError) {
+		return exitError.ExitCode(), nil
+	}
+	return -1, err
+}
+
+// runRecord is one line of runs.jsonl.
+type runRecord struct {
+	Seed           int64     `json:"seed"`
+	Device         string    `json:"device,omitempty"`
+	ExitCode       int       `json:"exit_code"`
+	LaunchError    string    `json:"launch_error,omitempty"`
+	TimedOut       bool      `json:"timed_out,omitempty"`
+	StartedAt      time.Time `json:"started_at"`
+	DurationMillis int64     `json:"duration_millis"`
+	RunDirectory   string    `json:"run_directory,omitempty"`
+	TraceError     string    `json:"trace_error,omitempty"`
+	traceSummary
+}
+
+type campaign struct {
+	configuration config
+	executor      commandExecutor
+	stdout        io.Writer
+	records       io.Writer
+	mutex         sync.Mutex
+	failures      int
+	unreadable    int
+}
+
+func runCampaign(ctx context.Context, configuration config, executor commandExecutor, stdout io.Writer) error {
+	if _, err := os.Stat(filepath.Join(configuration.outputDirectory, manifestFileName)); err == nil {
+		return fmt.Errorf("%s already exists in %s: pick a fresh --output so two campaigns do not share a directory",
+			manifestFileName, configuration.outputDirectory)
+	}
+	if err := os.MkdirAll(configuration.outputDirectory, 0o755); err != nil {
+		return fmt.Errorf("create campaign dir: %w", err)
+	}
+	binaryPath := configuration.sanderlingPath
+	if resolved, err := exec.LookPath(binaryPath); err == nil {
+		if absolute, err := filepath.Abs(resolved); err == nil {
+			binaryPath = absolute
+		}
+	}
+	version, err := probeVersion(ctx, configuration.sanderlingPath, executor)
+	if err != nil {
+		return err
+	}
+	host, _ := os.Hostname()
+	if err := writeManifest(configuration.outputDirectory, buildManifest(configuration, host, binaryPath, version, time.Now().UTC())); err != nil {
+		return fmt.Errorf("write %s: %w", manifestFileName, err)
+	}
+
+	recordsFile, err := os.OpenFile(filepath.Join(configuration.outputDirectory, recordsFileName),
+		os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o644)
+	if err != nil {
+		return fmt.Errorf("open %s: %w", recordsFileName, err)
+	}
+	defer recordsFile.Close()
+
+	sweep := &campaign{configuration: configuration, executor: executor, stdout: stdout, records: recordsFile}
+	fmt.Fprintf(stdout, "campaign %s: %d seeds, %d worker(s), %s\n",
+		configuration.arm, len(configuration.seeds), len(workerDevices(configuration.devices)), configuration.outputDirectory)
+	sweep.sweep(ctx)
+
+	fmt.Fprintf(stdout, "campaign complete: %d of %d runs failed, %d produced an unreadable trace\n",
+		sweep.failures, len(configuration.seeds), sweep.unreadable)
+	if sweep.failures > 0 {
+		return fmt.Errorf("%d of %d runs failed", sweep.failures, len(configuration.seeds))
+	}
+	if sweep.unreadable > 0 {
+		// A run that exits 0 and leaves a trace the analysis cannot read is a
+		// lost cell, not a successful campaign, and an unattended sweep has to
+		// say so rather than reporting no failures.
+		return fmt.Errorf("%d of %d runs produced an unreadable trace", sweep.unreadable, len(configuration.seeds))
+	}
+	return nil
+}
+
+func probeVersion(ctx context.Context, binary string, executor commandExecutor) (string, error) {
+	var output bytes.Buffer
+	code, err := executor(ctx, binary, []string{"version"}, &output)
+	if err != nil {
+		return "", fmt.Errorf("run %s version: %w", binary, err)
+	}
+	if code != 0 {
+		return "", fmt.Errorf("%s version exited %d: %s", binary, code, strings.TrimSpace(output.String()))
+	}
+	return strings.TrimSpace(output.String()), nil
+}
+
+// workerDevices returns one entry per concurrent worker. With no --devices
+// there is a single worker and no device to name.
+func workerDevices(devices []string) []string {
+	if len(devices) == 0 {
+		return []string{""}
+	}
+	return devices
+}
+
+func (c *campaign) sweep(ctx context.Context) {
+	queue := make(chan int64, len(c.configuration.seeds))
+	for _, seed := range c.configuration.seeds {
+		queue <- seed
+	}
+	close(queue)
+
+	var waitGroup sync.WaitGroup
+	for _, device := range workerDevices(c.configuration.devices) {
+		waitGroup.Add(1)
+		go func(device string) {
+			defer waitGroup.Done()
+			for seed := range queue {
+				if ctx.Err() != nil {
+					return
+				}
+				c.report(c.runSeed(ctx, seed, device))
+			}
+		}(device)
+	}
+	waitGroup.Wait()
+}
+
+func (c *campaign) runSeed(ctx context.Context, seed int64, device string) runRecord {
+	seedText := strconv.FormatInt(seed, 10)
+	directory := seedDirectory(c.configuration, seedText)
+	record := runRecord{Seed: seed, Device: device, StartedAt: time.Now().UTC()}
+
+	if err := os.MkdirAll(directory, 0o755); err != nil {
+		record.ExitCode = -1
+		record.LaunchError = err.Error()
+		return record
+	}
+	logFile, err := os.Create(filepath.Join(directory, "sanderling.log"))
+	if err != nil {
+		record.ExitCode = -1
+		record.LaunchError = err.Error()
+		return record
+	}
+	defer logFile.Close()
+
+	runCtx, cancelRun := context.WithTimeout(ctx, c.configuration.runTimeout)
+	defer cancelRun()
+	start := time.Now()
+	exitCode, runErr := c.executor(runCtx, c.configuration.sanderlingPath, runArguments(c.configuration, seedText, device), logFile)
+	if runCtx.Err() != nil && ctx.Err() == nil {
+		record.TimedOut = true
+	}
+	record.DurationMillis = time.Since(start).Milliseconds()
+	record.ExitCode = exitCode
+	if runErr != nil {
+		record.LaunchError = runErr.Error()
+	}
+
+	name, summary, err := summarizeRun(directory)
+	if name != "" {
+		record.RunDirectory = filepath.Join(filepath.Base(directory), name)
+	}
+	if err != nil {
+		record.TraceError = err.Error()
+		return record
+	}
+	record.traceSummary = summary
+	return record
+}
+
+func (c *campaign) report(record runRecord) {
+	c.mutex.Lock()
+	defer c.mutex.Unlock()
+	if record.ExitCode != 0 {
+		c.failures++
+	} else if record.TraceError != "" {
+		c.unreadable++
+	}
+	if err := json.NewEncoder(c.records).Encode(record); err != nil {
+		fmt.Fprintf(c.stdout, "warning: seed %d record: %v\n", record.Seed, err)
+	}
+	fmt.Fprintf(c.stdout, "seed=%d device=%q outcome=%s steps=%d exit=%d duration=%s\n",
+		record.Seed, record.Device, outcome(record), record.Steps, record.ExitCode,
+		time.Duration(record.DurationMillis)*time.Millisecond)
+}
+
+func outcome(record runRecord) string {
+	switch {
+	case record.TimedOut:
+		return "timed out"
+	case record.ExitCode != 0:
+		return "failed"
+	case record.FirstViolationOriginStep != nil:
+		return fmt.Sprintf("violation@%d(%s)", *record.FirstViolationOriginStep,
+			strings.Join(record.FirstViolationProperties, ","))
+	default:
+		return "clean"
+	}
+}

cmd/internal-tools/campaign/campaign_test.go

@@ -0,0 +1,362 @@
+package main
+
+import (
+	"bufio"
+	"bytes"
+	"context"
+	"encoding/json"
+	"fmt"
+	"io"
+	"os"
+	"path/filepath"
+	"slices"
+	"strconv"
+	"strings"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"github.com/priyanshujain/sanderling/internal/trace"
+)
+
+func testConfiguration(t *testing.T, outputDirectory string, extra ...string) config {
+	t.Helper()
+	arguments := append(baseArguments(), "--output", outputDirectory)
+	configuration, err := parseArguments(append(arguments, extra...), io.Discard)
+	if err != nil {
+		t.Fatal(err)
+	}
+	return configuration
+}
+
+// versionAnswering wraps a test executor so every fake answers `sanderling
+// version`, which the campaign probes before it writes the manifest.
+func versionAnswering(executor commandExecutor) commandExecutor {
+	return func(ctx context.Context, binary string, arguments []string, output io.Writer) (int, error) {
+		if len(arguments) > 0 && arguments[0] == "version" {
+			fmt.Fprintln(output, "stub-version")
+			return 0, nil
+		}
+		return executor(ctx, binary, arguments, output)
+	}
+}
+
+func readRecords(t *testing.T, campaignDirectory string) []runRecord {
+	t.Helper()
+	file, err := os.Open(filepath.Join(campaignDirectory, recordsFileName))
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer file.Close()
+	var records []runRecord
+	scanner := bufio.NewScanner(file)
+	for scanner.Scan() {
+		var record runRecord
+		if err := json.Unmarshal(scanner.Bytes(), &record); err != nil {
+			t.Fatalf("decode %q: %v", scanner.Text(), err)
+		}
+		records = append(records, record)
+	}
+	if err := scanner.Err(); err != nil {
+		t.Fatal(err)
+	}
+	return records
+}
+
+func writeFakeRun(t *testing.T, arguments []string, steps []trace.Step) {
+	t.Helper()
+	writeRunDirectory(t, argumentValue(arguments, "--output"), "20260101-000000", steps)
+}
+
+func TestRunCampaign_WritesManifestBeforeAnyRun(t *testing.T) {
+	directory := t.TempDir()
+	configuration := testConfiguration(t, directory)
+	var stdout bytes.Buffer
+
+	executor := versionAnswering(func(_ context.Context, _ string, arguments []string, _ io.Writer) (int, error) {
+		if _, err := os.Stat(filepath.Join(directory, manifestFileName)); err != nil {
+			t.Errorf("manifest missing when the first run started: %v", err)
+		}
+		writeFakeRun(t, arguments, []trace.Step{observedStep(1)})
+		return 0, nil
+	})
+	if err := runCampaign(context.Background(), configuration, executor, &stdout); err != nil {
+		t.Fatal(err)
+	}
+
+	body, err := os.ReadFile(filepath.Join(directory, manifestFileName))
+	if err != nil {
+		t.Fatal(err)
+	}
+	var recorded manifest
+	if err := json.Unmarshal(body, &recorded); err != nil {
+		t.Fatal(err)
+	}
+	if !slices.Equal(recorded.Seeds, []int64{1, 2, 3}) {
+		t.Errorf("intended seeds: got %v", recorded.Seeds)
+	}
+	if recorded.SanderlingVersion != "stub-version" {
+		t.Errorf("version: got %q", recorded.SanderlingVersion)
+	}
+	if recorded.Host == "" {
+		t.Error("host was not recorded")
+	}
+}
+
+func TestRunCampaign_RecordsPerRunSummary(t *testing.T) {
+	directory := t.TempDir()
+	configuration := testConfiguration(t, directory, "--seeds", "1-3")
+	var stdout bytes.Buffer
+
+	executor := versionAnswering(func(_ context.Context, _ string, arguments []string, output io.Writer) (int, error) {
+		fmt.Fprintln(output, "stub run log")
+		steps := []trace.Step{observedStep(1), observedStep(2), observedStep(3)}
+		if argumentValue(arguments, "--seed") == "2" {
+			violating := observedStep(3)
+			violating.Violations = []string{"listNeverEmpty"}
+			violating.Witnesses = map[string]trace.Witness{
+				"listNeverEmpty": {Reason: "list emptied", Step: 2, DetectedStep: 3},
+			}
+			steps[2] = violating
+		}
+		writeFakeRun(t, arguments, steps)
+		return 0, nil
+	})
+	if err := runCampaign(context.Background(), configuration, executor, &stdout); err != nil {
+		t.Fatal(err)
+	}
+
+	records := readRecords(t, directory)
+	if len(records) != 3 {
+		t.Fatalf("records: got %d, want 3", len(records))
+	}
+	for _, record := range records {
+		if record.Steps != 3 {
+			t.Errorf("seed %d steps: got %d, want 3", record.Seed, record.Steps)
+		}
+		if record.RunDirectory != fmt.Sprintf("seed-%d/20260101-000000", record.Seed) {
+			t.Errorf("seed %d run directory: got %q", record.Seed, record.RunDirectory)
+		}
+		if record.TraceError != "" {
+			t.Errorf("seed %d trace error: %s", record.Seed, record.TraceError)
+		}
+		violated := record.FirstViolationOriginStep != nil
+		if violated != (record.Seed == 2) {
+			t.Errorf("seed %d violation: got %v", record.Seed, record.FirstViolationOriginStep)
+		}
+		if record.Seed != 2 {
+			continue
+		}
+		if *record.FirstViolationOriginStep != 2 || *record.FirstViolationDetectedStep != 3 {
+			t.Errorf("seed 2 violation steps: origin %d detected %d",
+				*record.FirstViolationOriginStep, *record.FirstViolationDetectedStep)
+		}
+		if !slices.Equal(record.ViolatedProperties, []string{"listNeverEmpty"}) {
+			t.Errorf("seed 2 properties: got %v", record.ViolatedProperties)
+		}
+	}
+
+	log, err := os.ReadFile(filepath.Join(directory, "seed-1", "sanderling.log"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !strings.Contains(string(log), "stub run log") {
+		t.Errorf("per-run log: got %q", log)
+	}
+	if !strings.Contains(stdout.String(), "seed=2") {
+		t.Errorf("progress output missing seed 2: %q", stdout.String())
+	}
+}
+
+func TestRunCampaign_DistributesSeedsAcrossDeviceWorkers(t *testing.T) {
+	directory := t.TempDir()
+	configuration := testConfiguration(t, directory, "--seeds", "1-9", "--devices", "device-a,device-b,device-c")
+
+	var mutex sync.Mutex
+	assignments := map[int64]string{}
+	var inFlight atomic.Int32
+	var releaseOnce sync.Once
+	var timedOut atomic.Bool
+	release := make(chan struct{})
+
+	executor := versionAnswering(func(_ context.Context, _ string, arguments []string, _ io.Writer) (int, error) {
+		if inFlight.Add(1) >= 3 {
+			releaseOnce.Do(func() { close(release) })
+		}
+		select {
+		case <-release:
+		case <-time.After(5 * time.Second):
+			timedOut.Store(true)
+		}
+		seed, err := strconv.ParseInt(argumentValue(arguments, "--seed"), 10, 64)
+		if err != nil {
+			t.Errorf("seed argument: %v", err)
+		}
+		mutex.Lock()
+		if previous, seen := assignments[seed]; seen {
+			t.Errorf("seed %d ran twice (%s then %s)", seed, previous, argumentValue(arguments, "--device"))
+		}
+		assignments[seed] = argumentValue(arguments, "--device")
+		mutex.Unlock()
+		writeFakeRun(t, arguments, []trace.Step{observedStep(1)})
+		return 0, nil
+	})
+	if err := runCampaign(context.Background(), configuration, executor, io.Discard); err != nil {
+		t.Fatal(err)
+	}
+	if timedOut.Load() {
+		t.Fatal("three device workers never ran concurrently")
+	}
+	if len(assignments) != 9 {
+		t.Fatalf("seeds run: got %d, want 9", len(assignments))
+	}
+	used := map[string]int{}
+	for seed, device := range assignments {
+		if !slices.Contains(configuration.devices, device) {
+			t.Errorf("seed %d ran on unknown device %q", seed, device)
+		}
+		used[device]++
+	}
+	if len(used) != 3 {
+		t.Errorf("devices used: got %v, want all three", used)
+	}
+	if got := len(readRecords(t, directory)); got != 9 {
+		t.Errorf("records: got %d, want 9", got)
+	}
+}
+
+func TestRunCampaign_ContinuesAfterFailingRun(t *testing.T) {
+	directory := t.TempDir()
+	configuration := testConfiguration(t, directory, "--seeds", "1-3")
+
+	executor := versionAnswering(func(_ context.Context, _ string, arguments []string, output io.Writer) (int, error) {
+		if argumentValue(arguments, "--seed") == "2" {
+			fmt.Fprintln(output, "error: device offline")
+			return 1, nil
+		}
+		writeFakeRun(t, arguments, []trace.Step{observedStep(1)})
+		return 0, nil
+	})
+	err := runCampaign(context.Background(), configuration, executor, io.Discard)
+	if err == nil || !strings.Contains(err.Error(), "1 of 3 runs failed") {
+		t.Fatalf("expected a failure summary, got %v", err)
+	}
+
+	records := readRecords(t, directory)
+	if len(records) != 3 {
+		t.Fatalf("a failing run must not abort the campaign: got %d records", len(records))
+	}
+	for _, record := range records {
+		if record.Seed == 2 {
+			if record.ExitCode != 1 {
+				t.Errorf("seed 2 exit code: got %d, want 1", record.ExitCode)
+			}
+			if record.TraceError == "" {
+				t.Error("seed 2 produced no trace; that should be recorded")
+			}
+			continue
+		}
+		if record.ExitCode != 0 {
+			t.Errorf("seed %d exit code: got %d", record.Seed, record.ExitCode)
+		}
+	}
+}
+
+func TestRunCampaign_RefusesToReuseACampaignDirectory(t *testing.T) {
+	directory := t.TempDir()
+	configuration := testConfiguration(t, directory)
+	if err := os.WriteFile(filepath.Join(directory, manifestFileName), []byte("{}"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	executor := versionAnswering(func(context.Context, string, []string, io.Writer) (int, error) {
+		t.Error("no run should start in a directory that already holds a campaign")
+		return 0, nil
+	})
+	if err := runCampaign(context.Background(), configuration, executor, io.Discard); err == nil {
+		t.Fatal("expected a refusal to reuse the campaign directory")
+	}
+}
+
+func TestRunCampaign_AbortsWhenVersionProbeFails(t *testing.T) {
+	directory := t.TempDir()
+	configuration := testConfiguration(t, directory)
+	executor := func(_ context.Context, _ string, arguments []string, output io.Writer) (int, error) {
+		if arguments[0] != "version" {
+			t.Error("a run started despite an unusable binary")
+		}
+		fmt.Fprintln(output, "no such command")
+		return 2, nil
+	}
+	if err := runCampaign(context.Background(), configuration, executor, io.Discard); err == nil {
+		t.Fatal("expected the campaign to abort before writing a manifest it cannot attribute")
+	}
+	if _, err := os.Stat(filepath.Join(directory, manifestFileName)); err == nil {
+		t.Error("manifest was written despite an unusable binary")
+	}
+}
+
+func TestRunCampaign_UnreadableTraceIsNotASuccessfulCampaign(t *testing.T) {
+	campaignDirectory := filepath.Join(t.TempDir(), "cell")
+	configuration := testConfiguration(t, campaignDirectory, "--seeds", "1-2")
+
+	executor := versionAnswering(func(context.Context, string, []string, io.Writer) (int, error) {
+		return 0, nil
+	})
+
+	var stdout bytes.Buffer
+	err := runCampaign(context.Background(), configuration, executor, &stdout)
+	if err == nil || !strings.Contains(err.Error(), "2 of 2 runs produced an unreadable trace") {
+		t.Fatalf("a campaign whose runs left no readable trace must not report success: %v", err)
+	}
+	for _, record := range readRecords(t, campaignDirectory) {
+		if record.TraceError == "" {
+			t.Errorf("seed %d: expected a trace error, got none", record.Seed)
+		}
+	}
+}
+
+func TestRunCampaign_KillsAndRecordsAWedgedRun(t *testing.T) {
+	campaignDirectory := filepath.Join(t.TempDir(), "cell")
+	configuration := testConfiguration(t, campaignDirectory,
+		"--seeds", "1", "--duration", "50ms", "--run-timeout", "300ms")
+
+	executor := versionAnswering(func(ctx context.Context, _ string, _ []string, _ io.Writer) (int, error) {
+		<-ctx.Done()
+		return -1, ctx.Err()
+	})
+
+	var stdout bytes.Buffer
+	if err := runCampaign(context.Background(), configuration, executor, &stdout); err == nil {
+		t.Fatal("a campaign whose only run was killed must not report success")
+	}
+	records := readRecords(t, campaignDirectory)
+	if len(records) != 1 {
+		t.Fatalf("want one record, got %d", len(records))
+	}
+	if !records[0].TimedOut {
+		t.Error("a run killed by the run timeout must be recorded as timed out")
+	}
+	if !strings.Contains(stdout.String(), "timed out") {
+		t.Errorf("the progress line must name the outcome:\n%s", stdout.String())
+	}
+}
+
+func TestParseArguments_RunTimeoutMustExceedDuration(t *testing.T) {
+	_, err := parseArguments(append(baseArguments(),
+		"--output", t.TempDir(), "--duration", "5m", "--run-timeout", "1m"), io.Discard)
+	if err == nil {
+		t.Fatal("a run timeout below the duration would kill every run before it finished")
+	}
+}
+
+func TestParseArguments_RunTimeoutDefaultsToThreeTimesDuration(t *testing.T) {
+	configuration, err := parseArguments(append(baseArguments(),
+		"--output", t.TempDir(), "--duration", "4m"), io.Discard)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if configuration.runTimeout != 12*time.Minute {
+		t.Errorf("run timeout default: got %s, want 12m", configuration.runTimeout)
+	}
+}

cmd/internal-tools/campaign/end_to_end_test.go

@@ -0,0 +1,144 @@
+package main
+
+import (
+	"bytes"
+	"encoding/json"
+	"io"
+	"os"
+	"path/filepath"
+	"slices"
+	"strings"
+	"testing"
+)
+
+// stubSanderling answers `version`, writes a run directory shaped like the one
+// `sanderling test` produces, violates on seed 5 and fails on seed 7.
+const stubSanderling = `#!/bin/sh
+if [ "$1" = "version" ]; then
+  echo "stub 9.9.9"
+  exit 0
+fi
+seed=""
+output=""
+while [ $# -gt 0 ]; do
+  case "$1" in
+    --seed) seed="$2"; shift 2 ;;
+    --output) output="$2"; shift 2 ;;
+    *) shift ;;
+  esac
+done
+echo "stub run seed=$seed"
+run="$output/20260101-000000"
+mkdir -p "$run"
+echo "{\"seed\":$seed,\"platform\":\"web\"}" > "$run/meta.json"
+{
+  echo '{"step":1,"hierarchy":{"elements":[]}}'
+  echo '{"step":2,"hierarchy":{"elements":[]}}'
+  if [ "$seed" = "5" ]; then
+    echo '{"step":3,"hierarchy":{"elements":[]},"violations":["cartTotalMatches"],"witnesses":{"cartTotalMatches":{"reason":"total drifted","step":2,"detected_step":3}}}'
+  else
+    echo '{"step":3,"hierarchy":{"elements":[]}}'
+  fi
+} > "$run/trace.jsonl"
+if [ "$seed" = "7" ]; then
+  echo "driver crashed" >&2
+  exit 3
+fi
+`
+
+func TestRun_EndToEndAgainstStubBinary(t *testing.T) {
+	directory := t.TempDir()
+	binaryPath := filepath.Join(directory, "stub-sanderling")
+	if err := os.WriteFile(binaryPath, []byte(stubSanderling), 0o755); err != nil {
+		t.Fatal(err)
+	}
+	campaignDirectory := filepath.Join(directory, "campaign")
+	var stdout bytes.Buffer
+
+	err := run([]string{
+		"--spec", "/specs/folio.ts",
+		"--bundle-id", "app.folio",
+		"--platform", "web",
+		"--arm", "seeded-web",
+		"--generator", "seeded",
+		"--max-steps", "50",
+		"--duration", "30s",
+		"--seeds", "4-5,7",
+		"--devices", "worker-a,worker-b",
+		"--sanderling", binaryPath,
+		"--output", campaignDirectory,
+		"--", "--clear-data=false",
+	}, &stdout, io.Discard)
+	if err == nil || !strings.Contains(err.Error(), "1 of 3 runs failed") {
+		t.Fatalf("expected the failing seed to be reported, got %v", err)
+	}
+
+	body, err := os.ReadFile(filepath.Join(campaignDirectory, manifestFileName))
+	if err != nil {
+		t.Fatal(err)
+	}
+	var recorded manifest
+	if err := json.Unmarshal(body, &recorded); err != nil {
+		t.Fatal(err)
+	}
+	if recorded.SanderlingVersion != "stub 9.9.9" {
+		t.Errorf("version: got %q", recorded.SanderlingVersion)
+	}
+	if !slices.Equal(recorded.Seeds, []int64{4, 5, 7}) {
+		t.Errorf("intended seeds: got %v", recorded.Seeds)
+	}
+	if slices.Contains(recorded.ArgumentTemplate, "--device") {
+		t.Errorf("web template should carry no device flag: %v", recorded.ArgumentTemplate)
+	}
+	if recorded.ArgumentTemplate[len(recorded.ArgumentTemplate)-1] != "--clear-data=false" {
+		t.Errorf("template lost the passthrough flag: %v", recorded.ArgumentTemplate)
+	}
+
+	records := readRecords(t, campaignDirectory)
+	if len(records) != 3 {
+		t.Fatalf("records: got %d, want 3", len(records))
+	}
+	bySeed := map[int64]runRecord{}
+	for _, record := range records {
+		bySeed[record.Seed] = record
+	}
+	for _, seed := range []int64{4, 5, 7} {
+		record, ok := bySeed[seed]
+		if !ok {
+			t.Fatalf("seed %d is missing from %s", seed, recordsFileName)
+		}
+		if record.Steps != 3 {
+			t.Errorf("seed %d steps: got %d, want 3", seed, record.Steps)
+		}
+		if record.RunDirectory == "" {
+			t.Errorf("seed %d has no run directory", seed)
+		}
+		if !slices.Contains([]string{"worker-a", "worker-b"}, record.Device) {
+			t.Errorf("seed %d device: got %q", seed, record.Device)
+		}
+	}
+	violating := bySeed[5]
+	if violating.FirstViolationOriginStep == nil || *violating.FirstViolationOriginStep != 2 {
+		t.Fatalf("seed 5 origin step: got %v, want 2", violating.FirstViolationOriginStep)
+	}
+	if *violating.FirstViolationDetectedStep != 3 || violating.FirstViolationReason != "total drifted" {
+		t.Errorf("seed 5 violation: %+v", violating)
+	}
+	if bySeed[4].FirstViolationOriginStep != nil {
+		t.Errorf("seed 4 should be censored at the budget: %+v", bySeed[4])
+	}
+	if bySeed[7].ExitCode != 3 {
+		t.Errorf("seed 7 exit code: got %d, want 3", bySeed[7].ExitCode)
+	}
+
+	log, err := os.ReadFile(filepath.Join(campaignDirectory, "seed-7", "sanderling.log"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !strings.Contains(string(log), "driver crashed") {
+		t.Errorf("seed 7 log lost the stderr line: %q", log)
+	}
+	if !strings.Contains(stdout.String(), "outcome=violation@2") {
+		t.Errorf("progress output: %q", stdout.String())
+	}
+}

cmd/internal-tools/campaign/main.go

@@ -0,0 +1,202 @@
+// Command campaign sweeps a list of seeds for one experiment cell, writing a
+// directory an analysis pipeline can read without re-parsing raw traces.
+package main
+
+import (
+	"context"
+	"errors"
+	"flag"
+	"fmt"
+	"io"
+	"os"
+	"os/signal"
+	"path/filepath"
+	"strconv"
+	"strings"
+	"syscall"
+	"time"
+)
+
+type config struct {
+	specPath        string
+	bundleID        string
+	platform        string
+	arm             string
+	generator       string
+	maxSteps        int
+	duration        time.Duration
+	seeds           []int64
+	devices         []string
+	sanderlingPath  string
+	outputDirectory string
+	runTimeout      time.Duration
+	extraArguments  []string
+}
+
+const usage = `campaign sweeps seeds for one experiment cell of a sanderling evaluation.
+
+Usage:
+  campaign --spec <path> --bundle-id <id> --platform <android|ios|web>
+           --arm <label> --generator <seeded|llm> --max-steps <n>
+           --seeds <spec> --output <dir> [flags] [-- <sanderling test flags>]
+
+Everything after a bare -- is appended verbatim to every sanderling test call.
+`
+
+func parseArguments(arguments []string, stderr io.Writer) (config, error) {
+	flagSet := flag.NewFlagSet("campaign", flag.ContinueOnError)
+	flagSet.SetOutput(stderr)
+	flagSet.Usage = func() {
+		fmt.Fprint(stderr, usage)
+		flagSet.PrintDefaults()
+	}
+	var configuration config
+	var seedSpecification string
+	var deviceList string
+	flagSet.StringVar(&configuration.specPath, "spec", "", "path to the TypeScript spec (required)")
+	flagSet.StringVar(&configuration.bundleID, "bundle-id", "", "target app bundle ID (required)")
+	flagSet.StringVar(&configuration.platform, "platform", "android", "target platform: android, ios, web")
+	flagSet.StringVar(&configuration.arm, "arm", "", "experiment cell label recorded on every run (required)")
+	flagSet.StringVar(&configuration.generator, "generator", "seeded", "action generator: seeded or llm")
+	flagSet.IntVar(&configuration.maxSteps, "max-steps", 0, "per-run step budget (required, must be positive)")
+	flagSet.DurationVar(&configuration.duration, "duration", 5*time.Minute, "per-run wall-clock ceiling")
+	flagSet.StringVar(&seedSpecification, "seeds", "", "seeds to run: ranges and lists, e.g. 1-10,20,30-32 (required)")
+	flagSet.StringVar(&deviceList, "devices", "", "comma-separated device identifiers; one concurrent worker per device (on web these are worker labels, no device flag is passed)")
+	flagSet.DurationVar(&configuration.runTimeout, "run-timeout", 0, "kill a run that outlives this (default: three times --duration). A wedged run holds its worker for the rest of the sweep, and nothing else will send it a signal on an unattended host")
+	flagSet.StringVar(&configuration.sanderlingPath, "sanderling", "sanderling", "sanderling binary to invoke")
+	flagSet.StringVar(&configuration.outputDirectory, "output", "", "campaign directory to create (required)")
+	if err := flagSet.Parse(arguments); err != nil {
+		return config{}, err
+	}
+	configuration.extraArguments = flagSet.Args()
+
+	for name, value := range map[string]string{
+		"--spec":      configuration.specPath,
+		"--bundle-id": configuration.bundleID,
+		"--arm":       configuration.arm,
+		"--seeds":     seedSpecification,
+		"--output":    configuration.outputDirectory,
+	} {
+		if value == "" {
+			return config{}, fmt.Errorf("%s is required", name)
+		}
+	}
+	switch configuration.platform {
+	case "android", "ios", "web":
+	default:
+		return config{}, fmt.Errorf("unsupported platform: %q (android, ios, web)", configuration.platform)
+	}
+	switch configuration.generator {
+	case "seeded", "llm":
+	default:
+		return config{}, fmt.Errorf("unsupported generator: %q (seeded, llm)", configuration.generator)
+	}
+	if configuration.maxSteps <= 0 {
+		// Steps to first violation is right-censored at the budget, so a
+		// campaign without one has nothing to censor its clean runs at.
+		return config{}, fmt.Errorf("--max-steps must be positive: every run needs the same step budget")
+	}
+	if configuration.duration <= 0 {
+		return config{}, fmt.Errorf("--duration must be positive: %s", configuration.duration)
+	}
+	if configuration.runTimeout < 0 {
+		return config{}, fmt.Errorf("--run-timeout must not be negative: %s", configuration.runTimeout)
+	}
+	if configuration.runTimeout == 0 {
+		configuration.runTimeout = 3 * configuration.duration
+	}
+	if configuration.runTimeout <= configuration.duration {
+		return config{}, fmt.Errorf("--run-timeout %s must exceed --duration %s, or every run is killed before it finishes",
+			configuration.runTimeout, configuration.duration)
+	}
+	seeds, err := parseSeeds(seedSpecification)
+	if err != nil {
+		return config{}, fmt.Errorf("--seeds: %w", err)
+	}
+	configuration.seeds = seeds
+	devices, err := parseDevices(deviceList)
+	if err != nil {
+		return config{}, fmt.Errorf("--devices: %w", err)
+	}
+	configuration.devices = devices
+	return configuration, nil
+}
+
+func parseDevices(list string) ([]string, error) {
+	if strings.TrimSpace(list) == "" {
+		return nil, nil
+	}
+	var devices []string
+	seen := map[string]bool{}
+	for _, part := range strings.Split(list, ",") {
+		device := strings.TrimSpace(part)
+		if device == "" {
+			return nil, fmt.Errorf("empty device in %q", list)
+		}
+		if seen[device] {
+			return nil, fmt.Errorf("duplicate device %q", device)
+		}
+		seen[device] = true
+		devices = append(devices, device)
+	}
+	return devices, nil
+}
+
+// deviceFlag names the `sanderling test` flag that selects a target on this
+// platform. Web has no device, so its workers only bound concurrency.
+func deviceFlag(platform string) string {
+	switch platform {
+	case "android":
+		return "--device"
+	case "ios":
+		return "--ios-device"
+	default:
+		return ""
+	}
+}
+
+func seedDirectory(configuration config, seed string) string {
+	return filepath.Join(configuration.outputDirectory, "seed-"+seed)
+}
+
+// runArguments builds one `sanderling test` invocation. seed and device are
+// passed as strings so the same code produces both a real command and the
+// placeholder template recorded in campaign.json.
+func runArguments(configuration config, seed, device string) []string {
+	arguments := []string{
+		"test",
+		"--spec", configuration.specPath,
+		"--bundle-id", configuration.bundleID,
+		"--platform", configuration.platform,
+		"--arm", configuration.arm,
+		"--generator", configuration.generator,
+		"--max-steps", strconv.Itoa(configuration.maxSteps),
+		"--duration", configuration.duration.String(),
+		"--seed", seed,
+		"--output", seedDirectory(configuration, seed),
+	}
+	if flagName := deviceFlag(configuration.platform); flagName != "" && device != "" {
+		arguments = append(arguments, flagName, device)
+	}
+	return append(arguments, configuration.extraArguments...)
+}
+
+func run(arguments []string, stdout, stderr io.Writer) error {
+	configuration, err := parseArguments(arguments, stderr)
+	if err != nil {
+		return err
+	}
+	ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
+	defer cancel()
+	return runCampaign(ctx, configuration, executeCommand, stdout)
+}
+
+func main() {
+	if err := run(os.Args[1:], os.Stdout, os.Stderr); err != nil {
+		if errors.Is(err, flag.ErrHelp) {
+			return
+		}
+		fmt.Fprintf(os.Stderr, "error: %v\n", err)
+		os.Exit(1)
+	}
+}

cmd/internal-tools/campaign/main_test.go

@@ -0,0 +1,122 @@
+package main
+
+import (
+	"io"
+	"slices"
+	"strings"
+	"testing"
+	"time"
+)
+
+func baseArguments() []string {
+	return []string{
+		"--spec", "/specs/folio.ts",
+		"--bundle-id", "app.folio",
+		"--platform", "android",
+		"--arm", "seeded-baseline",
+		"--generator", "seeded",
+		"--max-steps", "300",
+		"--seeds", "1-3",
+		"--output", "/campaigns/a",
+	}
+}
+
+func TestParseArguments_AcceptsFullInvocation(t *testing.T) {
+	configuration, err := parseArguments(append(baseArguments(), "--devices", "emulator-5554,emulator-5556"), io.Discard)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if configuration.arm != "seeded-baseline" || configuration.maxSteps != 300 {
+		t.Errorf("arm/max-steps: got %q/%d", configuration.arm, configuration.maxSteps)
+	}
+	if !slices.Equal(configuration.seeds, []int64{1, 2, 3}) {
+		t.Errorf("seeds: got %v", configuration.seeds)
+	}
+	if !slices.Equal(configuration.devices, []string{"emulator-5554", "emulator-5556"}) {
+		t.Errorf("devices: got %v", configuration.devices)
+	}
+	if configuration.duration != 5*time.Minute {
+		t.Errorf("duration default: got %s", configuration.duration)
+	}
+	if configuration.sanderlingPath != "sanderling" {
+		t.Errorf("sanderling default: got %q", configuration.sanderlingPath)
+	}
+}
+
+func TestParseArguments_Rejections(t *testing.T) {
+	cases := []struct {
+		name      string
+		arguments []string
+		want      string
+	}{
+		{"missing spec", []string{"--bundle-id", "a", "--arm", "b", "--seeds", "1", "--output", "o"}, "--spec is required"},
+		{"missing arm", []string{"--spec", "s", "--bundle-id", "a", "--seeds", "1", "--output", "o", "--max-steps", "10"}, "--arm is required"},
+		{"missing output", []string{"--spec", "s", "--bundle-id", "a", "--arm", "b", "--seeds", "1", "--max-steps", "10"}, "--output is required"},
+		{"bad platform", append(baseArguments(), "--platform", "windows"), "unsupported platform"},
+		{"bad generator", append(baseArguments(), "--generator", "vibes"), "unsupported generator"},
+		{"zero max steps", append(baseArguments(), "--max-steps", "0"), "--max-steps must be positive"},
+		{"seed zero", append(baseArguments(), "--seeds", "0-2"), "not reproducible"},
+		{"duplicate device", append(baseArguments(), "--devices", "a,a"), "duplicate device"},
+	}
+	for _, testCase := range cases {
+		_, err := parseArguments(testCase.arguments, io.Discard)
+		if err == nil {
+			t.Errorf("%s: expected error", testCase.name)
+			continue
+		}
+		if !strings.Contains(err.Error(), testCase.want) {
+			t.Errorf("%s: got %q, want it to contain %q", testCase.name, err, testCase.want)
+		}
+	}
+}
+
+func TestRunArguments_PlatformDeviceFlagAndPassthrough(t *testing.T) {
+	cases := []struct {
+		platform string
+		want     []string
+	}{
+		{"android", []string{"--device", "target-1"}},
+		{"ios", []string{"--ios-device", "target-1"}},
+		{"web", nil},
+	}
+	for _, testCase := range cases {
+		arguments := append(baseArguments(), "--platform", testCase.platform, "--", "--clear-data=false")
+		configuration, err := parseArguments(arguments, io.Discard)
+		if err != nil {
+			t.Fatalf("%s: %v", testCase.platform, err)
+		}
+		got := runArguments(configuration, "7", "target-1")
+		if got[0] != "test" {
+			t.Errorf("%s: first argument is %q, want test", testCase.platform, got[0])
+		}
+		for _, pair := range [][2]string{
+			{"--seed", "7"},
+			{"--arm", "seeded-baseline"},
+			{"--max-steps", "300"},
+			{"--generator", "seeded"},
+			{"--output", "/campaigns/a/seed-7"},
+		} {
+			if value := argumentValue(got, pair[0]); value != pair[1] {
+				t.Errorf("%s: %s = %q, want %q", testCase.platform, pair[0], value, pair[1])
+			}
+		}
+		if testCase.want != nil {
+			if value := argumentValue(got, testCase.want[0]); value != testCase.want[1] {
+				t.Errorf("%s: %s = %q, want %q", testCase.platform, testCase.want[0], value, testCase.want[1])
+			}
+		} else if slices.Contains(got, "--device") || slices.Contains(got, "--ios-device") {
+			t.Errorf("web: unexpected device flag in %v", got)
+		}
+		if got[len(got)-1] != "--clear-data=false" {
+			t.Errorf("%s: passthrough argument lost: %v", testCase.platform, got)
+		}
+	}
+}
+
+func argumentValue(arguments []string, name string) string {
+	index := slices.Index(arguments, name)
+	if index < 0 || index+1 >= len(arguments) {
+		return ""
+	}
+	return arguments[index+1]
+}

cmd/internal-tools/campaign/manifest.go

@@ -0,0 +1,69 @@
+package main
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+	"path/filepath"
+	"time"
+)
+
+const (
+	manifestFileName  = "campaign.json"
+	recordsFileName   = "runs.jsonl"
+	seedPlaceholder   = "{seed}"
+	devicePlaceholder = "{device}"
+)
+
+// manifest is campaign.json: what the campaign INTENDED to run, written before
+// the first run so a host that dropped runs shows up as missing seeds rather
+// than as a smaller sample.
+type manifest struct {
+	Arm               string    `json:"arm"`
+	Generator         string    `json:"generator"`
+	Platform          string    `json:"platform"`
+	SpecPath          string    `json:"spec_path"`
+	BundleID          string    `json:"bundle_id"`
+	MaxSteps          int       `json:"max_steps"`
+	DurationMillis    int64     `json:"duration_millis"`
+	RunTimeoutMillis  int64     `json:"run_timeout_millis"`
+	Seeds             []int64   `json:"seeds"`
+	Devices           []string  `json:"devices"`
+	Host              string    `json:"host"`
+	SanderlingPath    string    `json:"sanderling_path"`
+	SanderlingVersion string    `json:"sanderling_version"`
+	StartedAt         time.Time `json:"started_at"`
+	ArgumentTemplate  []string  `json:"argument_template"`
+}
+
+func buildManifest(configuration config, host, binaryPath, version string, startedAt time.Time) manifest {
+	devices := configuration.devices
+	if devices == nil {
+		devices = []string{}
+	}
+	return manifest{
+		Arm:               configuration.arm,
+		Generator:         configuration.generator,
+		Platform:          configuration.platform,
+		SpecPath:          configuration.specPath,
+		BundleID:          configuration.bundleID,
+		MaxSteps:          configuration.maxSteps,
+		DurationMillis:    configuration.duration.Milliseconds(),
+		RunTimeoutMillis:  configuration.runTimeout.Milliseconds(),
+		Seeds:             configuration.seeds,
+		Devices:           devices,
+		Host:              host,
+		SanderlingPath:    binaryPath,
+		SanderlingVersion: version,
+		StartedAt:         startedAt,
+		ArgumentTemplate:  runArguments(configuration, seedPlaceholder, devicePlaceholder),
+	}
+}
+
+func writeManifest(directory string, value manifest) error {
+	body, err := json.MarshalIndent(value, "", "  ")
+	if err != nil {
+		return fmt.Errorf("marshal manifest: %w", err)
+	}
+	return os.WriteFile(filepath.Join(directory, manifestFileName), append(body, '\n'), 0o644)
+}

cmd/internal-tools/campaign/manifest_test.go

@@ -0,0 +1,70 @@
+package main
+
+import (
+	"encoding/json"
+	"io"
+	"os"
+	"path/filepath"
+	"slices"
+	"testing"
+	"time"
+)
+
+func TestBuildManifest_RecordsIntendedRunsAndTemplate(t *testing.T) {
+	configuration, err := parseArguments(append(baseArguments(), "--devices", "a,b", "--duration", "90s"), io.Discard)
+	if err != nil {
+		t.Fatal(err)
+	}
+	startedAt := time.Date(2026, 8, 12, 9, 0, 0, 0, time.UTC)
+	value := buildManifest(configuration, "worker-7", "/usr/local/bin/sanderling", "1.4.2", startedAt)
+
+	if !slices.Equal(value.Seeds, []int64{1, 2, 3}) {
+		t.Errorf("seeds: got %v", value.Seeds)
+	}
+	if value.Host != "worker-7" || value.SanderlingVersion != "1.4.2" || value.SanderlingPath != "/usr/local/bin/sanderling" {
+		t.Errorf("provenance: %+v", value)
+	}
+	if value.DurationMillis != 90_000 || value.MaxSteps != 300 {
+		t.Errorf("budget: duration=%d max_steps=%d", value.DurationMillis, value.MaxSteps)
+	}
+	if !value.StartedAt.Equal(startedAt) {
+		t.Errorf("started_at: got %s", value.StartedAt)
+	}
+	if got := argumentValue(value.ArgumentTemplate, "--seed"); got != seedPlaceholder {
+		t.Errorf("template seed: got %q, want %q", got, seedPlaceholder)
+	}
+	if got := argumentValue(value.ArgumentTemplate, "--device"); got != devicePlaceholder {
+		t.Errorf("template device: got %q, want %q", got, devicePlaceholder)
+	}
+	if got := argumentValue(value.ArgumentTemplate, "--output"); got != "/campaigns/a/seed-"+seedPlaceholder {
+		t.Errorf("template output: got %q", got)
+	}
+}
+
+func TestBuildManifest_EmptyDeviceListSerializesAsArray(t *testing.T) {
+	configuration, err := parseArguments(baseArguments(), io.Discard)
+	if err != nil {
+		t.Fatal(err)
+	}
+	directory := t.TempDir()
+	if err := writeManifest(directory, buildManifest(configuration, "host", "sanderling", "dev", time.Now())); err != nil {
+		t.Fatal(err)
+	}
+	body, err := os.ReadFile(filepath.Join(directory, manifestFileName))
+	if err != nil {
+		t.Fatal(err)
+	}
+	var decoded struct {
+		Devices []string `json:"devices"`
+		Arm     string   `json:"arm"`
+	}
+	if err := json.Unmarshal(body, &decoded); err != nil {
+		t.Fatal(err)
+	}
+	if decoded.Devices == nil || len(decoded.Devices) != 0 {
+		t.Errorf("devices: got %v, want []", decoded.Devices)
+	}
+	if decoded.Arm != "seeded-baseline" {
+		t.Errorf("arm: got %q", decoded.Arm)
+	}
+}

cmd/internal-tools/campaign/seeds.go

@@ -0,0 +1,79 @@
+package main
+
+import (
+	"fmt"
+	"strconv"
+	"strings"
+)
+
+// parseSeeds expands a seed specification such as "1-10,20,30-32" into the
+// explicit seed list a campaign intends to run.
+func parseSeeds(specification string) ([]int64, error) {
+	trimmed := strings.TrimSpace(specification)
+	if trimmed == "" {
+		return nil, fmt.Errorf("empty seed spec")
+	}
+	var seeds []int64
+	seen := map[int64]bool{}
+	for _, part := range strings.Split(trimmed, ",") {
+		part = strings.TrimSpace(part)
+		if part == "" {
+			return nil, fmt.Errorf("empty seed in %q", specification)
+		}
+		expanded, err := expandSeedPart(part)
+		if err != nil {
+			return nil, err
+		}
+		for _, seed := range expanded {
+			if seen[seed] {
+				return nil, fmt.Errorf("duplicate seed %d in %q", seed, specification)
+			}
+			seen[seed] = true
+			seeds = append(seeds, seed)
+		}
+	}
+	return seeds, nil
+}
+
+func expandSeedPart(part string) ([]int64, error) {
+	start, end, isRange := strings.Cut(part, "-")
+	if !isRange {
+		seed, err := parseSeed(part)
+		if err != nil {
+			return nil, err
+		}
+		return []int64{seed}, nil
+	}
+	first, err := parseSeed(strings.TrimSpace(start))
+	if err != nil {
+		return nil, fmt.Errorf("seed range %q: %w", part, err)
+	}
+	last, err := parseSeed(strings.TrimSpace(end))
+	if err != nil {
+		return nil, fmt.Errorf("seed range %q: %w", part, err)
+	}
+	if first > last {
+		return nil, fmt.Errorf("seed range %q: start %d is above end %d", part, first, last)
+	}
+	seeds := make([]int64, 0, last-first+1)
+	for seed := first; seed <= last; seed++ {
+		seeds = append(seeds, seed)
+	}
+	return seeds, nil
+}
+
+func parseSeed(text string) (int64, error) {
+	seed, err := strconv.ParseInt(text, 10, 64)
+	if err != nil {
+		return 0, fmt.Errorf("invalid seed %q: want a positive integer", text)
+	}
+	if seed == 0 {
+		// `sanderling test` reads --seed 0 as "derive a seed from the clock",
+		// so a campaign listing seed 0 records a run nobody can reproduce.
+		return 0, fmt.Errorf("seed 0 is not reproducible: sanderling test derives a random seed when --seed is 0, so list explicit non-zero seeds")
+	}
+	if seed < 0 {
+		return 0, fmt.Errorf("invalid seed %d: want a positive integer", seed)
+	}
+	return seed, nil
+}

cmd/internal-tools/campaign/seeds_test.go

@@ -0,0 +1,49 @@
+package main
+
+import (
+	"slices"
+	"strings"
+	"testing"
+)
+
+func TestParseSeeds_RangesAndLists(t *testing.T) {
+	cases := []struct {
+		specification string
+		want          []int64
+	}{
+		{"1-5", []int64{1, 2, 3, 4, 5}},
+		{"1,5,9", []int64{1, 5, 9}},
+		{"1-3,20,30-32", []int64{1, 2, 3, 20, 30, 31, 32}},
+		{" 7 , 8 ", []int64{7, 8}},
+		{"4-4", []int64{4}},
+	}
+	for _, testCase := range cases {
+		got, err := parseSeeds(testCase.specification)
+		if err != nil {
+			t.Fatalf("%q: %v", testCase.specification, err)
+		}
+		if !slices.Equal(got, testCase.want) {
+			t.Errorf("%q: got %v, want %v", testCase.specification, got, testCase.want)
+		}
+	}
+}
+
+func TestParseSeeds_RejectsSeedZero(t *testing.T) {
+	for _, specification := range []string{"0", "1,0,2", "0-3"} {
+		_, err := parseSeeds(specification)
+		if err == nil {
+			t.Fatalf("%q: expected rejection of seed 0", specification)
+		}
+		if !strings.Contains(err.Error(), "not reproducible") {
+			t.Errorf("%q: error should explain why seed 0 is rejected: %v", specification, err)
+		}
+	}
+}
+
+func TestParseSeeds_RejectsMalformed(t *testing.T) {
+	for _, specification := range []string{"", "   ", "abc", "1,,2", "5-1", "1-", "-5", "1-2-3", "1.5", "2,2"} {
+		if seeds, err := parseSeeds(specification); err == nil {
+			t.Errorf("%q: expected error, got %v", specification, seeds)
+		}
+	}
+}

cmd/internal-tools/campaign/summary.go

@@ -0,0 +1,148 @@
+package main
+
+import (
+	"bufio"
+	"encoding/json"
+	"fmt"
+	"maps"
+	"os"
+	"path/filepath"
+	"slices"
+	"strings"
+
+	"github.com/priyanshujain/sanderling/internal/trace"
+)
+
+// Hierarchy dumps make trace lines large; match the replay server's ceiling.
+const maxTraceLineBytes = 16 * 1024 * 1024
+
+// traceSummary is everything the analysis needs from one run, so it never has
+// to open trace.jsonl again.
+type traceSummary struct {
+	Steps                      int      `json:"steps"`
+	FirstViolationOriginStep   *int     `json:"first_violation_origin_step"`
+	FirstViolationDetectedStep *int     `json:"first_violation_detected_step"`
+	FirstViolationProperties   []string `json:"first_violation_properties,omitempty"`
+	FirstViolationReason       string   `json:"first_violation_reason,omitempty"`
+	FirstViolationIsError      bool     `json:"first_violation_is_error,omitempty"`
+	ViolatedProperties         []string `json:"violated_properties,omitempty"`
+}
+
+type traceLine struct {
+	Index int `json:"step"`
+	// Hierarchy is read only for its presence: the run-end finalize line is the
+	// one line carrying violations without an observed hierarchy.
+	Hierarchy  json.RawMessage          `json:"hierarchy"`
+	Violations []string                 `json:"violations"`
+	Witnesses  map[string]trace.Witness `json:"witnesses"`
+}
+
+// findRunDirectory returns the run directory `sanderling test` created inside
+// seedDirectory. Names are UTC timestamps, so the last one sorted is the newest.
+func findRunDirectory(seedDirectory string) (string, error) {
+	entries, err := os.ReadDir(seedDirectory)
+	if err != nil {
+		return "", fmt.Errorf("read seed dir: %w", err)
+	}
+	names := make([]string, 0, len(entries))
+	for _, entry := range entries {
+		if !entry.IsDir() {
+			continue
+		}
+		if _, err := os.Stat(filepath.Join(seedDirectory, entry.Name(), "meta.json")); err != nil {
+			continue
+		}
+		names = append(names, entry.Name())
+	}
+	if len(names) == 0 {
+		return "", fmt.Errorf("no run directory with meta.json under %s", seedDirectory)
+	}
+	slices.Sort(names)
+	return names[len(names)-1], nil
+}
+
+// summarizeRun locates the run under seedDirectory and reduces its trace to the
+// fields the analysis reads. The returned path is relative to seedDirectory.
+func summarizeRun(seedDirectory string) (string, traceSummary, error) {
+	name, err := findRunDirectory(seedDirectory)
+	if err != nil {
+		return "", traceSummary{}, err
+	}
+	summary, err := summarizeTrace(filepath.Join(seedDirectory, name, "trace.jsonl"))
+	if err != nil {
+		return name, traceSummary{}, err
+	}
+	return name, summary, nil
+}
+
+func summarizeTrace(tracePath string) (traceSummary, error) {
+	file, err := os.Open(tracePath)
+	if err != nil {
+		return traceSummary{}, fmt.Errorf("open trace: %w", err)
+	}
+	defer file.Close()
+
+	var summary traceSummary
+	violated := map[string]bool{}
+	scanner := bufio.NewScanner(file)
+	scanner.Buffer(make([]byte, 0, 64*1024), maxTraceLineBytes)
+	lineNumber := 0
+	for scanner.Scan() {
+		lineNumber++
+		raw := strings.TrimSpace(scanner.Text())
+		if raw == "" {
+			continue
+		}
+		var line traceLine
+		if err := json.Unmarshal([]byte(raw), &line); err != nil {
+			return traceSummary{}, fmt.Errorf("trace line %d: %w", lineNumber, err)
+		}
+		// The finalize line is synthetic: it reports obligations that never
+		// discharged, at an index one past the last step actually executed.
+		synthetic := len(line.Violations) > 0 && len(line.Hierarchy) == 0
+		if !synthetic && line.Index > summary.Steps {
+			summary.Steps = line.Index
+		}
+		for _, property := range line.Violations {
+			violated[property] = true
+			recordViolation(&summary, line, property)
+		}
+	}
+	if err := scanner.Err(); err != nil {
+		return traceSummary{}, fmt.Errorf("read trace: %w", err)
+	}
+	if len(violated) > 0 {
+		summary.ViolatedProperties = slices.Sorted(maps.Keys(violated))
+	}
+	slices.Sort(summary.FirstViolationProperties)
+	return summary, nil
+}
+
+// recordViolation folds one violated property into the first-violation fields.
+// The origin step (the step that armed the failed obligation) orders the event,
+// because that is the step count the survival analysis measures.
+func recordViolation(summary *traceSummary, line traceLine, property string) {
+	origin, detected := line.Index, line.Index
+	witness := line.Witnesses[property]
+	if witness.Step > 0 {
+		origin = witness.Step
+	}
+	if witness.DetectedStep > 0 {
+		detected = witness.DetectedStep
+	}
+	switch {
+	case summary.FirstViolationOriginStep == nil || origin < *summary.FirstViolationOriginStep:
+		summary.FirstViolationOriginStep = &origin
+		summary.FirstViolationDetectedStep = &detected
+		summary.FirstViolationProperties = []string{property}
+		summary.FirstViolationReason = witness.Reason
+		summary.FirstViolationIsError = witness.IsError
+	case origin == *summary.FirstViolationOriginStep:
+		summary.FirstViolationProperties = append(summary.FirstViolationProperties, property)
+		if detected < *summary.FirstViolationDetectedStep {
+			summary.FirstViolationDetectedStep = &detected
+			summary.FirstViolationReason = witness.Reason
+			summary.FirstViolationIsError = witness.IsError
+		}
+	}
+}

cmd/internal-tools/campaign/summary_test.go

@@ -0,0 +1,200 @@
+package main
+
+import (
+	"bytes"
+	"encoding/json"
+	"os"
+	"path/filepath"
+	"slices"
+	"testing"
+	"time"
+
+	"github.com/priyanshujain/sanderling/internal/hierarchy"
+	"github.com/priyanshujain/sanderling/internal/trace"
+)
+
+func observedStep(index int) trace.Step {
+	return trace.Step{
+		Index:     index,
+		Timestamp: time.Date(2026, 8, 12, 9, 0, index, 0, time.UTC),
+		Screen:    "Home",
+		Hierarchy: &hierarchy.Tree{Elements: []*hierarchy.Element{{ResourceID: "root"}}},
+	}
+}
+
+func writeRunDirectory(t *testing.T, seedDirectory, name string, steps []trace.Step) string {
+	t.Helper()
+	directory := filepath.Join(seedDirectory, name)
+	if err := os.MkdirAll(directory, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	meta, err := json.Marshal(trace.Meta{Seed: 11, Platform: "web", Arm: "seeded-baseline"})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(filepath.Join(directory, "meta.json"), meta, 0o644); err != nil {
+		t.Fatal(err)
+	}
+	var buffer bytes.Buffer
+	encoder := json.NewEncoder(&buffer)
+	for _, step := range steps {
+		if err := encoder.Encode(step); err != nil {
+			t.Fatal(err)
+		}
+	}
+	if err := os.WriteFile(filepath.Join(directory, "trace.jsonl"), buffer.Bytes(), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	return directory
+}
+
+func TestSummarizeRun_CleanRunIsCensored(t *testing.T) {
+	seedDirectory := t.TempDir()
+	writeRunDirectory(t, seedDirectory, "20260812-090000", []trace.Step{
+		observedStep(1), observedStep(2), observedStep(3), observedStep(4), observedStep(5),
+	})
+
+	name, summary, err := summarizeRun(seedDirectory)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if name != "20260812-090000" {
+		t.Errorf("run directory: got %q", name)
+	}
+	if summary.Steps != 5 {
+		t.Errorf("steps: got %d, want 5", summary.Steps)
+	}
+	if summary.FirstViolationOriginStep != nil || summary.FirstViolationDetectedStep != nil {
+		t.Errorf("clean run reported a violation: %+v", summary)
+	}
+	if len(summary.ViolatedProperties) != 0 {
+		t.Errorf("violated properties: got %v", summary.ViolatedProperties)
+	}
+}
+
+func TestSummarizeRun_UsesWitnessOriginNotDetectionStep(t *testing.T) {
+	seedDirectory := t.TempDir()
+	violating := observedStep(9)
+	violating.Violations = []string{"balanceNeverNegative"}
+	violating.Witnesses = map[string]trace.Witness{
+		"balanceNeverNegative": {Reason: "balance went negative", Step: 4, DetectedStep: 9},
+	}
+	writeRunDirectory(t, seedDirectory, "20260812-090000", []trace.Step{
+		observedStep(1), observedStep(2), violating, observedStep(10),
+	})
+
+	_, summary, err := summarizeRun(seedDirectory)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if summary.FirstViolationOriginStep == nil || *summary.FirstViolationOriginStep != 4 {
+		t.Fatalf("origin step: got %v, want 4", summary.FirstViolationOriginStep)
+	}
+	if summary.FirstViolationDetectedStep == nil || *summary.FirstViolationDetectedStep != 9 {
+		t.Fatalf("detected step: got %v, want 9", summary.FirstViolationDetectedStep)
+	}
+	if summary.FirstViolationReason != "balance went negative" {
+		t.Errorf("reason: got %q", summary.FirstViolationReason)
+	}
+	if !slices.Equal(summary.FirstViolationProperties, []string{"balanceNeverNegative"}) {
+		t.Errorf("first violation properties: got %v", summary.FirstViolationProperties)
+	}
+}
+
+func TestSummarizeRun_EarliestOriginWinsOverEarliestDetection(t *testing.T) {
+	seedDirectory := t.TempDir()
+	early := observedStep(3)
+	early.Violations = []string{"detectedFirst"}
+	early.Witnesses = map[string]trace.Witness{"detectedFirst": {Step: 3, DetectedStep: 3}}
+	late := observedStep(8)
+	late.Violations = []string{"armedFirst"}
+	late.Witnesses = map[string]trace.Witness{"armedFirst": {Step: 1, DetectedStep: 8, IsError: true}}
+	writeRunDirectory(t, seedDirectory, "20260812-090000", []trace.Step{observedStep(1), early, late})
+
+	_, summary, err := summarizeRun(seedDirectory)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if summary.FirstViolationOriginStep == nil || *summary.FirstViolationOriginStep != 1 {
+		t.Fatalf("origin step: got %v, want 1", summary.FirstViolationOriginStep)
+	}
+	if !summary.FirstViolationIsError {
+		t.Error("is_error should come from the earliest-origin violation")
+	}
+	if !slices.Equal(summary.ViolatedProperties, []string{"armedFirst", "detectedFirst"}) {
+		t.Errorf("violated properties: got %v", summary.ViolatedProperties)
+	}
+}
+
+func TestSummarizeRun_FinalizeLineIsNotAStep(t *testing.T) {
+	seedDirectory := t.TempDir()
+	finalize := trace.Step{
+		Index:      4,
+		Timestamp:  time.Now().UTC(),
+		Violations: []string{"eventuallySettles"},
+		Witnesses:  map[string]trace.Witness{"eventuallySettles": {Step: 2, DetectedStep: 4}},
+	}
+	writeRunDirectory(t, seedDirectory, "20260812-090000", []trace.Step{
+		observedStep(1), observedStep(2), observedStep(3), finalize,
+	})
+
+	_, summary, err := summarizeRun(seedDirectory)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if summary.Steps != 3 {
+		t.Errorf("steps: got %d, want 3 (the finalize line is synthetic)", summary.Steps)
+	}
+	if summary.FirstViolationOriginStep == nil || *summary.FirstViolationOriginStep != 2 {
+		t.Fatalf("origin step: got %v, want 2", summary.FirstViolationOriginStep)
+	}
+}
+
+func TestSummarizeRun_FallsBackToStepIndexWithoutWitness(t *testing.T) {
+	seedDirectory := t.TempDir()
+	violating := observedStep(6)
+	violating.Violations = []string{"noWitness"}
+	writeRunDirectory(t, seedDirectory, "20260812-090000", []trace.Step{observedStep(5), violating})
+
+	_, summary, err := summarizeRun(seedDirectory)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if summary.FirstViolationOriginStep == nil || *summary.FirstViolationOriginStep != 6 {
+		t.Fatalf("origin step: got %v, want 6", summary.FirstViolationOriginStep)
+	}
+	if summary.FirstViolationDetectedStep == nil || *summary.FirstViolationDetectedStep != 6 {
+		t.Fatalf("detected step: got %v, want 6", summary.FirstViolationDetectedStep)
+	}
+}
+
+func TestSummarizeRun_PicksNewestRunDirectory(t *testing.T) {
+	seedDirectory := t.TempDir()
+	writeRunDirectory(t, seedDirectory, "20260812-090000", []trace.Step{observedStep(1)})
+	writeRunDirectory(t, seedDirectory, "20260812-093000", []trace.Step{observedStep(1), observedStep(2)})
+
+	name, summary, err := summarizeRun(seedDirectory)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if name != "20260812-093000" || summary.Steps != 2 {
+		t.Errorf("got %q with %d steps", name, summary.Steps)
+	}
+}
+
+func TestSummarizeRun_MissingRunDirectory(t *testing.T) {
+	if _, _, err := summarizeRun(t.TempDir()); err == nil {
+		t.Fatal("expected an error when no run directory was produced")
+	}
+}
+
+func TestSummarizeTrace_MalformedLine(t *testing.T) {
+	seedDirectory := t.TempDir()
+	directory := writeRunDirectory(t, seedDirectory, "20260812-090000", []trace.Step{observedStep(1)})
+	if err := os.WriteFile(filepath.Join(directory, "trace.jsonl"), []byte("{\"step\":1}\n{not json\n"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	if _, _, err := summarizeRun(seedDirectory); err == nil {
+		t.Fatal("expected an error for a malformed trace line")
+	}
+}

cmd/sanderling/main.go

@@ -27,6 +27,8 @@ type testOptions struct {
 	iosAppPath     string
 	androidAppPath string
 	duration       time.Duration
+	maxSteps       int
+	arm            string
 	seed           int64
 	output         string
 	clearData      bool
@@ -60,9 +62,11 @@ func parseTestArgs(args []string, stderr io.Writer) (testOptions, error) {
 	flagSet.StringVar(&options.iosAppPath, "ios-app-path", "", "path to the .app bundle for iOS clear-state reinstall (simulator: simctl; device: devicectl)")
 	flagSet.StringVar(&options.androidAppPath, "android-app-path", "", "path to the .apk for Android clear-state reinstall; required to reset apps on OEM builds that deny `pm clear`")
 	flagSet.DurationVar(&options.duration, "duration", 5*time.Minute, "total test duration")
+	flagSet.IntVar(&options.maxSteps, "max-steps", 0, "stop after this many steps (0 = no cap; the duration deadline governs). A step budget is what makes two generators comparable, since one making a model call per step and one drawing from a PRNG are not comparable per second")
 	flagSet.Int64Var(&options.seed, "seed", 0, "RNG seed (0 = random)")
 	flagSet.StringVar(&options.output, "output", "./runs", "output directory for traces")
 	flagSet.BoolVar(&options.clearData, "clear-data", true, "clear app data before launching so each run starts from a fresh install; pass --clear-data=false to resume prior state")
+	flagSet.StringVar(&options.arm, "arm", "", "experiment cell label, recorded in meta.json so a directory of runs can be attributed to a cell")
 	flagSet.StringVar(&options.generator, "generator", "seeded", "action generator: seeded (weighted random) or llm (model picks from the same candidate set; requires generator = llm() in the spec)")
 	if err := flagSet.Parse(args); err != nil {
 		return testOptions{}, err
@@ -78,6 +82,9 @@ func parseTestArgs(args []string, stderr io.Writer) (testOptions, error) {
 	default:
 		return testOptions{}, fmt.Errorf("unsupported platform: %q (android, ios, web)", options.platform)
 	}
+	if options.maxSteps < 0 {
+		return testOptions{}, fmt.Errorf("--max-steps must not be negative: %d", options.maxSteps)
+	}
 	switch options.generator {
 	case "seeded", "llm":
 	default:

cmd/sanderling/main_test.go

@@ -259,3 +259,55 @@ func TestRun_TestSubcommand_PipelineErrors(t *testing.T) {
 		t.Errorf("expected bundle-resolution error for the missing spec, got %v", err)
 	}
 }
+
+func TestParseTestArgs_MaxStepsDefaultsToUncapped(t *testing.T) {
+	options, err := parseTestArgs([]string{
+		"--spec", "s.ts",
+		"--bundle-id", "com.example",
+	}, io.Discard)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if options.maxSteps != 0 {
+		t.Errorf("maxSteps default: got %d, want 0", options.maxSteps)
+	}
+}
+
+func TestParseTestArgs_MaxSteps(t *testing.T) {
+	options, err := parseTestArgs([]string{
+		"--spec", "s.ts",
+		"--bundle-id", "com.example",
+		"--max-steps", "300",
+	}, io.Discard)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if options.maxSteps != 300 {
+		t.Errorf("maxSteps: got %d, want 300", options.maxSteps)
+	}
+}
+
+func TestParseTestArgs_RejectsNegativeMaxSteps(t *testing.T) {
+	_, err := parseTestArgs([]string{
+		"--spec", "s.ts",
+		"--bundle-id", "com.example",
+		"--max-steps", "-1",
+	}, io.Discard)
+	if err == nil {
+		t.Fatal("expected an error for a negative --max-steps")
+	}
+}
+
+func TestParseTestArgs_ArmLabel(t *testing.T) {
+	options, err := parseTestArgs([]string{
+		"--spec", "s.ts",
+		"--bundle-id", "com.example",
+		"--arm", "seeded-identifier",
+	}, io.Discard)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if options.arm != "seeded-identifier" {
+		t.Errorf("arm: got %q, want seeded-identifier", options.arm)
+	}
+}

cmd/sanderling/test_run.go

@@ -18,9 +18,11 @@ func runTestPipeline(ctx context.Context, options testOptions, stdout io.Writer)
 		IosAppPath:     options.iosAppPath,
 		AndroidAppPath: options.androidAppPath,
 		Duration:       options.duration,
+		MaxSteps:       options.maxSteps,
 		Seed:           options.seed,
 		Output:         options.output,
 		ClearData:      options.clearData,
 		Generator:      options.generator,
+		Arm:            options.arm,
 	}, stdout)
 }

conformance/gates.sh

@@ -17,12 +17,20 @@
 #   ./gates.sh                          run the simulator gates
 #   BACKEND=device IOS_DEVICE="iPhone" ./gates.sh
 #   BACKEND=android ANDROID_DEVICE="663c91b1" ./gates.sh
+#   RUNS=1 SEEDS=303 ./gates.sh         re-run just the seed that failed
 #   ./gates.sh --self-test              run the offline analyzer tests only
 #
 # Tunables (environment):
 #   RUNS=5            number of serial runs
 #   DURATION=3m       per-run fuzz duration
-#   SEED=0            fuzz seed
+#   SEEDS="101 202 303 404 505"
+#                     one fuzz seed per run, whitespace separated, consumed in
+#                     order. The values differ so the runs explore different
+#                     action streams, and they are fixed so a failing gate can
+#                     be re-run exactly; the results table prints the seed each
+#                     run used. A 0 is rejected: sanderling test reads --seed 0
+#                     as "derive the seed from the clock", which is the one
+#                     thing a gate cannot reproduce.
 #   P95_LIMIT_MS      G5 p95 step-latency ceiling in ms (default 2500 for iOS,
 #                     5500 for the android backend's higher and more variable
 #                     per-step USB cost, especially cold right after a reboot)
@@ -36,7 +44,7 @@ folio_directory="$(cd "${script_directory}/../examples/folio" && pwd)"
 BACKEND="${BACKEND:-simulator}"
 RUNS="${RUNS:-5}"
 DURATION="${DURATION:-3m}"
-SEED="${SEED:-0}"
+SEEDS="${SEEDS:-101 202 303 404 505}"
 # The p95 ceiling is backend-specific: a physical Android device drives every
 # step over USB (snapshot + settle + adb round-trips), so its per-step floor is
 # several times the iOS simulator's in-process cost. 2500ms was calibrated on
@@ -51,6 +59,11 @@ SANDERLING="${SANDERLING:-sanderling}"
 IOS_DEVICE="${IOS_DEVICE:-iPhone 17 Pro}"
 ANDROID_DEVICE="${ANDROID_DEVICE:-}"
 
+if [[ -n "${SEED:-}" ]]; then
+  echo "gates.sh: SEED is not read any more; set SEEDS to one seed per run" >&2
+  exit 2
+fi
+
 bundle_id="app.folio"
 spec_path="${folio_directory}/sanderling/spec.ts"
 android_apk="${folio_directory}/app/androidApp/build/outputs/apk/debug/androidApp-debug.apk"
@@ -246,10 +259,40 @@ orphan_processes() {
 
 # ---- run orchestration -----------------------------------------------------
 
+# Split a seed list into the global seed_list, one entry per requested run.
+# sanderling test reads --seed 0 as "derive the seed from the clock", so a 0
+# here would make that run unreproducible and is refused.
+seed_list=()
+select_seeds() {
+  local requested="$1"
+  local -a candidates=()
+  read -r -a candidates <<<"$2"
+  if [[ "${#candidates[@]}" -lt "$requested" ]]; then
+    echo "gates.sh: SEEDS has ${#candidates[@]} value(s) but RUNS=${requested}; give one seed per run" >&2
+    return 1
+  fi
+  local index seed
+  for ((index = 0; index < ${#candidates[@]}; index++)); do
+    seed="${candidates[index]}"
+    case "$seed" in
+      '' | *[!0-9]*)
+        echo "gates.sh: SEEDS takes whitespace-separated positive integers; got \"${seed}\"" >&2
+        return 1
+        ;;
+    esac
+    if [[ "$((10#$seed))" -eq 0 ]]; then
+      echo "gates.sh: SEEDS may not contain 0; sanderling test derives a clock seed from 0, so a gate configured with it cannot be reproduced" >&2
+      return 1
+    fi
+  done
+  seed_list=("${candidates[@]:0:requested}")
+}
+
 invoke_sanderling() {
   local output_directory="$1"
   local output_log="$2"
   local exit_status_file="$3"
+  local seed="$4"
 
   local platform target_flags=()
   if [[ "$BACKEND" == "android" ]]; then
@@ -281,7 +324,7 @@ invoke_sanderling() {
     --bundle-id "$bundle_id" \
     "${target_flags[@]}" \
     --duration "$DURATION" \
-    --seed "$SEED" \
+    --seed "$seed" \
     --output "$output_directory" \
     >"$output_log" 2>&1 || status=$?
   printf '%s' "$status" >"$exit_status_file"
@@ -300,6 +343,8 @@ collect_run_artifacts() {
 }
 
 run_gates() {
+  select_seeds "$RUNS" "$SEEDS" || exit 2
+
   if [[ "$BACKEND" == "android" ]]; then
     # Build only; invoke_sanderling reinstalls per run via adb (gradle's ddmlib
     # install is flaky on some physical devices).
@@ -326,7 +371,7 @@ run_gates() {
   local gate_root="${script_directory}/runs/${timestamp}"
   mkdir -p "$gate_root"
   echo "gate root: ${gate_root}"
-  echo "backend=${BACKEND} runs=${RUNS} duration=${DURATION} seed=${SEED} p95_limit_ms=${P95_LIMIT_MS}"
+  echo "backend=${BACKEND} runs=${RUNS} duration=${DURATION} seeds=${seed_list[*]} p95_limit_ms=${P95_LIMIT_MS}"
 
   local all_latencies="${gate_root}/all-latencies.txt"
   : >"$all_latencies"
@@ -335,10 +380,12 @@ run_gates() {
   local run_index
   for ((run_index = 1; run_index <= RUNS; run_index++)); do
     local run_directory="${gate_root}/run-${run_index}"
+    local seed="${seed_list[run_index - 1]}"
     mkdir -p "$run_directory"
-    echo "run ${run_index}/${RUNS} -> ${run_directory}"
+    printf '%s' "$seed" >"${run_directory}/seed"
+    echo "run ${run_index}/${RUNS} seed=${seed} -> ${run_directory}"
 
-    invoke_sanderling "$run_directory" "${run_directory}/output.log" "${run_directory}/exit_status"
+    invoke_sanderling "$run_directory" "${run_directory}/output.log" "${run_directory}/exit_status" "$seed"
     collect_run_artifacts "$run_directory"
 
     g1[run_index]=$(gate_exit_zero      "$run_directory" && echo PASS || echo FAIL)
@@ -383,11 +430,12 @@ run_gates() {
   done
 
   echo
-  printf 'run  G1    G2    G3    G4    G5\n'
+  printf 'run  seed   G1    G2    G3    G4    G5\n'
   local verdict="PASS"
   for ((run_index = 1; run_index <= RUNS; run_index++)); do
-    printf '%-4s %-5s %-5s %-5s %-5s %-5s\n' \
-      "$run_index" "${g1[run_index]}" "${g2[run_index]}" "${g3[run_index]}" "${g4[run_index]}" "${g5[run_index]}"
+    printf '%-4s %-6s %-5s %-5s %-5s %-5s %-5s\n' \
+      "$run_index" "${seed_list[run_index - 1]}" \
+      "${g1[run_index]}" "${g2[run_index]}" "${g3[run_index]}" "${g4[run_index]}" "${g5[run_index]}"
     for cell in "${g1[run_index]}" "${g2[run_index]}" "${g3[run_index]}" "${g4[run_index]}" "${g5[run_index]}"; do
       [[ "$cell" == "PASS" ]] || verdict="FAIL"
     done
@@ -399,6 +447,7 @@ run_gates() {
     return 0
   fi
   echo "GATES FAIL"
+  echo "reproduce one run with: RUNS=1 SEEDS=<seed from the table> BACKEND=${BACKEND} $0"
   return 1
 }
 
@@ -447,6 +496,19 @@ self_test() {
   assert "G4 doubled text caught" FAIL \
     "$(gate_no_doubled_text "${testdata}/g4-doubled-text" && echo PASS || echo FAIL)"
 
+  assert "seeds default to one distinct value per run" "101 202 303 404 505" \
+    "$(select_seeds 5 "101 202 303 404 505" >/dev/null 2>&1 && echo "${seed_list[*]}")"
+  assert "seeds override honoured, extras ignored" "11 22" \
+    "$(select_seeds 2 "11 22 33" >/dev/null 2>&1 && echo "${seed_list[*]}")"
+  assert "seed 0 rejected" FAIL \
+    "$(select_seeds 1 "0" >/dev/null 2>&1 && echo PASS || echo FAIL)"
+  assert "padded seed 00 rejected" FAIL \
+    "$(select_seeds 1 "00" >/dev/null 2>&1 && echo PASS || echo FAIL)"
+  assert "non-numeric seed rejected" FAIL \
+    "$(select_seeds 1 "abc" >/dev/null 2>&1 && echo PASS || echo FAIL)"
+  assert "seed list shorter than RUNS rejected" FAIL \
+    "$(select_seeds 5 "101 202" >/dev/null 2>&1 && echo PASS || echo FAIL)"
+
   local pass_p95 slow_p95
   pass_p95="$(emit_step_latencies "${testdata}/pass/trace.jsonl" | p95_of)"
   slow_p95="$(emit_step_latencies "${testdata}/g5-slow-p95/trace.jsonl" | p95_of)"

examples/folio/README.md

@@ -82,8 +82,8 @@ Traces land in `./sanderling/runs/<timestamp>/`.
 
 The same `sanderling/spec.ts` runs under either generator: `--generator seeded`
 (the default weighted fuzzer) or `--generator llm`, where a vision model picks
-from the SAME weighted candidate set — reading the screenshot plus a numbered,
-weight-annotated list of concrete actions — and returns one number. The spec's
+from the SAME weighted candidate set (reading the screenshot plus a numbered,
+weight-annotated list of concrete actions) and returns one number. The spec's
 `generator = llm({ model, instructions })` export configures it.
 
 ```sh

examples/folio/sanderling/spec.ts

@@ -192,7 +192,7 @@ export const actionsRoot = weighted(
 // The LLM generator is orthogonal to actionsRoot: with `--generator llm` a model
 // picks from the SAME weighted candidate set above, reading the screenshot and a
 // numbered, weight-annotated list; the default `--generator seeded` ignores it.
-// instructions describe only WHAT the app is, never HOW to test it — the model
+// instructions describe only WHAT the app is, never HOW to test it. The model
 // figures out how to surface bugs on its own. With a plain OpenAI key, drop the
 // vendor prefix from the model id.
 export const generator = llm({

internal/driver/chrome/driver.go

@@ -97,19 +97,31 @@ func New() *Driver {
 }
 
 func (d *Driver) Launch(ctx context.Context, bundleID string, clearState bool, _ map[string]string) error {
+	// Allocate the browser against the driver's own context before anything
+	// caller-bound runs. chromedp starts Chrome under whichever context first
+	// calls Run, so allocating under a caller deadline would tie the browser
+	// process to this one call and kill it the moment Launch returns.
+	if err := chromedp.Run(d.tabCtx); err != nil {
+		return err
+	}
+	// Everything after allocation goes through runCtx, so a caller deadline or
+	// a SIGTERM aborts a launch that would otherwise wait forever on a target
+	// that accepts the connection and never answers.
+	runCtx, cancel := d.runCtx(ctx)
+	defer cancel()
 	if clearState {
-		if err := d.clearState(bundleID); err != nil {
+		if err := d.clearState(runCtx, bundleID); err != nil {
 			return err
 		}
 	}
-	if err := chromedp.Run(d.tabCtx, chromedp.Navigate(bundleID)); err != nil {
+	if err := chromedp.Run(runCtx, chromedp.Navigate(bundleID)); err != nil {
 		return err
 	}
 	// After navigation, read CSS custom properties --frame-w / --frame-h (common
 	// mobile-frame convention) so screenshots fit the app without grey borders.
 	// Falls back to the body scroll dimensions if the properties are absent.
 	var dims [2]int64
-	if err := chromedp.Run(d.tabCtx, chromedp.Evaluate(`
+	if err := chromedp.Run(runCtx, chromedp.Evaluate(`
 		(function() {
 			const s = getComputedStyle(document.documentElement);
 			const pw = parseInt(s.getPropertyValue('--frame-w'), 10);
@@ -118,7 +130,7 @@ func (d *Driver) Launch(ctx context.Context, bundleID string, clearState bool, _
 			const h = isNaN(ph) ? document.body.scrollHeight : ph;
 			return [w, h];
 		})()`, &dims)); err == nil && dims[0] > 0 && dims[1] > 0 {
-		_ = chromedp.Run(d.tabCtx, chromedp.EmulateViewport(dims[0], dims[1]))
+		_ = chromedp.Run(runCtx, chromedp.EmulateViewport(dims[0], dims[1]))
 	}
 	return nil
 }
@@ -130,8 +142,8 @@ func (d *Driver) Launch(ctx context.Context, bundleID string, clearState bool, _
 // which needs no navigation. sessionStorage is per-tab and outside that
 // domain's reach; it only survives when a relaunch reuses a tab already on
 // the target origin, which is the one case where script can reach it.
-func (d *Driver) clearState(bundleID string) error {
-	if err := chromedp.Run(d.tabCtx, network.ClearBrowserCookies()); err != nil {
+func (d *Driver) clearState(runCtx context.Context, bundleID string) error {
+	if err := chromedp.Run(runCtx, network.ClearBrowserCookies()); err != nil {
 		return fmt.Errorf("clear cookies: %w", err)
 	}
 	origin := securityOrigin(bundleID)
@@ -139,12 +151,12 @@ func (d *Driver) clearState(bundleID string) error {
 		return nil
 	}
 	clearForOrigin := storage.ClearDataForOrigin(origin, string(storage.TypeAll))
-	if err := chromedp.Run(d.tabCtx, clearForOrigin); err != nil {
+	if err := chromedp.Run(runCtx, clearForOrigin); err != nil {
 		return fmt.Errorf("clear storage for %s: %w", origin, err)
 	}
 	script := fmt.Sprintf(
 		`location.origin === %q && (sessionStorage.clear(), true)`, origin)
-	return chromedp.Run(d.tabCtx, chromedp.ActionFunc(func(ctx context.Context) error {
+	return chromedp.Run(runCtx, chromedp.ActionFunc(func(ctx context.Context) error {
 		_, exception, err := runtime.Evaluate(script).Do(ctx)
 		if err != nil {
 			return fmt.Errorf("clear session storage: %w", err)
@@ -357,6 +369,24 @@ func (d *Driver) Hierarchy(ctx context.Context) (string, error) {
 	script := `
 (function() {
   const route = window.location.hash.replace(/^#/, '').split('?')[0] || '/';
+  // clickable and editable are resolved through the SAME selector sets
+  // pkg/spec/src/web-runtime.ts uses, so the goja host (which reads this dump)
+  // and the V8 host (which reads the DOM directly) cannot mean different things
+  // by one fact on one platform. Testing el.onclick instead made every React
+  // root a full-viewport tap target here and nowhere else.
+  const NON_TEXT_INPUT_TYPES =
+    ['button','submit','checkbox','radio','range','color','file','image','reset'];
+  function isEditableElement(el) {
+    if (el.isContentEditable) return true;
+    const tag = el.tagName.toLowerCase();
+    if (tag === 'textarea') return true;
+    if (tag === 'input') return !NON_TEXT_INPUT_TYPES.includes((el.type || '').toLowerCase());
+    return false;
+  }
+  const clickableSet = new Set(document.querySelectorAll(
+    'a, button, input, select, textarea, [role="button"], [onclick]'));
+  const editableSet = new Set(Array.from(
+    document.querySelectorAll('input, textarea, [contenteditable]')).filter(isEditableElement));
   function buildTree(el, isRoot) {
     const rect = el.getBoundingClientRect();
     const attrs = {};
@@ -373,15 +403,19 @@ func (d *Driver) Hierarchy(ctx context.Context) (string, error) {
     if (el.className && typeof el.className === 'string' && el.className.trim()) {
       attrs['class'] = el.className.trim();
     }
+    // The goja host reads scrollable off this attribute (internal/verifier
+    // worker.go targets). Without it every web element looks unscrollable there,
+    // so the goja-side enumeration offers no scroll while the V8 picker, which
+    // computes the same overflow test in web-runtime.ts, offers plenty.
+    if (el.scrollHeight > el.clientHeight || el.scrollWidth > el.clientWidth) {
+      attrs['scrollable'] = 'true';
+    }
     if (isRoot) attrs['sanderling-screen'] = route;
-    const isClickable = !!(el.onclick || el.tagName === 'A' || el.tagName === 'BUTTON' ||
-      el.tagName === 'INPUT' || el.tagName === 'SELECT' ||
-      el.getAttribute('role') === 'button' || el.getAttribute('onclick'));
-    const isEditable = el.isContentEditable || tag === 'textarea' ||
-      (tag === 'input' && !['button','submit','checkbox','radio','range','color','file','image','reset']
-        .includes((el.type || '').toLowerCase()));
+    const isClickable = clickableSet.has(el);
+    const isEditable = editableSet.has(el);
     const children = [];
     for (const child of el.children) {
+      if (child.tagName === 'HEAD') continue;
       children.push(buildTree(child, false));
     }
     return {
@@ -392,10 +426,22 @@ func (d *Driver) Hierarchy(ctx context.Context) (string, error) {
       focused: document.activeElement === el || null,
       checked: el.checked || null,
       selected: el.selected || null,
-      editable: isEditable || null,
+      // Emitted as a plain boolean, never null: internal/hierarchy falls back to
+      // the native heuristic when the field is absent, which reads any class
+      // name containing "EditText" as an Android text widget. On web that is a
+      // CSS class, so a page styling a div with it made the goja host offer
+      // typing into a div the web runtime never calls editable.
+      editable: isEditable,
     };
   }
-  return buildTree(document.body, true);
+  // Rooted at documentElement, not body, because collectTargets in
+  // pkg/spec/src/web-runtime.ts walks querySelectorAll("*") and therefore sees
+  // html. Page-level scrolling lives on html on a standard page, so a dump
+  // rooted at body hides it from the goja host and the two enumerations
+  // disagree on exactly the page scroll. The head subtree is skipped: it is all
+  // zero-bounds, so it changes no eligible set, and it would otherwise pull
+  // script and title text into the trace and the replay view.
+  return buildTree(document.documentElement, true);
 })()`
 
 	var result any

internal/driver/chrome/driver_test.go

@@ -6,6 +6,7 @@ import (
 	"context"
 	"encoding/json"
 	"errors"
+	"net"
 	"net/http"
 	"net/http/httptest"
 	"testing"
@@ -185,6 +186,125 @@ func TestHierarchy_EditableFlag(t *testing.T) {
 	}
 }
 
+// TestHierarchy_ClickableMatchesTheWebRuntimeSelector pins clickable to the
+// same membership test pkg/spec/src/web-runtime.ts applies. The dump used to
+// test el.onclick, which React sets on its root container for event delegation,
+// so the whole viewport became a tap target in this dump and in no other
+// enumeration of the same page.
+func TestHierarchy_ClickableMatchesTheWebRuntimeSelector(t *testing.T) {
+	const html = `<body>` +
+		`<div id="root"><button id="go">go</button></div>` +
+		`<textarea id="bio"></textarea>` +
+		`<div id="plain">text</div>` +
+		`<div id="rolebutton" role="button">act</div>` +
+		`<script>document.getElementById("root").onclick = function () {};</script>` +
+		`</body>`
+
+	d := New()
+	defer d.Terminate(context.Background())
+	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+	defer cancel()
+	if err := d.Launch(ctx, "data:text/html,"+html, false, nil); err != nil {
+		t.Fatalf("Launch: %v", err)
+	}
+	dump, err := d.Hierarchy(ctx)
+	if err != nil {
+		t.Fatalf("Hierarchy: %v", err)
+	}
+
+	type node struct {
+		Attributes map[string]string `json:"attributes"`
+		Children   []node            `json:"children"`
+		Clickable  *bool             `json:"clickable"`
+	}
+	var root node
+	if err := json.Unmarshal([]byte(dump), &root); err != nil {
+		t.Fatalf("unmarshal hierarchy: %v", err)
+	}
+	clickableByID := map[string]*bool{}
+	var walk func(n node)
+	walk = func(n node) {
+		if id := n.Attributes["resource-id"]; id != "" {
+			clickableByID[id] = n.Clickable
+		}
+		for _, c := range n.Children {
+			walk(c)
+		}
+	}
+	walk(root)
+
+	isClickable := func(id string) bool {
+		return clickableByID[id] != nil && *clickableByID[id]
+	}
+	for _, id := range []string{"go", "bio", "rolebutton"} {
+		if !isClickable(id) {
+			t.Errorf("%q: clickable = %v, want true", id, clickableByID[id])
+		}
+	}
+	for _, id := range []string{"root", "plain"} {
+		if isClickable(id) {
+			t.Errorf("%q: clickable = true, want false/absent (an onclick property is not a target)", id)
+		}
+	}
+}
+
+// TestHierarchy_ScrollableAttribute covers the fact the goja host reads off the
+// attributes map. The V8 picker computes the same overflow test in
+// web-runtime.ts, so leaving it out of the dump made the two enumerations
+// disagree on web: the model policy could never be offered a scroll the seeded
+// policy could draw.
+func TestHierarchy_ScrollableAttribute(t *testing.T) {
+	const html = `<body style="margin:0">` +
+		`<div id="overflowing" style="width:100px;height:100px;overflow:auto">` +
+		`<div style="width:100px;height:900px"></div></div>` +
+		`<div id="sideways" style="width:100px;height:50px;overflow:auto">` +
+		`<div style="width:900px;height:20px"></div></div>` +
+		`<div id="fits" style="width:100px;height:100px;overflow:auto">` +
+		`<div style="width:50px;height:50px"></div></div>` +
+		`</body>`
+
+	d := New()
+	defer d.Terminate(context.Background())
+	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+	defer cancel()
+	if err := d.Launch(ctx, "data:text/html,"+html, false, nil); err != nil {
+		t.Fatalf("Launch: %v", err)
+	}
+	dump, err := d.Hierarchy(ctx)
+	if err != nil {
+		t.Fatalf("Hierarchy: %v", err)
+	}
+
+	type node struct {
+		Attributes map[string]string `json:"attributes"`
+		Children   []node            `json:"children"`
+	}
+	var root node
+	if err := json.Unmarshal([]byte(dump), &root); err != nil {
+		t.Fatalf("unmarshal hierarchy: %v", err)
+	}
+	scrollableByID := map[string]string{}
+	var walk func(n node)
+	walk = func(n node) {
+		if id := n.Attributes["resource-id"]; id != "" {
+			scrollableByID[id] = n.Attributes["scrollable"]
+		}
+		for _, c := range n.Children {
+			walk(c)
+		}
+	}
+	walk(root)
+
+	for _, id := range []string{"overflowing", "sideways"} {
+		if scrollableByID[id] != "true" {
+			t.Errorf("%q: scrollable = %q, want \"true\"", id, scrollableByID[id])
+		}
+	}
+	if scrollableByID["fits"] != "" {
+		t.Errorf("%q: scrollable = %q, want absent", "fits", scrollableByID["fits"])
+	}
+}
+
 // TestRunCtx_CallerCancelPropagates confirms that cancelling the caller's
 // context cancels the chromedp-bound context returned by runCtx. This is the
 // channel by which step deadlines and Ctrl-C reach in-flight CDP calls.
@@ -221,3 +341,136 @@ func TestRunCtx_TabCancelPropagates(t *testing.T) {
 		t.Fatal("derived ctx did not cancel after tab cancellation")
 	}
 }
+
+// TestHierarchy_RootsAtDocumentElementWithoutHead pins the dump's root to html,
+// because collectTargets in pkg/spec/src/web-runtime.ts walks
+// querySelectorAll("*") and therefore sees html. A dump rooted at body hides
+// page-level scrolling, which lives on html on a standard page, so the two
+// enumerations disagree on exactly that candidate. The head subtree stays out:
+// it is all zero-bounds, so it changes no eligible set, and it would otherwise
+// carry script and title text into the trace.
+func TestHierarchy_RootsAtDocumentElementWithoutHead(t *testing.T) {
+	const html = `<html><head><title>secret title</title>` +
+		`<script>window.marker = 1;</script></head>` +
+		`<body><div id="content">hello</div></body></html>`
+
+	d := New()
+	defer d.Terminate(context.Background())
+	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+	defer cancel()
+	if err := d.Launch(ctx, "data:text/html,"+html, false, nil); err != nil {
+		t.Fatalf("Launch: %v", err)
+	}
+	dump, err := d.Hierarchy(ctx)
+	if err != nil {
+		t.Fatalf("Hierarchy: %v", err)
+	}
+
+	type node struct {
+		Attributes map[string]string `json:"attributes"`
+		Children   []node            `json:"children"`
+	}
+	var root node
+	if err := json.Unmarshal([]byte(dump), &root); err != nil {
+		t.Fatalf("unmarshal hierarchy: %v", err)
+	}
+	if root.Attributes["tag"] != "html" {
+		t.Errorf("root tag: got %q, want html", root.Attributes["tag"])
+	}
+	if root.Attributes["sanderling-screen"] == "" {
+		t.Error("the root must still carry sanderling-screen")
+	}
+
+	tags := map[string]bool{}
+	var walk func(n node)
+	walk = func(n node) {
+		tags[n.Attributes["tag"]] = true
+		for _, child := range n.Children {
+			walk(child)
+		}
+	}
+	walk(root)
+	if !tags["body"] {
+		t.Error("body must appear under the root")
+	}
+	for _, tag := range []string{"head", "title", "script"} {
+		if tags[tag] {
+			t.Errorf("the head subtree must stay out of the dump, found %q", tag)
+		}
+	}
+}
+
+// TestLaunch_HonorsCallerDeadline points the driver at a listener that accepts
+// the connection and never answers. Chrome has no page-load deadline of its
+// own, so a Launch that ignored its caller context waited forever: an
+// unattended campaign worker aimed at an unreachable target wedged with no
+// diagnostic and did not even answer SIGTERM.
+func TestLaunch_HonorsCallerDeadline(t *testing.T) {
+	listener, err := net.Listen("tcp", "127.0.0.1:0")
+	if err != nil {
+		t.Fatalf("listen: %v", err)
+	}
+	defer listener.Close()
+	held := make(chan net.Conn, 8)
+	go func() {
+		for {
+			conn, err := listener.Accept()
+			if err != nil {
+				return
+			}
+			held <- conn
+		}
+	}()
+	defer func() {
+		close(held)
+		for conn := range held {
+			conn.Close()
+		}
+	}()
+
+	d := New()
+	defer d.Terminate(context.Background())
+
+	ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
+	defer cancel()
+	done := make(chan error, 1)
+	go func() { done <- d.Launch(ctx, "http://"+listener.Addr().String(), false, nil) }()
+	select {
+	case err := <-done:
+		if err == nil {
+			t.Fatal("Launch returned nil against a target that never answers")
+		}
+		if !errors.Is(err, context.DeadlineExceeded) && !errors.Is(err, context.Canceled) {
+			t.Fatalf("Launch error = %v, want a context error", err)
+		}
+	case <-time.After(45 * time.Second):
+		t.Fatal("Launch ignored the caller deadline and blocked")
+	}
+}
+
+// TestLaunch_KeepsBrowserAliveAfterCallerContextEnds guards the trap that made
+// Launch use the driver context in the first place: chromedp starts Chrome
+// under whichever context runs first, so allocating under the caller's
+// short-lived context would kill the browser as soon as Launch returned.
+func TestLaunch_KeepsBrowserAliveAfterCallerContextEnds(t *testing.T) {
+	d := New()
+	defer d.Terminate(context.Background())
+
+	ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
+	if err := d.Launch(ctx, "data:text/html,<body>alive</body>", false, nil); err != nil {
+		t.Fatalf("Launch: %v", err)
+	}
+	cancel()
+
+	var text string
+	if err := chromedp.Run(d.tabCtx,
+		chromedp.Evaluate(`document.body.textContent`, &text)); err != nil {
+		t.Fatalf("browser died with the caller context: %v", err)
+	}
+	if text != "alive" {
+		t.Errorf("body text = %q, want \"alive\"", text)
+	}
+	if err := d.Launch(context.Background(), "data:text/html,<body>again</body>", false, nil); err != nil {
+		t.Fatalf("second Launch after the first caller context ended: %v", err)
+	}
+}

internal/driver/chrome/fact_parity_test.go

@@ -0,0 +1,343 @@
+//go:build browser
+
+package chrome
+
+import (
+	"context"
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+	"path/filepath"
+	"runtime"
+	"slices"
+	"testing"
+	"time"
+
+	"github.com/chromedp/chromedp"
+
+	"github.com/priyanshujain/sanderling/internal/bundler"
+	"github.com/priyanshujain/sanderling/internal/hierarchy"
+)
+
+// One page, two fact producers.
+//
+// Sanderling enumerates candidate actions from two independent hosts. The goja
+// host reads the hierarchy dump this driver builds; the V8 host reads the live
+// DOM through collectTargets in pkg/spec/src/web-runtime.ts. Both feed the SAME
+// eligibility rule (pkg/spec/src/targets.ts), so the two enumerations agree only
+// as far as the two producers agree on the facts.
+//
+// internal/verifier/host_parity_test.go and pkg/spec/test/host-parity.test.ts
+// pin the other half: given identical facts, both hosts select identical
+// candidates. They hand-author those facts on both sides, so neither producer is
+// on their path, and three producer bugs lived in that gap: the dump emitted no
+// `scrollable` attribute at all, it derived `clickable` from el.onclick (which
+// React assigns to its root container, making the whole viewport a tap target
+// here and nowhere else), and it rooted at document.body while the web runtime
+// walks the whole document, hiding `html` and with it every page-level scroll.
+//
+// This test starts from one real DOM in a real browser and compares what the two
+// producers derive from it, element by element. It found a fourth: the dump left
+// `editable` unset rather than false, and an unset field sends the parser into
+// the native EditText class-name heuristic, which reads a CSS class as an
+// Android text widget.
+
+// elementFacts is one element as a producer reports it: the tag, for readable
+// failures, and every fact acceptsTarget consults.
+type elementFacts struct {
+	tag            string
+	clickable      bool
+	enabled        bool
+	editable       bool
+	scrollable     bool
+	positiveBounds bool
+}
+
+// factRow pairs an element's facts with the id both producers key on, kept in
+// enumeration order because the order is part of the picker's parity contract.
+type factRow struct {
+	id    string
+	facts elementFacts
+}
+
+func TestHierarchy_DerivesTheSameFactsAsTheWebRuntime(t *testing.T) {
+	server := httptest.NewServer(http.FileServer(http.Dir("testdata")))
+	defer server.Close()
+
+	d := New()
+	defer d.Terminate(context.Background())
+	ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
+	defer cancel()
+	if err := d.Launch(ctx, server.URL+"/fact-parity.html", false, nil); err != nil {
+		t.Fatalf("Launch: %v", err)
+	}
+
+	dump, err := d.Hierarchy(ctx)
+	if err != nil {
+		t.Fatalf("Hierarchy: %v", err)
+	}
+	fromDump := factsFromHierarchyDump(t, dump)
+	fromWebRuntime := factsFromWebRuntime(ctx, t, d)
+
+	requireEveryElementNamed(t, "the hierarchy dump", fromDump)
+	requireEveryElementNamed(t, "the web runtime", fromWebRuntime)
+	requireBothPolarities(t, fromWebRuntime)
+	compareEnumeratedElements(t, fromDump, fromWebRuntime)
+	compareDerivedFacts(t, fromDump, fromWebRuntime)
+}
+
+// factsFromHierarchyDump reads the dump the way the goja host does: parse it,
+// then take exactly the fields internal/verifier/worker.go targets() reads off
+// each element.
+func factsFromHierarchyDump(t *testing.T, dump string) []factRow {
+	t.Helper()
+	tree, err := hierarchy.Parse(dump)
+	if err != nil {
+		t.Fatalf("parse hierarchy: %v", err)
+	}
+	rows := make([]factRow, 0, len(tree.Elements))
+	for _, element := range tree.Elements {
+		rows = append(rows, factRow{
+			id: element.ResourceID,
+			facts: elementFacts{
+				tag:        element.Attributes["tag"],
+				clickable:  element.Clickable,
+				enabled:    element.Enabled,
+				editable:   element.Editable,
+				scrollable: element.Attributes["scrollable"] == "true",
+				positiveBounds: hasPositiveBounds(
+					element.Bounds.Width(),
+					element.Bounds.Height(),
+				),
+			},
+		})
+	}
+	return rows
+}
+
+// factsFromWebRuntime installs pkg/spec/test/dom-facts-probe.ts into the page
+// and calls it. The probe is bundled through the production web bundler and
+// reports what the production collectTargets derives, so the facts come from the
+// shipped runtime rather than a copy of it; only the id-carrying readback is
+// test-only.
+func factsFromWebRuntime(
+	ctx context.Context,
+	t *testing.T,
+	d *Driver,
+) []factRow {
+	t.Helper()
+	specSource := filepath.Join(repoRootDir(t), "pkg", "spec")
+	probe, err := bundler.BundleWeb(bundler.WebOptions{
+		EntryFile:      filepath.Join(specSource, "test", "dom-facts-probe.ts"),
+		WebRuntimeFile: filepath.Join(specSource, "src", "web-runtime.ts"),
+	})
+	if err != nil {
+		t.Fatalf("bundle dom facts probe: %v", err)
+	}
+	if err := d.InstallBundle(ctx, probe.JavaScript); err != nil {
+		t.Fatalf("install dom facts probe: %v", err)
+	}
+
+	var encoded string
+	script := `JSON.stringify(window.__sanderlingDomFacts__())`
+	if err := chromedp.Run(d.tabCtx, chromedp.Evaluate(script, &encoded)); err != nil {
+		t.Fatalf("read web runtime facts: %v", err)
+	}
+	var wire []struct {
+		ID         string `json:"id"`
+		Tag        string `json:"tag"`
+		Clickable  bool   `json:"clickable"`
+		Enabled    bool   `json:"enabled"`
+		Editable   bool   `json:"editable"`
+		Scrollable bool   `json:"scrollable"`
+		Width      int    `json:"width"`
+		Height     int    `json:"height"`
+	}
+	if err := json.Unmarshal([]byte(encoded), &wire); err != nil {
+		t.Fatalf("decode web runtime facts: %v", err)
+	}
+	rows := make([]factRow, 0, len(wire))
+	for _, item := range wire {
+		rows = append(rows, factRow{
+			id: item.ID,
+			facts: elementFacts{
+				tag:            item.Tag,
+				clickable:      item.Clickable,
+				enabled:        item.Enabled,
+				editable:       item.Editable,
+				scrollable:     item.Scrollable,
+				positiveBounds: hasPositiveBounds(item.Width, item.Height),
+			},
+		})
+	}
+	return rows
+}
+
+// hasPositiveBounds is the positiveBounds fact of pkg/spec/src/targets.ts,
+// applied to both producers so the geometry comparison cannot drift from the
+// rule it stands in for.
+func hasPositiveBounds(width, height int) bool {
+	return width > 0 && height > 0
+}
+
+// requireEveryElementNamed fails when a producer enumerates an element the
+// fixture gave no id, because such an element cannot be joined and would sit in
+// the comparison invisibly.
+func requireEveryElementNamed(t *testing.T, producer string, rows []factRow) {
+	t.Helper()
+	for index, row := range rows {
+		if row.id == "" {
+			t.Fatalf(
+				"%s enumerated an unnamed <%s> at position %d; every element in "+
+					"testdata/fact-parity.html must carry an id to be comparable",
+				producer,
+				row.facts.tag,
+				index,
+			)
+		}
+	}
+}
+
+// requireBothPolarities keeps the comparison from going vacuous. A fixture, or
+// an emulated viewport, that leaves a fact constant would compare false against
+// false on every element and pass while proving nothing about that fact.
+func requireBothPolarities(t *testing.T, rows []factRow) {
+	t.Helper()
+	for _, fact := range []struct {
+		name string
+		read func(elementFacts) bool
+	}{
+		{"clickable", func(f elementFacts) bool { return f.clickable }},
+		{"enabled", func(f elementFacts) bool { return f.enabled }},
+		{"editable", func(f elementFacts) bool { return f.editable }},
+		{"scrollable", func(f elementFacts) bool { return f.scrollable }},
+		{"positiveBounds", func(f elementFacts) bool { return f.positiveBounds }},
+	} {
+		var sawTrue, sawFalse bool
+		for _, row := range rows {
+			if fact.read(row.facts) {
+				sawTrue = true
+			} else {
+				sawFalse = true
+			}
+		}
+		if !sawTrue || !sawFalse {
+			t.Errorf(
+				"testdata/fact-parity.html no longer exercises %s both ways (the web "+
+					"runtime saw true=%v, false=%v), so comparing it proves nothing",
+				fact.name,
+				sawTrue,
+				sawFalse,
+			)
+		}
+	}
+}
+
+// compareEnumeratedElements is the check that the two producers walk the same
+// document. It is what notices a producer that roots at body and never sees
+// `html`, or one that enumerates the head subtree the other drops.
+func compareEnumeratedElements(
+	t *testing.T,
+	fromDump, fromWebRuntime []factRow,
+) {
+	t.Helper()
+	dumpIDs := enumeratedIDs(fromDump)
+	webIDs := enumeratedIDs(fromWebRuntime)
+	for _, row := range fromWebRuntime {
+		if !slices.Contains(dumpIDs, row.id) {
+			t.Errorf(
+				"the hierarchy dump never enumerated %q (<%s>); the web runtime does, "+
+					"so the goja host cannot offer a single action on it",
+				row.id,
+				row.facts.tag,
+			)
+		}
+	}
+	for _, row := range fromDump {
+		if !slices.Contains(webIDs, row.id) {
+			t.Errorf(
+				"the web runtime never enumerated %q (<%s>); the hierarchy dump does, "+
+					"so the V8 host cannot offer a single action on it",
+				row.id,
+				row.facts.tag,
+			)
+		}
+	}
+	if len(dumpIDs) == len(webIDs) && !slices.Equal(dumpIDs, webIDs) {
+		t.Errorf(
+			"the two producers enumerate the same elements in different order\n"+
+				" dump=%v\n  web=%v",
+			dumpIDs,
+			webIDs,
+		)
+	}
+}
+
+// compareDerivedFacts compares the facts fact by fact, over every element both
+// producers saw, so a failure names the element and the fact that diverged.
+func compareDerivedFacts(t *testing.T, fromDump, fromWebRuntime []factRow) {
+	t.Helper()
+	webByID := map[string]elementFacts{}
+	for _, row := range fromWebRuntime {
+		webByID[row.id] = row.facts
+	}
+	for _, row := range fromDump {
+		web, ok := webByID[row.id]
+		if !ok {
+			continue
+		}
+		dump := row.facts
+		if dump.tag != web.tag {
+			t.Errorf(
+				"%q: the hierarchy dump calls it <%s>, the web runtime <%s>; the id "+
+					"join is resolving to different elements",
+				row.id,
+				dump.tag,
+				web.tag,
+			)
+		}
+		for _, fact := range []struct {
+			name string
+			dump bool
+			web  bool
+		}{
+			{"clickable", dump.clickable, web.clickable},
+			{"enabled", dump.enabled, web.enabled},
+			{"editable", dump.editable, web.editable},
+			{"scrollable", dump.scrollable, web.scrollable},
+			{"positiveBounds", dump.positiveBounds, web.positiveBounds},
+		} {
+			if fact.dump != fact.web {
+				t.Errorf(
+					"%q (<%s>): the hierarchy dump derives %s=%v, the web runtime "+
+						"derives %s=%v",
+					row.id,
+					dump.tag,
+					fact.name,
+					fact.dump,
+					fact.name,
+					fact.web,
+				)
+			}
+		}
+	}
+}
+
+func enumeratedIDs(rows []factRow) []string {
+	ids := make([]string, 0, len(rows))
+	for _, row := range rows {
+		ids = append(ids, row.id)
+	}
+	return ids
+}
+
+func repoRootDir(t *testing.T) string {
+	t.Helper()
+	_, thisFile, _, ok := runtime.Caller(0)
+	if !ok {
+		t.Fatal("cannot resolve caller path")
+	}
+	return filepath.Clean(
+		filepath.Join(filepath.Dir(thisFile), "..", "..", ".."),
+	)
+}

internal/driver/chrome/testdata/fact-parity.html

@@ -0,0 +1,72 @@
+<!doctype html>
+<html id="page">
+  <head id="page-head">
+    <meta id="page-charset" charset="utf-8" />
+    <title id="page-title">fact parity</title>
+    <style id="page-style">
+      :root {
+        --frame-w: 420px;
+        --frame-h: 640px;
+      }
+      body {
+        margin: 0;
+      }
+      #scroller {
+        width: 320px;
+        height: 120px;
+        overflow: auto;
+      }
+      #scroller-content {
+        height: 600px;
+      }
+      #sideways {
+        width: 320px;
+        height: 40px;
+        overflow: auto;
+      }
+      #sideways-content {
+        width: 900px;
+        height: 20px;
+      }
+      #fits {
+        width: 320px;
+        height: 120px;
+        overflow: auto;
+      }
+      #fits-content {
+        height: 40px;
+      }
+      #collapsed {
+        display: none;
+      }
+      #filler {
+        height: 900px;
+      }
+    </style>
+  </head>
+  <body id="page-body">
+    <button id="save">save</button>
+    <button id="cancel" disabled>cancel</button>
+    <a id="link" href="#detail">detail</a>
+    <select id="choice">
+      <option id="choice-first">first</option>
+    </select>
+    <input id="amount" type="text" value="10" />
+    <input id="agree" type="checkbox" />
+    <textarea id="notes"></textarea>
+    <div id="bio" contenteditable="true">bio</div>
+    <div id="menu" role="button">menu</div>
+    <div id="attribute-click" onclick="void 0">attribute click</div>
+    <div id="delegating-root">delegating root</div>
+    <div id="plain">plain</div>
+    <div id="edit-text-field" class="EditTextField">styled</div>
+    <span id="collapsed">collapsed</span>
+    <div id="scroller"><div id="scroller-content"></div></div>
+    <div id="sideways"><div id="sideways-content"></div></div>
+    <div id="fits"><div id="fits-content"></div></div>
+    <div id="filler"></div>
+    <script id="page-script">
+      document.getElementById("delegating-root").onclick = function () {};
+    </script>
+  </body>
+</html>

internal/driver/chrome/translate.go

@@ -68,7 +68,7 @@ func TranslateStringSelector(selector string) (string, bool, error) {
 //   - everything else passes through, including non-ASCII
 //
 // Callers should not pass this output into identifier contexts (class names,
-// tag names) — use an attribute selector form (`[class~="..."]`) instead.
+// tag names). Use an attribute selector form (`[class~="..."]`) instead.
 func cssEscape(value string) string {
 	var builder strings.Builder
 	builder.Grow(len(value))

internal/ltl/evaluator_test.go

@@ -45,9 +45,9 @@ func TestThunk_TransitionFromHoldToViolate(t *testing.T) {
 	wantSequence := []Verdict{
 		VerdictHolds,    // true
 		VerdictHolds,    // true
-		VerdictViolated, // false — latches
-		VerdictViolated, // true after violation — still violated
-		VerdictViolated, // true after violation — still violated
+		VerdictViolated, // false: latches
+		VerdictViolated, // true after violation: still violated
+		VerdictViolated, // true after violation: still violated
 	}
 	for index, want := range wantSequence {
 		got := evaluator.Observe()

internal/runner/llm_source.go

@@ -34,7 +34,7 @@ const (
 // screenshot, picks ONE number, and echoes that action so a mismatch can be
 // caught. The app-specific description (spec instructions) is appended.
 const llmSystemPrompt = "You are exercising a UI to find bugs. Each turn you get a screenshot and a numbered list of concrete actions, each with a weight hinting how much the test author wants it exercised (higher = more). " +
-	"Pick the ONE action most likely to make progress or expose a defect. Bugs often hide in repeated or rapid actions, so once a screen works, deliberately stress it — for example submitting the same form twice in a row to check it is not applied more than once — rather than only advancing. " +
+	"Pick the ONE action most likely to make progress or expose a defect. Bugs often hide in repeated or rapid actions, so once a screen works, deliberately stress it (for example submitting the same form twice in a row to check it is not applied more than once) rather than only advancing. " +
 	"Respond with your reasoning, the chosen number, and chosen_action copied verbatim from that line. For a typing action, also provide the text to enter."
 
 // llmSource selects each step's action with an OpenAI-compatible vision model

internal/runner/llm_source_test.go

@@ -16,6 +16,8 @@ import (
 	"strings"
 	"testing"
 
+	"github.com/priyanshujain/sanderling/internal/driver"
+	mockdriver "github.com/priyanshujain/sanderling/internal/driver/mock"
 	"github.com/priyanshujain/sanderling/internal/hierarchy"
 	"github.com/priyanshujain/sanderling/internal/llmclient"
 	"github.com/priyanshujain/sanderling/internal/verifier"
@@ -276,6 +278,100 @@ func TestPickSourcesSeededByDefault(t *testing.T) {
 	}
 }
 
+// seededFixtureSpec declares no generator = llm(...), so --generator llm has
+// nothing to build a picker from.
+const seededFixtureSpec = `
+import { always, taps, typing, weighted } from "@sanderling/spec";
+globalThis.properties = { ok: always(() => true) };
+globalThis.actions = weighted([1, taps], [1, typing]);
+`
+
+func newSeededVerifier(t *testing.T) *verifier.Verifier {
+	t.Helper()
+	verifierInstance, err := verifier.New()
+	if err != nil {
+		t.Fatal(err)
+	}
+	if err := verifierInstance.Load(bundleSpec(t, seededFixtureSpec)); err != nil {
+		t.Fatal(err)
+	}
+	if _, ok := verifierInstance.LLMConfig(); ok {
+		t.Fatal("seeded fixture spec must not register an llm action backend")
+	}
+	return verifierInstance
+}
+
+// TestPickSourcesRejectsLLMWithoutSpecConfig pins the abort. Falling back to the
+// seeded picker here completes the run, writes a well-formed output directory,
+// and records it under the requested arm, so a campaign cell silently reports
+// the wrong policy's numbers.
+func TestPickSourcesRejectsLLMWithoutSpecConfig(t *testing.T) {
+	for name, activeDriver := range map[string]driver.DeviceDriver{
+		"native": nil,
+		"web":    &webMockDriver{Driver: mockdriver.New()},
+	} {
+		t.Run(name, func(t *testing.T) {
+			action, extractor, err := pickSources(Options{
+				Driver:    activeDriver,
+				Verifier:  newSeededVerifier(t),
+				Generator: "llm",
+				Logger:    slog.New(slog.NewTextHandler(io.Discard, nil)),
+			})
+			if err == nil {
+				t.Fatalf("pickSources = (%T, %T), want an error", action, extractor)
+			}
+			if action != nil || extractor != nil {
+				t.Errorf("sources = (%v, %v), want both nil alongside the error", action, extractor)
+			}
+			if !strings.Contains(err.Error(), "generator = llm(...)") {
+				t.Errorf("error = %q, want it to name the missing spec declaration", err)
+			}
+		})
+	}
+}
+
+// TestPickSourcesOnWebComposesLLMWithWebExtractors covers the second half of the
+// same claim: the picker is chosen by --generator and the extractor source by
+// the driver, so the llm policy runs on web instead of being silently replaced
+// by the V8 picker.
+func TestPickSourcesOnWebComposesLLMWithWebExtractors(t *testing.T) {
+	fake := newFakeOpenRouter(t)
+	_, verifierInstance := newLLMSource(t, fake)
+	action, extractor, err := pickSources(Options{
+		Driver:    &webMockDriver{Driver: mockdriver.New()},
+		Verifier:  verifierInstance,
+		Generator: "llm",
+		Logger:    slog.New(slog.NewTextHandler(io.Discard, nil)),
+	})
+	if err != nil {
+		t.Fatalf("pickSources: %v", err)
+	}
+	if _, ok := action.(*llmSource); !ok {
+		t.Errorf("action source = %T, want *llmSource on web with --generator llm", action)
+	}
+	if _, ok := extractor.(webSource); !ok {
+		t.Errorf("extractor source = %T, want webSource so overrides still come from V8", extractor)
+	}
+}
+
+func TestPickSourcesOnWebSeededKeepsBothOnV8(t *testing.T) {
+	action, extractor, err := pickSources(Options{
+		Driver:    &webMockDriver{Driver: mockdriver.New()},
+		Verifier:  newSeededVerifier(t),
+		Generator: "seeded",
+		Logger:    slog.New(slog.NewTextHandler(io.Discard, nil)),
+	})
+	if err != nil {
+		t.Fatalf("pickSources: %v", err)
+	}
+	if _, ok := action.(webSource); !ok {
+		t.Errorf("action source = %T, want webSource for the seeded web path", action)
+	}
+	if _, ok := extractor.(webSource); !ok {
+		t.Errorf("extractor source = %T, want webSource for the seeded web path", extractor)
+	}
+}
+
 func TestLLMSourceDrivesExecutedActions(t *testing.T) {
 	fake := newFakeOpenRouter(t)
 	source, verifierInstance := newLLMSource(t, fake)

internal/runner/source.go

@@ -3,6 +3,7 @@ package runner
 import (
 	"context"
 	"encoding/json"
+	"errors"
 	"fmt"
 	"log/slog"
 
@@ -64,39 +65,48 @@ func (s webSource) ExtractorOverrides(ctx context.Context) (map[int]json.RawMess
 
 // pickSources selects the runtime's action and extractor sources ONCE at setup
 // from the driver's capabilities, the --generator flag, and the spec, so the
-// step loop never type-asserts. With --generator llm and a spec-declared
-// generator = llm({...}) it constructs the chat-completions client and returns
-// an llmSource for selection while extractor overrides still come from the goja
-// path. Otherwise the seeded goja picker drives.
+// step loop never type-asserts.
+//
+// The two axes are independent. The DRIVER decides where extractor overrides
+// come from: the web path reads the values its extractors computed in V8, every
+// other path has none. The --generator flag decides who picks the action, and
+// the llm picker composes with either extractor source because it reads the
+// goja-side candidate list and screenshot, both of which the runner populates on
+// every platform.
+//
+// A missing generator = llm(...) is fatal rather than a fallback to the seeded
+// picker. Falling back silently corrupts a comparison campaign: the run
+// completes and the output directory looks correct while the wrong policy drove
+// it.
 func pickSources(options Options) (ActionSource, ExtractorSource, error) {
+	seeded := ActionSource(gojaSource{verifier: options.Verifier})
+	extractor := ExtractorSource(gojaSource{verifier: options.Verifier})
 	if web, ok := options.Driver.(driver.WebDriver); ok {
 		source := webSource{web: web}
-		return source, source, nil
+		seeded, extractor = source, source
+	}
+	if options.Generator != "llm" {
+		return seeded, extractor, nil
+	}
+	config, ok := options.Verifier.LLMConfig()
+	if !ok {
+		return nil, nil, errors.New("--generator llm: the spec declares no generator = llm(...); add one to the spec or drop --generator llm")
+	}
+	client, err := llmclient.New()
+	if err != nil {
+		return nil, nil, fmt.Errorf("llm action generator: %w", err)
 	}
 	logger := options.Logger
 	if logger == nil {
 		logger = slog.Default()
 	}
-	if options.Generator == "llm" {
-		config, ok := options.Verifier.LLMConfig()
-		if !ok {
-			logger.Warn("--generator llm requested but spec declares no generator = llm(); using the seeded picker")
-		} else {
-			client, err := llmclient.New()
-			if err != nil {
-				return nil, nil, fmt.Errorf("llm action generator: %w", err)
-			}
-			action := &llmSource{
-				verifier:     options.Verifier,
-				client:       client,
-				model:        config.Model,
-				instructions: config.Instructions,
-				logger:       logger,
-				history:      newActionHistory(llmHistorySize),
-			}
-			return action, gojaSource{verifier: options.Verifier}, nil
-		}
+	action := &llmSource{
+		verifier:     options.Verifier,
+		client:       client,
+		model:        config.Model,
+		instructions: config.Instructions,
+		logger:       logger,
+		history:      newActionHistory(llmHistorySize),
 	}
-	source := gojaSource{verifier: options.Verifier}
-	return source, source, nil
+	return action, extractor, nil
 }

internal/sidecarassets/embed.go

@@ -39,11 +39,35 @@ func Extract(dir string) (string, error) {
 		}
 	}
 
-	if err := os.WriteFile(jarPath, embeddedJAR, 0o644); err != nil {
+	if err := writeAtomic(jarPath, embeddedJAR); err != nil {
 		return "", fmt.Errorf("write %s: %w", jarPath, err)
 	}
-	if err := os.WriteFile(checksumPath, []byte(checksum), 0o644); err != nil {
+	if err := writeAtomic(checksumPath, []byte(checksum)); err != nil {
 		return "", fmt.Errorf("write checksum: %w", err)
 	}
 	return jarPath, nil
 }
+
+// writeAtomic publishes content at path through a rename, so a reader never
+// observes a partial file. dir is shared between every sanderling process on
+// the host, so a campaign running one worker per device has several of them
+// extracting the same JAR at once on a cold host; a plain write let one
+// process spawn a JVM against another's half-written file.
+func writeAtomic(path string, content []byte) error {
+	temporary, err := os.CreateTemp(filepath.Dir(path), filepath.Base(path)+".*.partial")
+	if err != nil {
+		return err
+	}
+	defer os.Remove(temporary.Name())
+	if _, err := temporary.Write(content); err != nil {
+		temporary.Close()
+		return err
+	}
+	if err := temporary.Close(); err != nil {
+		return err
+	}
+	if err := os.Chmod(temporary.Name(), 0o644); err != nil {
+		return err
+	}
+	return os.Rename(temporary.Name(), path)
+}

internal/sidecarassets/embed_withsidecar_test.go

@@ -6,8 +6,11 @@ import (
 	"bytes"
 	"crypto/sha256"
 	"encoding/hex"
+	"fmt"
 	"os"
 	"path/filepath"
+	"strings"
+	"sync"
 	"testing"
 )
 
@@ -89,3 +92,76 @@ func TestExtract_RewritesIfChecksumMissing(t *testing.T) {
 		t.Errorf("checksum should have been written: %v", err)
 	}
 }
+
+// TestExtract_ReadersNeverSeeAPartialJAR covers the cold-host case a parallel
+// campaign hits: every worker process extracts into the same shared directory
+// under os.TempDir, and one of them spawns a JVM against the JAR while another
+// is still writing it. A plain write truncates the file in place, so the
+// reader gets a short JAR and the JVM dies on a corrupt archive.
+func TestExtract_ReadersNeverSeeAPartialJAR(t *testing.T) {
+	directory := t.TempDir()
+	want := EmbeddedSHA256()
+	jarPath, err := Extract(directory)
+	if err != nil {
+		t.Fatal(err)
+	}
+	checksumPath := jarPath + ".sha256"
+
+	stop := make(chan struct{})
+	torn := make(chan string, 1)
+	var readers sync.WaitGroup
+	for range 4 {
+		readers.Add(1)
+		go func() {
+			defer readers.Done()
+			for {
+				select {
+				case <-stop:
+					return
+				default:
+				}
+				content, err := os.ReadFile(jarPath)
+				if err != nil {
+					continue
+				}
+				sum := sha256.Sum256(content)
+				if got := hex.EncodeToString(sum[:]); got != want {
+					select {
+					case torn <- fmt.Sprintf("read %d bytes, sha256 %s", len(content), got):
+					default:
+					}
+					return
+				}
+			}
+		}()
+	}
+
+	for range 8 {
+		// Dropping the checksum is what a cold host looks like: no extraction
+		// has been recorded, so every caller writes the JAR again.
+		if err := os.Remove(checksumPath); err != nil && !os.IsNotExist(err) {
+			t.Fatal(err)
+		}
+		if _, err := Extract(directory); err != nil {
+			t.Fatal(err)
+		}
+	}
+	close(stop)
+	readers.Wait()
+
+	select {
+	case detail := <-torn:
+		t.Fatalf("a reader observed a partial JAR while another caller extracted: %s", detail)
+	default:
+	}
+
+	entries, err := os.ReadDir(directory)
+	if err != nil {
+		t.Fatal(err)
+	}
+	for _, entry := range entries {
+		if strings.Contains(entry.Name(), ".partial") {
+			t.Errorf("Extract left a staging file behind: %s", entry.Name())
+		}
+	}
+}

internal/testrun/testrun.go

@@ -31,9 +31,13 @@ type Options struct {
 	IosAppPath     string
 	AndroidAppPath string
 	Duration       time.Duration
+	MaxSteps       int
 	Seed           int64
 	Output         string
 	ClearData      bool
+	// Arm labels the experiment cell this run belongs to and is recorded in
+	// meta.json so a directory of runs can be attributed to a cell.
+	Arm string
 	// Generator selects the action picker: "llm" or the default seeded picker.
 	Generator string
 
@@ -47,6 +51,32 @@ type Options struct {
 }
 
 // Execute runs the full test pipeline: bundle, launch app, verify properties.
+// buildRunMeta assembles the run's meta.json. Model and Instructions are
+// recorded only when the LLM picker is the one that will actually run, so a
+// spec that declares generator = llm() but is run under the seeded picker does
+// not label its trace with a model it never called.
+func buildRunMeta(options Options, bundleSHA256 string, seed int64, host string, llmConfig verifier.LLMConfig, hasLLMConfig bool) trace.Meta {
+	meta := trace.Meta{
+		Seed:              seed,
+		SpecPath:          options.Spec,
+		BundleSHA256:      bundleSHA256,
+		Platform:          options.Platform,
+		BundleID:          options.BundleID,
+		StartedAt:         time.Now().UTC(),
+		SanderlingVersion: "0.0.1",
+		Arm:               options.Arm,
+		Generator:         options.Generator,
+		MaxSteps:          options.MaxSteps,
+		DurationMillis:    options.Duration.Milliseconds(),
+		Host:              host,
+	}
+	if options.Generator == "llm" && hasLLMConfig {
+		meta.Model = llmConfig.Model
+		meta.Instructions = llmConfig.Instructions
+	}
+	return meta
+}
+
 func Execute(ctx context.Context, options Options, stdout io.Writer) error {
 	switch options.Platform {
 	case "android":
@@ -140,15 +170,9 @@ func Execute(ctx context.Context, options Options, stdout io.Writer) error {
 		return fmt.Errorf("trace writer: %w", err)
 	}
 	defer traceWriter.Close()
-	meta := trace.Meta{
-		Seed:              seed,
-		SpecPath:          options.Spec,
-		BundleSHA256:      bundle.SHA256,
-		Platform:          options.Platform,
-		BundleID:          options.BundleID,
-		StartedAt:         time.Now().UTC(),
-		SanderlingVersion: "0.0.1",
-	}
+	hostname, _ := os.Hostname()
+	llmConfig, hasLLMConfig := verifierInstance.LLMConfig()
+	meta := buildRunMeta(options, bundle.SHA256, seed, hostname, llmConfig, hasLLMConfig)
 	if err := traceWriter.WriteMeta(meta); err != nil {
 		return fmt.Errorf("trace meta: %w", err)
 	}
@@ -159,9 +183,14 @@ func Execute(ctx context.Context, options Options, stdout io.Writer) error {
 	}()
 	fmt.Fprintf(stdout, "trace dir: %s\n", runDirectory)
 
-	fmt.Fprintf(stdout, "running for %s (seed=%d)\n", options.Duration, seed)
+	if options.MaxSteps > 0 {
+		fmt.Fprintf(stdout, "running for %s or %d steps, whichever comes first (seed=%d)\n", options.Duration, options.MaxSteps, seed)
+	} else {
+		fmt.Fprintf(stdout, "running for %s (seed=%d)\n", options.Duration, seed)
+	}
 	summary, err := runner.Run(ctx, runner.Options{
 		Duration:    options.Duration,
+		MaxSteps:    options.MaxSteps,
 		IdleTimeout: 1 * time.Second,
 		BundleID:    options.BundleID,
 		Driver:      activeDriver,

internal/testrun/testrun_test.go

@@ -5,6 +5,9 @@ import (
 	"path/filepath"
 	"strings"
 	"testing"
+	"time"
+
+	"github.com/priyanshujain/sanderling/internal/verifier"
 )
 
 func TestResolveSeed_UsesConfiguredWhenNonZero(t *testing.T) {
@@ -181,3 +184,50 @@ func TestResolveSpecAPIPath_ReturnsEmptyWhenMissing(t *testing.T) {
 		t.Fatalf("got %q, want empty (no sanderling source tree reachable)", got)
 	}
 }
+
+func TestBuildRunMeta_RecordsArmMembership(t *testing.T) {
+	options := Options{
+		Spec:      "spec.ts",
+		BundleID:  "com.example",
+		Platform:  "android",
+		Duration:  3 * time.Minute,
+		MaxSteps:  300,
+		Arm:       "llm-visible-text",
+		Generator: "llm",
+	}
+	meta := buildRunMeta(options, "deadbeef", 7, "farm-01",
+		verifier.LLMConfig{Model: "claude-sonnet-5", Instructions: "exercise the outbox"}, true)
+
+	if meta.Arm != "llm-visible-text" || meta.Generator != "llm" {
+		t.Errorf("arm membership: got arm=%q generator=%q", meta.Arm, meta.Generator)
+	}
+	if meta.Model != "claude-sonnet-5" || meta.Instructions != "exercise the outbox" {
+		t.Errorf("llm config: got model=%q instructions=%q", meta.Model, meta.Instructions)
+	}
+	if meta.MaxSteps != 300 || meta.DurationMillis != 180000 {
+		t.Errorf("budget: got maxSteps=%d durationMillis=%d", meta.MaxSteps, meta.DurationMillis)
+	}
+	if meta.Host != "farm-01" || meta.Seed != 7 {
+		t.Errorf("host and seed: got host=%q seed=%d", meta.Host, meta.Seed)
+	}
+}
+
+func TestBuildRunMeta_OmitsModelWhenSeededPickerRuns(t *testing.T) {
+	options := Options{Platform: "android", Generator: "seeded", Duration: time.Minute}
+	meta := buildRunMeta(options, "deadbeef", 1, "farm-01",
+		verifier.LLMConfig{Model: "claude-sonnet-5", Instructions: "hunt bugs"}, true)
+
+	if meta.Model != "" || meta.Instructions != "" {
+		t.Errorf("a seeded run must not be labelled with a model it never called: model=%q instructions=%q",
+			meta.Model, meta.Instructions)
+	}
+}
+
+func TestBuildRunMeta_OmitsModelWhenSpecDeclaresNoLLMGenerator(t *testing.T) {
+	options := Options{Platform: "android", Generator: "llm", Duration: time.Minute}
+	meta := buildRunMeta(options, "deadbeef", 1, "farm-01", verifier.LLMConfig{}, false)
+
+	if meta.Model != "" {
+		t.Errorf("model recorded without a spec-declared llm generator: %q", meta.Model)
+	}
+}

internal/trace/writer.go

@@ -92,7 +92,7 @@ type Action struct {
 	// LLMChoice is the 1-based number the model picked from the candidate list;
 	// LLMChosenAction is the action description it echoed for that number. The
 	// runner strict-skips when the echo disagrees with the numbered entry, so on
-	// a recorded action the two always agree — the replay UI shows them to
+	// a recorded action the two always agree; the replay UI shows them to
 	// confirm the reasoning matched the executed action.
 	LLMChoice       int    `json:"llm_choice,omitempty"`
 	LLMChosenAction string `json:"llm_chosen_action,omitempty"`
@@ -126,6 +126,24 @@ type Meta struct {
 	StartedAt         time.Time  `json:"started_at"`
 	EndedAt           *time.Time `json:"ended_at,omitempty"`
 	SanderlingVersion string     `json:"sanderling_version"`
+
+	// Arm labels the experiment cell this run belongs to, set from --arm. A
+	// directory of runs cannot be attributed to a cell after the fact without
+	// it, which makes any factorial computed from such a directory unanalysable.
+	Arm string `json:"arm,omitempty"`
+	// Generator, Model and Instructions record which picker ran and how it was
+	// configured. Two runs that differ in any of these are different arms.
+	Generator    string `json:"generator,omitempty"`
+	Model        string `json:"model,omitempty"`
+	Instructions string `json:"instructions,omitempty"`
+	// MaxSteps and DurationMillis are the budget the run was given, which has
+	// to be identical across arms for a comparison to mean anything.
+	MaxSteps       int   `json:"max_steps,omitempty"`
+	DurationMillis int64 `json:"duration_millis,omitempty"`
+	// Host is the machine that produced the run. Campaigns are split across
+	// several hosts, so a per-host effect has to be detectable rather than
+	// invisible.
+	Host string `json:"host,omitempty"`
 }
 
 type Writer struct {

internal/trace/writer_test.go

@@ -429,3 +429,66 @@ func readLines(t *testing.T, path string) []string {
 	}
 	return lines
 }
+
+func TestWriteMeta_ArmMembershipRoundTrip(t *testing.T) {
+	directory := t.TempDir()
+	writer, err := NewWriter(directory)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer writer.Close()
+
+	meta := Meta{
+		Seed:              7,
+		SpecPath:          "spec.ts",
+		BundleSHA256:      "deadbeef",
+		Platform:          "android",
+		BundleID:          "com.example",
+		StartedAt:         time.Date(2026, 8, 12, 10, 0, 0, 0, time.UTC),
+		SanderlingVersion: "0.0.1",
+		Arm:               "llm-visible-text",
+		Generator:         "llm",
+		Model:             "claude-sonnet-5",
+		Instructions:      "exercise the outbox",
+		MaxSteps:          300,
+		DurationMillis:    180000,
+		Host:              "emulator-farm-01",
+	}
+	if err := writer.WriteMeta(meta); err != nil {
+		t.Fatal(err)
+	}
+
+	body, err := os.ReadFile(filepath.Join(directory, "meta.json"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	var got Meta
+	if err := json.Unmarshal(body, &got); err != nil {
+		t.Fatalf("meta.json is not valid JSON: %v\n%s", err, body)
+	}
+	if got != meta {
+		t.Errorf("meta round-trip mismatch:\n got: %+v\nwant: %+v", got, meta)
+	}
+}
+
+func TestWriteMeta_OmitsArmMembershipWhenUnset(t *testing.T) {
+	directory := t.TempDir()
+	writer, err := NewWriter(directory)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer writer.Close()
+
+	if err := writer.WriteMeta(Meta{Seed: 1, Platform: "web"}); err != nil {
+		t.Fatal(err)
+	}
+	body, err := os.ReadFile(filepath.Join(directory, "meta.json"))
+	if err != nil {
+		t.Fatal(err)
+	}
+	for _, key := range []string{"arm", "generator", "model", "instructions", "max_steps", "duration_millis", "host"} {
+		if strings.Contains(string(body), `"`+key+`"`) {
+			t.Errorf("meta.json carries %q when unset:\n%s", key, body)
+		}
+	}
+}

pkg/spec/src/ltl.ts

@@ -12,8 +12,8 @@ export function next(predicate: () => boolean): Formula {
   return globalThis.__sanderling__.next(predicate);
 }
 
-// An unbounded `eventually` never forces a violation within a finite run —
-// prefer `.within(n, unit)` when you want the verifier to fail a property
+// An unbounded `eventually` never forces a violation within a finite run.
+// Prefer `.within(n, unit)` when you want the verifier to fail a property
 // that stalls.
 export function eventually(predicate: () => boolean): EventuallyFormula {
   return globalThis.__sanderling__.eventually(predicate);

pkg/spec/src/runtime-entry.ts

@@ -151,7 +151,7 @@ export function installRuntime(
   defineLockedGlobal("__sanderlingExtractors__", () => evaluateExtractors());
   // __sanderlingSetupAction__ walks ONLY the setup generator once, for the LLM
   // action generator (Go), which drives selection itself and must not run the
-  // seeded action root — but still wants setup's precondition steps (e.g. login)
+  // seeded action root, but still wants setup's precondition steps (e.g. login)
   // to run first. Returns null when setup is unset or yields nothing.
   defineLockedGlobal("__sanderlingSetupAction__", () => {
     resolveRoot();

pkg/spec/src/types.ts

@@ -113,7 +113,7 @@ export interface State {
 
 /**
  * State as observed inside a web (V8/browser) extractor. Adds the live
- * `document` and `window` handles. `state.document` is V8-only — goja-side
+ * `document` and `window` handles. `state.document` is V8-only, so goja-side
  * predicates do not see it; if you need DOM data in a predicate, surface it
  * via an `extract()`.
  */

pkg/spec/src/web-runtime.ts

@@ -514,6 +514,21 @@ function pointOf(element: Element): Candidate {
   };
 }
 
+// HEAD_SELECTOR is the one subtree the enumeration leaves out. It never renders,
+// so no verb can reach it, and the hierarchy dump the goja host reads
+// (internal/driver/chrome/driver.go) drops it as well. Enumerating it here would
+// put the two hosts on different element sets for every page that has a <head>.
+const HEAD_SELECTOR = "head, head *";
+
+// targetElements is the walk the target list is built from: the document in
+// pre-order, minus the head subtree.
+function targetElements(): HTMLElement[] {
+  const inHead = new Set<Element>(Array.from(document.querySelectorAll(HEAD_SELECTOR)));
+  return Array.from(document.querySelectorAll<HTMLElement>("*")).filter(
+    (element) => !inHead.has(element),
+  );
+}
+
 // collectTargets walks the document ONCE and reports every element with the facts
 // the shared eligibility rule reads. The tappable/editable membership sets are
 // resolved by selector first so the DOM's answer to "clickable" and "editable"
@@ -525,7 +540,7 @@ function collectTargets(): TargetElement[] {
       isEditableElement,
     ),
   );
-  return Array.from(document.querySelectorAll<HTMLElement>("*")).map((element) => ({
+  return targetElements().map((element) => ({
     ...pointOf(element),
     clickable: clickable.has(element),
     enabled: !(element as HTMLButtonElement).disabled,
@@ -577,6 +592,7 @@ export const __testing__ = {
   host,
   seedBigInt,
   collectTargets,
+  targetElements,
   TAPPABLE_SELECTOR,
   EDITABLE_SELECTOR,
   resetTargetCache,

pkg/spec/test/dom-facts-probe.ts

@@ -0,0 +1,41 @@
+/// <reference lib="dom" />
+
+// The V8 host's fact producer, read back element by element.
+//
+// internal/driver/chrome/fact_parity_test.go bundles this probe into a live page
+// and compares what it returns against the hierarchy dump the goja host reads
+// off the SAME page. Both sides derive their facts here, in production code:
+// collectTargets is the shipped walk, and targetElements is the shipped
+// enumeration domain, so the id at index i names the element facts[i] describes.
+// The dump carries that id as `resource-id`, which is what the Go side joins on.
+
+import { __testing__ } from "../src/web-runtime.ts";
+
+const { collectTargets, targetElements } = __testing__;
+
+function domFacts(): unknown[] {
+  const elements = targetElements();
+  const facts = collectTargets();
+  if (elements.length !== facts.length) {
+    throw new Error(
+      `collectTargets reported ${facts.length} targets over ${elements.length} elements`,
+    );
+  }
+  return elements.map((element, index) => {
+    const target = facts[index]!;
+    return {
+      id: element.id,
+      tag: element.tagName.toLowerCase(),
+      clickable: target.clickable,
+      enabled: target.enabled,
+      editable: target.editable,
+      scrollable: target.scrollable,
+      width: target.width ?? 0,
+      height: target.height ?? 0,
+    };
+  });
+}
+
+type FactsGlobal = { __sanderlingDomFacts__: () => unknown[] };
+
+(globalThis as unknown as FactsGlobal).__sanderlingDomFacts__ = domFacts;

replay-ui/src/__tests__/action-format.test.ts

@@ -7,7 +7,7 @@ import {
 } from "../lib/action-format";
 import { summary } from "./fixtures";
 
-// Bug class: selector parsing mislabels every action row — splitting on the
+// Bug class: selector parsing mislabels every action row. Splitting on the
 // wrong colon, treating a value-with-colon as the kind, or dropping the prefix
 // ellipsis would render the wrong target tag for every step.
 describe("parseSelector", () => {

replay-ui/src/hooks/useTheme.ts

@@ -28,7 +28,7 @@ export function useTheme(): { theme: ThemeMode; toggle: () => void } {
     try {
       window.localStorage.setItem(STORAGE_KEY, theme);
     } catch {
-      // private mode or storage disabled — theme still applies for the session.
+      // private mode or storage disabled. Theme still applies for the session.
     }
   }, [theme]);
 

replay-ui/src/lib/run-history.ts

@@ -18,8 +18,8 @@ export interface RunHistory {
 // that armed it, so the checker records the violation on the DETECTION step
 // while its witness names the CAUSE step. The timeline dot (the backend's
 // markViolations) already sits on the cause step; mirror that here so the
-// Violations tab and property lanes light up on the same step — the guilty
-// action — not the unrelated action that happened to be running when the
+// Violations tab and property lanes light up on the same step: the guilty
+// action, not the unrelated action that happened to be running when the
 // obligation resolved. Steps are cloned, never mutated in place.
 export function relocateViolationsToCause(
   steps: (Step | null)[],

sidecar/build.gradle.kts

@@ -26,7 +26,7 @@ val maestroVersion = "2.4.0"
 // Maestro 2.4.0 still declares grpc-netty:1.50.2 and grpc-okhttp:1.50.2 which
 // were compiled against AbstractManagedChannelImplBuilder removed in grpc-core 1.64+.
 // Force them to the version we pin so they stay binary-compatible.
-// Also exclude GraalVM JS — not used by our sidecar and causes shadow JAR expansion errors.
+// Also exclude GraalVM JS: not used by our sidecar and causes shadow JAR expansion errors.
 configurations.all {
     resolutionStrategy.eachDependency {
         if (requested.group == "io.grpc" &&

test/browser/browser_test.go

@@ -130,8 +130,8 @@ func bundleSpec(t *testing.T, specPath string) (goja, web []byte) {
 	t.Helper()
 	specSrc := specSrcDir(t)
 	aliases := map[string]string{
-		"@sanderling/spec":                    filepath.Join(specSrc, "index.ts"),
-		"@sanderling/spec/defaults":           filepath.Join(specSrc, "defaults/index.ts"),
+		"@sanderling/spec":                     filepath.Join(specSrc, "index.ts"),
+		"@sanderling/spec/defaults":            filepath.Join(specSrc, "defaults/index.ts"),
 		"@sanderling/spec/defaults/properties": filepath.Join(specSrc, "defaults/properties.ts"),
 	}
 	defines := map[string]string{"SANDERLING_SEED": strconv.Itoa(fixtureSeed)}