Files
sanderling/pkg/spec/test/api.test.ts
pj 26b49b379a fix ltl semantics and unify action enumeration (#71)
* fix(ltl): give every thunk a construction identity

Two distinct unnamed predicates both described as "Thunk(...)", so obligation
collapse merged their residuals and could drop a live violation. Identity is
assigned at construction and the fields are unexported, so a thunk cannot be
built without one.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* fix(ltl): reduce a thrown-predicate residual instead of panicking

The verifier substitutes an ErrorFormula for the residual of a property whose
predicate threw, and that residual is fed back in on the next step. reduce had
no case for it, so the run crashed. It re-reports the same failure now.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* fix(ltl): make a bounded always the dual of a bounded eventually

G<=n(f) and not F<=n(not f) disagreed on traces where the inner was still
pending when the window closed, so nnf's negation normal form was not semantics
preserving. Both sides now range over the observations at which their inner can
definitely resolve: the eventually keeps a pending inner as a disjunct, and the
always discharges vacuously at window close.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* fix(ltl): arm a one-shot root once per run

A root that carries its own horizon is one obligation for the whole run, not one
per observation. Re-instantiating a top-level eventually monitored G F<=n(p)
instead of F<=n(p) and left one live obligation per step behind; a bounded
always restarted its window every step and never closed.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* fix(verifier): stop wrapping a top-level eventually in always

`eventually(p).within(300, "seconds")` as a property meant "within 300 seconds
of every step", which spawned an obligation per step with its own resolved
deadline. A 553-step run carried 553 of them and serialized a 75 KB residual.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* fix(ltl): serialize the resolved deadline of a bounded window

Two obligations spawned at different steps from one duration-bounded formula
differ only in the deadline the evaluator resolved for them, so they serialized
identically and the trace erased a distinction the evaluator makes. The authored
window stays in amount/unit; the resolved deadline rides alongside.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* fix(verifier): split a witness's origin step from its detection step

A deferred obligation spans two steps: the one that armed it and the one whose
reduction failed. They were conflated under one index, so the extractor snapshot
(which is the detecting step's state) was reported against the origin step.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* fix(runner): record a witness's detection step in the trace

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* feat(replay-ui): show the step a violation was detected at

The witness evidence is the detecting step's state, so say which step that is
and let a reader jump to it.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* fix(verifier): record the extractor state the predicates actually read

On the web path extractor bodies are evaluated in V8 and injected here, but only
the goja value was replaced. The trace diff and the violation witness therefore
described a state no property ever saw.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* refactor(spec): one candidate producer over one target-eligibility rule

Both hosts routed verbs themselves and both policies enumerated their own
actions, and all four drifted. Web sent `swipes` to scrollable containers only,
so swipe-to-dismiss on a list row was reachable on native and unreachable on
web; the model policy folded gestures its own way and could not reach what the
seeded picker drew.

A host now reports facts about every element and never decides which verb may
act on it: targets.ts acceptsTarget owns that for both. pick.ts builtinCandidates
is the single enumeration, and the model policy reads it through
__sanderlingEnumerateBuiltin__ instead of reimplementing it in Go.

Gesture verbs change with it: scrolls stay vertical over scrollable containers,
swipes go free-form in all four directions from any element with real bounds.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* fix(runner): name a builtin scroll by its drag origin

A builtin gesture carries endpoints and no selector, so every scroll rendered as
"Scroll down " in the prompt's recent-action memory and two scrollable regions
were indistinguishable.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* fix(chrome): clear storage over cdp instead of scripting an opaque origin

Launch runs while the tab is still on about:blank, whose opaque origin denies
storage access, so localStorage.clear() threw SecurityError and every web run
died at launch. Storage.clearDataForOrigin needs no navigation. The exception
helper lands here because "Uncaught" is what hid this for so long.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* fix(chrome): enable the swiftshader webgl fallback

Headless Chrome runs with --disable-gpu, and without this flag it refuses the
software WebGL backend: getContext returns null, so a canvas-rendered app paints
nothing and every screenshot is identical black.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* fix(web): resolve testTag through data-testid or id

Compose Multiplatform emits its testTag into the element id, which the native
table already accepts via the resource-id alias. The two web selector tables
were the only place that rejected it.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* test(spec): type-check the spec api as part of make test

The fake runtime in api.test.ts did not return a chainable handle from extract,
so the file had not type-checked since named() was added. Wiring the check into
make test stops it drifting again.

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J

* docs(manual): one-shot eventually and the gesture verbs

Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J
2026-08-12 18:06:04 +05:30

400 lines
12 KiB
TypeScript

import assert from "node:assert/strict";
import { test } from "node:test";
import {
DoubleTap,
InputText,
LongPress,
PressKey,
Scroll,
Swipe,
Tap,
Wait,
actions,
always,
doubleTaps,
eventually,
extract,
from,
keyedBy,
longPresses,
next,
now,
pressKeys,
scrolls,
swipes,
taps,
typing,
waitOnce,
weighted,
whenRoute,
} from "../src/index.ts";
import { setSamplerRng } from "../src/actions.ts";
import { Pcg } from "../src/pcg.ts";
import type { GeneratorNode } from "../src/action-tree.ts";
import type {
AccessibilityElement,
EventuallyFormula,
Extracted,
Formula,
State,
SanderlingRuntime,
} from "../src/types.ts";
// extract() and the LTL constructors stay host-bound on __sanderling__; the
// action factories now return plain data and need no runtime. This fake records
// the host-bound calls so the forwarding tests can assert them.
interface RecordedRuntime extends SanderlingRuntime {
extracts: Array<(state: State) => unknown>;
extractNames: Array<string | undefined>;
alwaysArgs: Array<(() => boolean) | Formula>;
nowPredicates: Array<() => boolean>;
nextPredicates: Array<() => boolean>;
eventuallyPredicates: Array<() => boolean>;
withinCalls: Array<{ amount: number; unit: string }>;
impliesCalls: number;
orCalls: number;
andCalls: number;
notCalls: number;
}
function makeChainableFormula(record: RecordedRuntime): Formula {
return {
__sanderlingFormula: true,
implies(other: Formula): Formula {
record.impliesCalls++;
void other;
return makeChainableFormula(record);
},
or(other: Formula): Formula {
record.orCalls++;
void other;
return makeChainableFormula(record);
},
and(other: Formula): Formula {
record.andCalls++;
void other;
return makeChainableFormula(record);
},
not(): Formula {
record.notCalls++;
return makeChainableFormula(record);
},
};
}
function makeChainableEventually(record: RecordedRuntime): EventuallyFormula {
const base = makeChainableFormula(record);
return {
...base,
within(amount, unit) {
record.withinCalls.push({ amount, unit });
return makeChainableFormula(record);
},
};
}
function installFakeRuntime(): RecordedRuntime {
const calls = {
extracts: [] as Array<(state: State) => unknown>,
extractNames: [] as Array<string | undefined>,
alwaysArgs: [] as Array<(() => boolean) | Formula>,
nowPredicates: [] as Array<() => boolean>,
nextPredicates: [] as Array<() => boolean>,
eventuallyPredicates: [] as Array<() => boolean>,
withinCalls: [] as Array<{ amount: number; unit: string }>,
impliesCalls: 0,
orCalls: 0,
andCalls: 0,
notCalls: 0,
};
const runtime: SanderlingRuntime = {
extract: <T>(getter: (state: State) => T, name?: string): Extracted<T> => {
calls.extracts.push(getter as (state: State) => unknown);
calls.extractNames.push(name);
const handle: Extracted<T> = {
current: undefined as unknown as T,
previous: undefined,
named: () => handle,
};
return handle;
},
always: (predicateOrFormula: (() => boolean) | Formula): Formula => {
calls.alwaysArgs.push(predicateOrFormula);
return makeChainableFormula(recorded);
},
now: (predicate: () => boolean): Formula => {
calls.nowPredicates.push(predicate);
return makeChainableFormula(recorded);
},
next: (predicate: () => boolean): Formula => {
calls.nextPredicates.push(predicate);
return makeChainableFormula(recorded);
},
eventually: (predicate: () => boolean): EventuallyFormula => {
calls.eventuallyPredicates.push(predicate);
return makeChainableEventually(recorded);
},
};
const recorded = Object.assign(runtime, calls) as RecordedRuntime;
// web-runtime.ts locks __sanderling__ as non-writable (configurable) when it
// is imported earlier in the same bun-test process, so a plain assignment
// would throw. defineProperty installs (and reinstalls) the fake regardless.
Object.defineProperty(globalThis, "__sanderling__", {
value: recorded,
writable: false,
configurable: true,
enumerable: false,
});
return recorded;
}
test("extract forwards the getter to the runtime", () => {
const runtime = installFakeRuntime();
const getter = (state: State) => state.snapshots["balance"];
extract<unknown>(getter);
assert.equal(runtime.extracts.length, 1);
assert.equal(runtime.extracts[0], getter);
assert.equal(runtime.extractNames[0], undefined);
});
test("extract accepts an explicit name", () => {
const runtime = installFakeRuntime();
const getter = (state: State) => state.snapshots["balance"];
extract<unknown>("ledgerBalance", getter);
assert.equal(runtime.extracts.length, 1);
assert.equal(runtime.extracts[0], getter);
assert.equal(runtime.extractNames[0], "ledgerBalance");
});
test("extract throws when given a name with no getter", () => {
installFakeRuntime();
assert.throws(
() => (extract as unknown as (n: string) => unknown)("orphan"),
/getter is required/,
);
});
test("always wraps a predicate into a formula via the runtime", () => {
const runtime = installFakeRuntime();
const predicate = () => true;
const formula = always(predicate);
assert.equal(runtime.alwaysArgs[0], predicate);
assert.equal(formula.__sanderlingFormula, true);
});
test("now/next/eventually forward predicates", () => {
const runtime = installFakeRuntime();
const p1 = () => true;
const p2 = () => false;
const p3 = () => true;
now(p1);
next(p2);
eventually(p3);
assert.equal(runtime.nowPredicates[0], p1);
assert.equal(runtime.nextPredicates[0], p2);
assert.equal(runtime.eventuallyPredicates[0], p3);
});
test("eventually().within forwards unit and amount", () => {
const runtime = installFakeRuntime();
eventually(() => true).within(3, "seconds");
assert.deepEqual(runtime.withinCalls[0], { amount: 3, unit: "seconds" });
});
test("formula chaining exposes implies/or/and/not", () => {
const runtime = installFakeRuntime();
const a = now(() => true);
const b = now(() => false);
a.implies(b).or(b).and(b).not();
assert.equal(runtime.impliesCalls, 1);
assert.equal(runtime.orCalls, 1);
assert.equal(runtime.andCalls, 1);
assert.equal(runtime.notCalls, 1);
});
test("Tap returns a TapAction descriptor", () => {
assert.deepEqual(Tap({ on: "id:login_continue" }), {
kind: "Tap",
on: "id:login_continue",
});
});
test("DoubleTap returns a DoubleTapAction descriptor", () => {
assert.deepEqual(DoubleTap({ on: "id:save" }), { kind: "DoubleTap", on: "id:save" });
});
test("LongPress returns a LongPressAction descriptor", () => {
assert.deepEqual(LongPress({ on: "id:row" }), { kind: "LongPress", on: "id:row" });
});
test("Scroll returns a ScrollAction descriptor", () => {
assert.deepEqual(Scroll({ direction: "down", in: "id:list" }), {
kind: "Scroll",
direction: "down",
in: "id:list",
});
});
test("Tap accepts an AccessibilityElement", () => {
const element: AccessibilityElement = {
id: "login_continue",
find: () => undefined,
findAll: () => [],
};
const action = Tap({ on: element });
assert.equal(action.kind, "Tap");
assert.equal(action.on, element);
});
test("InputText returns an InputTextAction descriptor", () => {
assert.deepEqual(InputText({ into: "id:phone", text: "+1234567890" }), {
kind: "InputText",
into: "id:phone",
text: "+1234567890",
});
});
test("Swipe returns a SwipeAction descriptor", () => {
assert.deepEqual(
Swipe({ from: { x: 10, y: 20 }, to: { x: 30, y: 40 }, durationMillis: 400 }),
{ kind: "Swipe", from: { x: 10, y: 20 }, to: { x: 30, y: 40 }, durationMillis: 400 },
);
});
test("PressKey returns a PressKeyAction descriptor", () => {
assert.deepEqual(PressKey({ key: "back" }), { kind: "PressKey", key: "back" });
});
test("Wait returns a WaitAction descriptor", () => {
assert.deepEqual(Wait({ durationMillis: 500 }), { kind: "Wait", durationMillis: 500 });
});
test("actions returns an actions node carrying the generator", () => {
const generator = () => [Tap({ on: "id:x" })];
const node = actions(generator);
assert.equal(node.kind, "actions");
assert.equal((node as { generate: unknown }).generate, generator);
});
test("weighted returns a weighted node carrying the branches", () => {
const node = weighted([80, taps], [20, swipes]);
assert.equal(node.kind, "weighted");
assert.deepEqual((node as { branches: unknown }).branches, [
[80, taps],
[20, swipes],
]);
});
test("builtin factories return builtin nodes carrying their verb", () => {
const cases: Array<[GeneratorNode, string]> = [
[taps, "taps"],
[doubleTaps, "doubleTaps"],
[longPresses, "longPresses"],
[scrolls, "scrolls"],
[typing, "typing"],
[swipes, "swipes"],
[waitOnce, "waitOnce"],
[pressKeys, "pressKeys"],
];
for (const [node, verb] of cases) {
assert.equal(node.kind, "builtin");
assert.equal((node as { verb: string }).verb, verb);
}
});
test("from with a single item returns it without drawing", () => {
const sampler = from(["only"]);
setSamplerRng(new Pcg(42n, 0n));
try {
assert.equal(sampler.generate(), "only");
} finally {
setSamplerRng(null);
}
});
test("from draws intN(len) from the active picker rng", () => {
const items = ["a", "b", "c"];
const sampler = from(items);
const rng = new Pcg(42n, 0n);
const oracle = new Pcg(42n, 0n);
setSamplerRng(rng);
try {
const index = oracle.intN(items.length);
assert.equal(sampler.generate(), items[index]);
} finally {
setSamplerRng(null);
}
});
test("from falls back to the first item outside a picker walk", () => {
setSamplerRng(null);
assert.equal(from(["a", "b", "c"]).generate(), "a");
});
function elementWithChildren(cells: Record<string, string>): AccessibilityElement {
return {
find: selector => {
if (typeof selector === "string" || Array.isArray(selector)) return undefined;
const tag = (selector as Record<string, string>).testTag;
if (!tag) return undefined;
const text = cells[tag];
if (text === undefined) return undefined;
return { text, find: () => undefined, findAll: () => [] };
},
findAll: () => [],
};
}
test("keyedBy joins testTag-resolved texts with a stable delimiter", () => {
installFakeRuntime();
const row = elementWithChildren({
TxnDate: "2026-04-26",
TxnNote: "Coffee",
TxnAmount: "$5.00",
});
const key = keyedBy(row, ["TxnDate", "TxnNote", "TxnAmount"]);
assert.equal(key, "2026-04-26\x1fCoffee\x1f$5.00");
});
test("keyedBy returns empty string for an undefined element", () => {
assert.equal(keyedBy(undefined, ["TxnDate"]), "");
});
test("keyedBy substitutes empty strings for missing children", () => {
const row = elementWithChildren({ TxnDate: "2026-04-26" });
assert.equal(keyedBy(row, ["TxnDate", "TxnNote", "TxnAmount"]), "2026-04-26\x1f\x1f");
});
test("whenRoute body is skipped when the current route does not match", () => {
const route = { current: "home" as string | null };
let bodyCalled = false;
const node = whenRoute(route, "ledger", () => {
bodyCalled = true;
return [Tap({ on: "id:x" })];
});
assert.equal(node.kind, "actions");
assert.deepEqual((node as { generate: () => unknown }).generate(), []);
assert.equal(bodyCalled, false);
});
test("whenRoute runs the body when the current route matches", () => {
const route = { current: "ledger" as string | null };
const node = whenRoute(route, "ledger", () => [Tap({ on: "id:x" })]);
const result = (node as { generate: () => Array<{ kind: string }> }).generate();
assert.equal(result.length, 1);
assert.equal(result[0]?.kind, "Tap");
});
test("whenRoute accepts an array of allowed routes", () => {
const route = { current: "add-account" as string | null };
const node = whenRoute(route, ["home", "add-account"], () => [Tap({ on: "id:x" })]);
assert.equal((node as { generate: () => unknown[] }).generate().length, 1);
});
test("whenRoute body is skipped for a null route", () => {
const route = { current: null as string | null };
const node = whenRoute(route, ["home"], () => [Tap({ on: "id:x" })]);
assert.deepEqual((node as { generate: () => unknown }).generate(), []);
});