18 Commits

Include renames

Author	SHA1	Message	Date
pj	047076f9c3	chore: move subject-specific experiments to an ignored playground module ... test/okcredit and scripts/corpus were untracked and unignored, so a single git add -A would commit them. They move to playground/, which declares its own go.mod so `go test ./...` at the root stops compiling a 566-line package it only skips. Also renames the subject's bundle identifiers, used as fixture strings in two tests of generic logic, to a neutral one.	2026-08-22 21:00:51 +05:30
pj	9b4ff5f247	record what the model picker did, and make both policies see the same actions (#74) ... * feat(llmclient): parse usage and the served model An LLM-in-the-loop evaluation has to report tokens per action and cost per defect, and the client discarded both counters. Served model is recorded separately from the requested one because a router can substitute a differently-priced variant. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(trace): record one typed outcome per model-driven step llm-calls.jsonl carries the prompts as sent, the candidate list as the model saw it, the screenshot reference, the raw response, tokens, latency and how the step ended. It sits beside trace.jsonl rather than inside it because every trace line already carries a full hierarchy and both the replay server and the campaign summarizer scan all of them; folding prompts in would grow the lines those readers parse for data neither reads. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(verifier): expose the step a snapshot was observed at It lags the runner's current step whenever a transitional tree caused an observation to be skipped, which is exactly when the model is shown an older screen than the step it is choosing for. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(runner): a guard-skipped step is no longer a silent log line The strict echo-skip left only a logger.Warn, so a step the guard discarded was indistinguishable in the trace from a picker that legitimately declined. Any yield or actions-per-hour figure computed from model traces mixed the two. Every path that ends a step without a model-chosen action now records its own outcome. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(runner): record when a chosen action was never dispatched A step could carry a next_action that the foreground guard or an apply error stopped from running, and nothing said so. An executed-action count read off trace.jsonl included actions that acted on nothing. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * docs(manual): document llm-calls.jsonl Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(analyze): survival analysis over campaign directories Steps to first violation with clean runs right-censored at the budget, since per-run yield is a binary at 11 to 45 percent and separating two arms on it would need roughly 80 runs per arm. Kaplan-Meier, log-rank, Wilcoxon rank-sum with Vargha-Delaney A12, Holm within each family. A hand-rolled log-rank that is subtly wrong is a silent-wrong-number generator and would be believed, so every statistic is validated against a published worked example with the source named in the test: R survdiff on aml, Freireich 6-MP, Hollander and Wolfe 1973 for the rank sum, printed p.adjust output for Holm. Two could not be: the k>2 log-rank, guarded by calibration instead, and the tie-corrected variance, checked against an exact permutation variance. Failed and timed-out runs are excluded as missing data and counted by reason, never treated as censored observations, which would bias the result. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(verifier): select the candidate label source Candidates takes the label source as an argument rather than storing it, which is what keeps the asymmetry structural: the seeded picker selects by index and never calls Candidates, so the mode cannot reach it. That asymmetry is load-bearing, because it makes the two seeded cells of the factorial a manipulation check with identical draw streams. The identifier ladder deliberately has no text rung. A fallback that reached for text would silently turn one arm back into the other. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(runner): thread the label source to the model picker Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(trace): record the label source as arm membership Recorded for seeded runs too, unlike model and instructions. Without it the two seeded cells are indistinguishable in the artifact and the manipulation check cannot be grouped. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(cli): add --label-source Unknown values are rejected at parse time rather than falling back to the default, matching the generator check: a campaign that completes with the wrong arm and a correct-looking output directory is worse than one that fails. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(verifier): dedup candidates by what they execute, not how they read The dedup key was the rendered description, which embeds the label, so two distinct controls sharing a visible label collapsed to one entry and the survivor carried the first one's action. The second control was not mislabelled, it was absent from the candidate list, so no policy could reach it. Two scrollable containers collapsed the same way, leaving the second unscrollable. The key is now the executable Action struct itself plus whether the model supplies the typed text, so a new Action field cannot silently fall out of it. Descriptions may now repeat; the numbering disambiguates and the echo guard is index-anchored, not description-anchored. This also makes the label source a pure observation-channel change. It was not one before: the label fed the dedup key, so the two arms of the labelling factor enumerated different-sized candidate lists, in both directions depending on the screen. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(runner): report every action that was chosen and never dispatched applyAction could return nil without calling the driver, so the trace showed an action that looked executed and acted on nothing. Six paths did it: a tap, double-tap or long-press whose coordinates do not resolve and which carries no selector, a long-press whose selector is stale, an empty key press, and a zero-duration wait. It now reports whether it dispatched, and the runner records the reason and clears lastAction so the verifier never attributes the next state to an action that did not run. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * test(runner): the echo guard admits a repeated description Descriptions can now repeat after candidates dedup by what they execute. The guard is index-anchored, so this pins that a repeated string cannot make it misfire in either direction. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(campaign): count dispatched actions, not steps A step where the policy declined has no action, and a step whose action was never dispatched did nothing. Both were being counted as actions by everything downstream. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(analyze): divide by actions that ran Defects per thousand actions counted every step, including steps that chose nothing and steps whose action was never dispatched. The inflation is policy-dependent, so it does not cancel between arms: on the fixture campaign the model arm's yield was reported at 60.3 per thousand against a true 120.7, because half its steps did nothing. A runs.jsonl without the count is refused by name and line rather than read as zero actions, which would report every per-action rate wrongly. The report also carries steps beside actions now, so the gap is visible rather than folded into a denominator. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(verifier): lower authored actions the way the seeded arm does The authored descriptor path had no parity guard and diverged from the wire format on almost every verb. A Wait lost its duration and was skipped as a zero-duration wait. A Scroll lost its endpoints and its 250ms. A target that resolved to nothing became a tap at the origin, a phantom focus tap, or a swipe to (0,0) instead of being dropped. An authored target object with no x property panicked the whole run at candidate enumeration: ToInteger was called on a nil goja.Value. A target on the screen origin is still kept, so the drop rule cannot swallow it. Builtins were never affected. They serialize through the same path the seeded arm uses, which the existing policy parity test covers. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(verifier): decode a container-only scroll Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(spec): carry the container on an authored scroll serializeAction sent the container's own point as both endpoints, so an authored Scroll({in, direction}) reached the driver as a drag from a point to itself and did nothing, on the seeded arm. The wire now carries the selector and leaves the drag to the runner, which sizes it from the container's bounds and has always had tested support for it that nothing could produce. No rng runs in the serializer, which lowers an already-drawn action, so the draw stream does not move. Builtin scrolls compute both endpoints and their bytes are unchanged. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * test(runner): both policies must dispatch the same authored action Compares the recorded driver calls across 13 authored shapes. The builtin path had a parity guard and the authored path had none, which is why it drifted on almost every verb. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(hierarchy): match identifiers by role prefix idPrefix: is id: with starts-with in place of equality, so a list whose rows are named <role>_<record id> is reachable by the durable half. The Android package prefix is skipped the same way id: skips it. Routing both prefix kinds through matchAttr also makes the object form work: {descPrefix: ...} matched nothing on the native side while the web runtime honoured it. * feat(chrome): translate idPrefix to a starts-with id match * feat(spec): match idPrefix in the web runtime The DOM has no package prefix, so the native rule reduces to [id^=]. Both prefix kinds now go through the one key table, which drops the separate descPrefix branch that string and object selectors each carried. * feat(sidecar): match idPrefix in the tap-by-selector path * docs(manual): document the idPrefix selector * feat(replay-ui): render idPrefix targets as a prefix tag * fix(spec): read the injected seed per call Binding it at module scope bound it to whenever the module was first imported, so a test file that imported the runtime before setting SANDERLING_SEED froze the seed at zero for every file after it. The bundler still replaces the expression with a literal. * test(chrome): compare both selector matchers over one live page Selector matching is written once per runtime: internal/hierarchy over the dump, web-runtime.ts over the DOM. Nothing made the two agree, and a selector that resolves on one and not the other is silent, since an empty match yields no action and the run still passes. * fix(hierarchy): give id and desc one meaning in both selector forms The object form fell through to the raw attribute map, which carries no id or desc key on any platform, so {id: "save"} matched nothing while "id:save" matched. The repo's own web spec uses the object form thirty times. Both forms now resolve through one switch. Adds the accepted-key list and UnknownSelectorKeys with it, since the same silence hides any mistyped key. A key some element carries is always accepted, so raw driver attributes stay reachable. * test(hierarchy): pin both selector forms and the unknown-key report * feat(verifier): fail the spec on a selector key that cannot match An empty match is indistinguishable from a screen with no such element, so a mistyped key generates no action for the whole run and the campaign finishes clean having explored nothing. The goja boundary now throws, naming the key and the accepted list. * feat(spec): reject an unknown object-selector key in the web runtime Same rule and the same message as the native side: a key no element can carry throws instead of matching nothing. The accepted list is one list, committed as a fixture both suites assert, so a spec cannot be accepted by one runtime and rejected by the other. * test(spec): pin the unknown-key diagnostic to one text The two runtimes each claimed to raise the other's message and nothing checked it. Both now render the committed text for the committed key. * fix(spec): match a merged label by its leading name on web too The native desc rule accepts the label or the label at the head of an iOS merged label; both web translators compared the whole string, so the same selector matched natively and missed on web. The live-page parity test caught it. * test(chrome): drive the live-page parity test through both selector forms * docs(manual): document object-selector key rules * feat(spec): refuse a multi-item authored sampler while enumerating from().generate() draws from the picker's rng, which exists only inside walkActions. The model policy enumerates authored leaves outside that walk, so the sampler silently yielded its first item on every step: measured over 30 draws the seeded arm reached three targets in roughly equal proportion and the model was offered only the first. The two policies had different action spaces and nothing said so. A single-item sampler short-circuits before the rng, so both policies get the same value and it is not refused. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(verifier): stop the run on a sampler the model cannot draw, and offer disabled targets Candidates returns an error now. The refusal is thrown at the draw and wrapped with the source of the leaf that made it, since generate() cannot know which leaf it is inside. Only that marked refusal is fatal: this walk calls every leaf on every step, so promoting the rest would kill model runs the seeded arm survives. Authored actions on a disabled target are no longer dropped from the model's candidate list. The seeded picker executes whatever the leaf authored, and a control the application forgot to re-enable is exactly where boundary defects live, so a policy that cannot attempt it cannot find them. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(runner): abort on a candidate enumeration that refused Recorded as candidates_failed before the run stops, so the trace says why. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(spec): refuse a multi-value generator while enumerating integers, strings, emails and edgeCaseText read the same rng from() does, so under the model policy an authored InputText typed the same value on every step while the seeded arm varied it. That is a silently different experiment, not just a silently different action space. Single-valued spans are exempt, because both policies then get the same value: between(7,7), a zero-length string, and a one-entry corpus. length(4,4) is still refused, since the length is pinned but each character is drawn from 62. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * test(verifier): setup still draws, and the seeded stream is unmoved Setup runs through the picker with the rng under both policies, so a generator there is legitimate and must keep working. Interleaving enumeration and setup catches the flag leaking out of the model's walk. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * docs(manual): value generators are refused under the model policy too Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(folio): enumerate authored targets and values instead of sampling Sampling inside an authored leaf is refused under the model policy now, because the draw collapses to its first item there. Each sampled leaf offers one action per value instead. Lists are short, three rather than five, because the two form leaves also carry their submit and the seeded picker splits a leaf's probability across the actions it returns. The doubleTaps path that reaches the planted defect is unchanged at 5.88 percent, since no root or defaults weight moved. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(folio-web): enumerate authored targets and values, declare the llm generator The two edge-case typing leaves become the typing builtin at their combined weight: that text is deliberately not domain-specific, so naming the field and leaving the text to the policy is the designed path, and it keeps the seeded arm on the corpus while the model writes its own. Total weight is unchanged at 165, so every surviving branch keeps its share and submitTxn stays at 9.70 percent. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * docs: minimal changes, self-documenting code, tests as first-class Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(spec): key web attrs by the names the markup writes attrs was spread from element.dataset, whose DOMStringMap keys are camelCase, so a spec reading attrs["data-cents"] the way every native host reports it read undefined. In folio-web that left ledgerTxnCount and ledgerBalance permanently zero: someTransactionExists could never be satisfied, balanceMatchesTransaction Delta could never fire, and totalBalanceMatchesAccounts compared 0 to 0 and passed vacuously. Three properties reported nothing because the harness was blind, not because the application was correct. The handle also fills hintText and editable now, so an authored InputText on web names its field the way the same action names it on Android instead of rendering as Type "12.34" into "". Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(verifier): name a web handle by the same ladder as a tree element The handle fallback read only text, which is textContent and therefore always empty for an input, so the model could not tell the amount field from the note field. It now mirrors visibleLabel's ladder rather than introducing a second naming scheme. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * docs(manual): attrs carries raw attribute names on web too Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(runner): confirm focus moved before typing InputText tapped its target, slept, then typed. Android and web both inject into whatever holds focus, so a tap that missed sent the whole string somewhere else and nothing reported it. On an emulator with a floating keyboard panel parked over the password field, the tap pressed the keyboard's emoji key and every step appended the password to the email instead, forever, because the setup leaf is guarded on the password being empty. The hierarchy is re-read after the tap and the target, or something in its subtree, must hold focus. Platforms whose hierarchy carries no focused attribute skip the read, so they pay nothing. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(campaign): signal a timed-out run so it reaps its sidecar CommandContext kills outright, so a run stopped by --run-timeout never ran its own shutdown and left a sidecar holding a port and a quarter gigabyte, reparented to init and deaf to SIGTERM. The timeout exists for unattended hosts, which is exactly where nobody is watching to reap what it leaves. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * perf(runner): confirm focus only when another element holds it Measured over 717 InputText steps: nothing was focused before the tap 23.8 percent of the time, the target already held focus 60.4 percent, and a different element held it 15.8 percent. Silent corruption is only reachable from that third class, and all four real rejections observed came from it. Gating there keeps every rejection, skips 84.2 percent of the extra hierarchy reads, and recovers about 8 percent of Android run time. The pre-tap and post-tap conditions are now the same predicate stated once. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(campaign): record both clocks a run was measured on Duration came from the monotonic clock, which does not advance while a host sleeps: one calibration run under-reported by about 15 minutes. A run now carries monotonic_millis for how long it worked and wall_clock_millis for how much time passed, which is what makes a sleep visible at all. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(analyze): divide per-hour rates by time actually worked A host asleep mid-run tested nothing, and charging that sleep to an arm reports it slower for a reason unrelated to the arm. The legend also claimed wall clock while the number was monotonic. Campaigns written before the split are still read through the old field name so their run hours do not silently zero. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * test(campaign): wait for the trap instead of racing it The reaping test gave the wedged script one second to install its TERM trap, so a loaded machine signalled it first and the test failed for a reason it does not test. It now waits for the script to say the trap exists, then cancels. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(ltl): keep the authored window on a step-bounded obligation reduce decremented StepBound into the residual, so the trace reported the remaining window rather than the authored one: a within(1915, "steps") showed up as 1875 after 40 steps, and the replay UI renders that string verbatim. The duration case was fixed when bounded windows were made to serialize their resolved deadline; the step case was not, and withinFor's comment claimed otherwise. The window is now immutable and the closing observation is resolved once, which mirrors Deadline exactly. A step counts observations the evaluator reduced, not steps the runner executed, because a skipped step gave the property no chance to discharge and transitional-step rate is itself policy-dependent. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * test(ltl): pin that a slow policy does not fail on time alone Same 300-observation trace at two cadences: a 300 second bound holds for the seeded arm and violates for the model arm eight observations before the predicate fires, while a step bound holds for both. Green before and after, because the step unit already worked; this pins the property rather than fixing it. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * test(spec): guard the step unit on the authoring surface Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(folio-web): bound the reachability properties by steps At one model call per step the model arm takes 359 seconds where the seeded arm takes 47, so a second-based deadline reported violations that were the arm's speed rather than the application's behaviour. The three cross-arm reachability properties now bound by steps, derived at the measured 6.383 steps per second. The two auth-transition properties keep seconds: a user waits through those regardless of which policy is driving. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * docs(manual): a step bound counts observations, not runner steps Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(campaign): make the label source a cell dimension A 2x2 of policy against labelling needs the runner to express both factors. It could only express the policy, so half the factorial had to go through --extra, where the manifest would not record what was actually run. Rejected at parse rather than on dispatch: a sweep that finds the bad value on run 1 of 40 has already spent a cell's worth of device time. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(campaign): record the label source in the manifest A finished sweep should say which cell it ran without anyone having to remember the invocation. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(chrome): name a web field by its hint, not its CSS class visibleLabel reads hintText first for an editable element. The dump never emitted it, so an empty web input fell through text, description and descendant text to its class name, and the model was shown an identifier no user can read on exactly the fields a labelling experiment varies. Same ladder as fieldHint in web-runtime.ts, so one field is named one way on both hosts. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(web-runtime): answer clickable for an element reached through ax The handle hardcoded true, so every text node and container a spec reached through state.ax claimed to be a tap target while the enumeration and the hierarchy dump both resolved it through the tappable selector. The parity test now compares the handle against the enumeration element by element in a real browser, which is where the three answers have to agree. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * docs: every target runs on this machine, so start one rather than skip it Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(runner): a selector the tree cannot resolve is not a focus failure otherElementHoldsFocus answered true when FindNode returned nothing, so an unresolvable target read as "another element holds focus". confirmFocus then re-dumped, resolved nothing again, and errored unconditionally. Three of those in a row abort the run. Not knowing where the target is says nothing about where the text would land. The guard's real case, a resolved target with focus outside its subtree, still errors exactly as before. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(chrome): emit data-testid so both resolvers name the same element The V8 host names a web target by data-testid and TapSelector translates that selector into a CSS attribute match, but the dump carried no such attribute and no alias could supply one, since an alias only redirects to a key that already holds the value. tree.Find was therefore always nil for exactly the selectors examples/folio-web tags with. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(verifier): name an element only when the selector names it alone ax.findAll stamped every result with the query selector, and resolveCoordinates prefers the tree lookup over the element's own coordinates, so N sibling candidates all executed on the first match. On folio's Home screen the fuzzer could never open any account but the first. The gate tests identity rather than cardinality: no node other than this one answers to the rendered string, checked with the same lookup the runner runs. A rendered object selector can resolve somewhere the query never matched, so counting the query would call that unique. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(web-runtime): hold the V8 host to the same naming gate elementHandle stamped the query selector on every result the same way, so the merge carried the sibling collision onto web for authored ax targets. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * test(runner): sibling taps reach the driver at their own coordinates Drives 40 real draws from a spec that taps each card, through the picker, the serializer and DecodeAction, and asserts on the points the driver saw. Against the shared-selector bug all 40 landed on the first card. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(runner): an ambiguous name loses to the coordinates it was built from Attribute values match by substring, so a selector that named one element where the candidate was built can name several in the tree it resolves against, and the lookup sent every one of them to the first match. The host gates blank an ambiguous tag at enumeration time; this closes the gap between that moment and the action. A bare-string target carries no coordinates, so the first match stays the answer there rather than dropping an authored action. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(verifier): record an element-valued extractor instead of dropping it An element carries find/findAll host functions, so json.Marshal refused the whole value and the encoder answered nil. ChangedExtractors then emitted no entry: no error, no warning, no value. Project the value the way the web host already does (functions dropped, cycles and over-deep branches null, non-finite numbers null) and turn whatever is still beyond JSON into an error the author sees, rather than a missing extractor. * test(verifier): an unrecordable extractor value is reported, not dropped * test(runner): element-valued extractors reach the trace * docs(spec-language): say what a trace records for an element-valued extractor * docs(claude): add delegation and record-keeping sections delegation says to do installs, builds, test runs and greps in subagents and keep the main context for decisions. record-keeping says a finished task updates the files that describe its subject, writes down what was found, corrects old assumptions in place and verifies against the repository. * feat(driver): declare undelivered-action errors and three optional capabilities ErrGestureUndelivered marks a coordinate gesture that reached no element and ErrSelectorMatchedNothing a selector that named nothing, so the runner can tell them apart from a device fault. Scroller lets a driver whose scroll is not a finger drag take Scroll separately from Swipe. ExceptionReporter and NavigationReporter carry an app's uncaught errors and document-replacing navigations to the runner. * feat(driver): add escape to the pressKey surface escape is a key a spec has real use for and no platform could send it. android maps it to KEYCODE_ESCAPE, the ios companion to HID usage 41 and the in-simulator runner to XCUIKeyboardKey.escape, and the Key union accepts it so it can be written at all. * fix(ios): refuse a gesture the screen has no surface under the hierarchy reaches past the screen wherever a scroll container holds content below the fold, so an action derived from it can name a point no touch lands on. tap, double tap, long press and swipe now report ErrGestureUndelivered for such a point, the far edge exclusive because a touch at x == screenWidth arrives at screenWidth-1. resolveSelectorCenter reports ErrSelectorMatchedNothing rather than a bare error. * feat(ios): derive scrollable from the snapshot's tree depth the companion now emits each node's depth, so the hierarchy mapper can find the containers that clip content reaching past their own frame and mark them scrollable:true, the same fact android reads off uiautomator and the web driver derives from overflow. a dump without depth makes every element a root and roots are never marked, so the legacy bridge reports no scroll rather than a guessed one. * fix(sidecar): stop dropping gestures, selectors and keys in silence a point outside the screen is refused with OUT_OF_RANGE, a selector that matches nothing with NOT_FOUND, and a key with no device-driver equivalent throws instead of pressing nothing. parseBounds also reads uiautomator's [left,top][right,bottom] form, which is what a device actually reports and which left every by-selector tap on a device resolving to nothing. * fix(sidecar): map the driver's refusals onto the gesture errors OUT_OF_RANGE becomes ErrGestureUndelivered on tap, long press, double tap, swipe and the selector fallback; NOT_FOUND on TapSelector becomes ErrSelectorMatchedNothing. without this the runner reads either as a plain apply failure and counts it toward the failure streak. * fix(selectors): resolve text to the innermost match and scan the root in both forms an element's text is its whole subtree's text on web and on ios, so every ancestor of a matching element matched too, up to the root. a match a descendant also makes is now dropped, in internal/hierarchy, in the chrome xpath translation and in the page-side web runtime, so all three resolvers name the same element. a raw attribute now matches on a substring (exact for true/false) the way the docs describe, and tree-level FindBySelector considers the root, so ax.find("id:page") and ax.find({id: "page"}) agree. * feat(hierarchy): store the tree shape and tolerate an unreadable boolean flag a Tree marshalled to json kept only the flat element array, so a stored tree decoded with a nil Root and resolved no selector. it now stores each element's pre-order depth and rebuilds Root from it, re-seating elements so Tree.Elements and &node.Element stay one pointer. a stored tree without depths keeps the old shape. a boolean field the producer sent as something other than a boolean now leaves the flag unset and increments UnreadableFlags rather than failing the whole dump. * fix(chrome): emit every markup attribute and read checked and selected off the property the dump emitted a fixed standard attribute set, so a spec reading data-cents or data-account-id saw undefined on the goja host and nothing at all in the trace. it now keys every attribute by the name the markup writes, derived keys overwriting. checked and selected come from the dom property rather than whatever a component left on the object, which is also what the page-side element handle now reports, so a ticked box reads as ticked instead of reporting its starting state forever. * fix(chrome): scroll a gesture point into view and dispatch trusted input getBoundingClientRect keeps reporting elements the growing document pushed below the emulated viewport, and input coordinates are viewport-relative, so a click below the fold was hit-tested to the document root and the step read as an action that landed. every gesture now scrolls the point back in and reports ErrGestureUndelivered when nothing is under it; a selector that names no node reports ErrSelectorMatchedNothing rather than waiting. swipe dispatches a real touch stream instead of page-synthesized pointer events, scroll is a wheel so its distance is exact rather than a fling, and the second tap of a double tap carries click count 2 so dblclick actually fires. * feat(chrome): read the page's exceptions and navigations, and hold the picker state across them a page navigation replaces the runtime, so the seeded picker restarted the seed's stream at its first draw on every reload and a trace could not tell a reload from a generator repeating itself. the driver now drains the main-frame navigations it saw, reports the page's buffered uncaught errors so state.exceptions is the page's list on the goja host too, and carries the picker's draw position out of v8 and back in around each decision. * feat(trace): version each step and record its logs, exceptions and navigations a step now carries trace_version, the platform log lines and uncaught errors behind state.logs and state.exceptions, the document-replacing navigations seen since the previous step, and observation_error naming why a device read produced no tree. version 0 is a step written before those fields existed, which is what separates a trace that cannot answer the question from a step that had nothing to report. * feat(runner): bound every device call and record the actions that never reached the app observation and apply now run under a timeout, so a driver that stops answering ends the step rather than the run. an undelivered gesture and a selector that matched nothing are recorded as their own skip reasons instead of counting toward the apply-failure streak, a failed observation is counted apart from a screen with nothing on it, and the summary names both. resolveCoordinates hands a point outside the viewport to the driver rather than dropping it: only the driver knows whether it can scroll that point back into reach. exceptions and navigations are collected per step and a Scroll goes to a driver's Scroller when it has one. * feat(verifier): expose extractor names and rebuilt property formulas an offline replay of a trace needs the name-to-index mapping the spec fixed at load, because a trace records extractor values by name, and needs each property's formula built over this verifier's own predicates so a rewritten formula observes exactly what the engine's evaluator does. * feat(testrun): expose the seeded bundle a run loaded BundleSpec produces the goja bundle a run of a spec loaded, seeded as that run was. an offline replay has to load the same javascript, and the seed is one of the bundle's defines, so it is part of the bundle's identity. * feat(tracecorpus): load recorded runs for offline measures reads a run directory's meta and every step, and refuses a step whose trace_version is not the current one: an older step stores no element depths, so its hierarchy decodes with a nil root and a structural hash over it is the empty string for every screen. Discover walks a tree for the directories holding both meta.json and trace.jsonl. * refactor(seedspec): move seed spec parsing out of the campaign command the campaign tool and the sweep tools that drive it have to read a seed specification the same way, or a sweep records an intent that differs from what ran. parseSeeds becomes seedspec.Parse with no behaviour change. * feat(analyze): time an event at the step it was detected and report the quartiles an obligation that never discharges is reported when the run ends, and timing it by the step that armed it recorded a liveness failure flushed at the budget as a violation found on step 1. the survival analysis now measures the detected step, falling back to the origin for campaigns written before the field existed, and says how many events that moved. the report gains the first and third quartiles beside the median. * feat(analyze): add the seed-paired signed-rank comparison and record the holm family --paired contrasts two arms running the same seeds seed by seed with the wilcoxon signed-rank test rather than treating them as two independent samples, reporting the per-seed differences, the sign, a12 within pairs and the seeds usable in one arm only. --question names the family holm corrected within, and the family size is recorded next to the p-values rather than left to the reader to reconstruct. * test(analyze): recover planted effects through the tool's own entry point a pipeline exercised only on data whose answer nobody knows reports that it runs, not that it is right. these plant effects whose value follows from the generating model and require the tool to recover them from campaign directories it reads off disk. * feat(label-coverage): report the addressable share of an app's interactive surface reads the hierarchies a run already recorded and splits each screen's interactive elements by the strongest selector that can name them, so a spec's reach over an app is a number rather than an impression. * feat(exploration-reach): count the distinct structural states a stored run visited the state is the settle path's structural hash of the recorded hierarchy, the same function the drivers wait on, so a state boundary here is the one the harness itself uses. --reference reports the observation at which two runs' hierarchies first differ. trace only: no device, no replay. * feat(defect-identity): count distinct defects across stored runs a property reports at most once per run, so a run-level count is just the number of properties violated. a defect is identified across runs by the property, the action attributed as the origin of the failed obligation and the screen the witness observed. * feat(oracle-reduction): replay stored traces under four reduced oracles re-evaluates each trace offline under the full engine, a crash-only detector, a single-state check and a single-step property triple, and reports what each refutes: the oracles vary while the traces stay fixed, which separates a defect an oracle cannot express from one an explorer never reached. a disagreement with the verdicts a run recorded exits nonzero rather than being counted as a finding. * feat(implementation-sweep): run one campaign against every implementation of a requirement installs, builds and serves each implementation on its own port, then hands the campaign tool the same seed slice, step budget and generator for all of them, so a difference between implementations is not a difference in exploration. the generator and platform are fixed rather than exposed. * feat(corpus-sweep): run one specification against a served corpus of implementations same fixed campaign as implementation-sweep, over a corpus that needs no build. each implementation gets its own port: the corpus holds pairs that write the same localStorage key, and one shared origin is one stored record shared between them. * docs(manual): document innermost text matching, escape and the web scroll verb text: names the innermost match and both selector forms scan the same set, root included. escape joins the key list, with a per-platform note and the rule that a key the platform cannot send fails the action. scroll and swipe are one gesture on a touch device and two different ones in a browser, so say which reaches what. * test(browser): assert an uncaught page exception reaches the trace the page buffered its uncaught errors in v8 and nothing carried them out, so state.exceptions was empty on the host and no trace held one, leaving an offline crash oracle nothing to read. asserts the recorded trace steps rather than the summary. * feat(trace): a step can name the precondition it could not meet A step that never had the app under test in front of it observed something else, and nothing in the trace said so. Index 0 carries the startup gate's verdict, so a run that never started is a trace holding that record and nothing else rather than a run that explored and found nothing. * fix(runner): budget the foreground gate in time, not in polls Eight polls is not a budget. Each poll costs whatever the driver's idle wait happens to take, so the same launch cleared the gate on one device and exhausted it on another: across 80 runs of one app, the gate reported "app never reached foreground" on 38 of 40 Android 14 runs and 0 of 40 Android 16 runs, and it was wrong every time. On API 34 settleForForeground returned in ~100ms, so the eight polls gave up 1.2s into a launch whose window drew at ~1.9s; on API 36 the same eight polls spanned 3s and covered it. The Android 14 runs then spent their first step on the launch animation instead of the app, which is the one-step offset that came out of that campaign looking like a platform difference. The gate now polls for a fixed 15s at a 250ms floor, so its verdict is the same duration on every device, and a verdict of "not in front" ends the run instead of warning and carrying on: a run that never got its app on screen holds no evidence about the app, and the trace records why at step 0. * test(runner): the gate keeps looking until its budget runs out Locks the three facts the campaign was missing: a window that draws after more polls than the old count allowed still clears the gate, an app that never comes forward ends the run with a typed error, and both the startup verdict and every mid-run step the guard could not recover are readable off trace.jsonl. * feat(campaign): count the runs that were never in the app A run that failed its precondition has zero steps and no violations, which is what a short clean run looks like too. The summary now counts the trace records naming an unmet precondition, so a campaign directory answers "how many of these were never in the app" without grepping any log. * docs(triage): name the trace field a run that never started leaves * fix(selectors): tag names the whole tag, not a substring of it matchSelectorKind had no case for tag, so it fell through to the raw attribute path and matched by substring. web-runtime.ts compiles tag to a CSS type selector, so tag:li resolved to <todo-list> on the Go side and to nothing on the web side. * test(chrome): both resolvers agree on tag where a container's name contains its child's * fix(make): build the binary instead of matching the build directory build/ exists at the repo root, so make build was satisfied by the directory and left a stale bin/sanderling in place. * feat(verifier): expose the property names a loaded spec registered * feat(testrun): refuse a run against a spec that registers no properties A spec with no properties drove the app and reported no violations, which is indistinguishable from a spec that judged something and found nothing. Execute now aborts after loading the spec unless the run asks for the opt-out by name. * feat(cli): --allow-no-properties opts a run out of the refusal * docs(cli): document --allow-no-properties * feat(bundle-check): fail a spec that bundles but registers no properties * test(bundle-check): cover the zero-property refusal and pin the reported bundle * feat(folio-web): predicates for counting commits against submit actions * feat(folio-web): judge one commit per submit over a home-card window Replaces totalBalanceMatchesAccounts and balanceMatchesTransactionDelta, which compared two consecutive steps on one screen and so could not see a double submission that lands across a navigation. * fix(folio-web): keep submit live for 400ms after saving Defers the navigation back so the button is tappable while the label reads Saved, widening the double-submit window the counting property is there to catch. * feat(confusion-matrix): score the checker against a blind reviewer Cross-tabulates the properties that fired against the human verdict, one cell per implementation, over a sweep whose implementations all passed their own generated tests. An implementation that failed to build, has no usable run, or carries no filed verdict is listed as missing data rather than counted as a clean cell. Landing the package in one commit because the intermediate splits would not link. * test(confusion-matrix): reject malformed inputs and keep missing data out of the cells * test(confusion-matrix): cover cell assignment, precision and recall * fix(chrome): focus descends into the shadow root document.activeElement names the host, not the node focused inside it, so a Compose-for-wasm app that mounts its tree in a shadow root reported focus on div#app forever. confirmFocus could never be satisfied and every InputText step aborted the run after three tries. selectAllScript already descends the boundary; the tree builder did not. * test(implementation-sweep): supply the binaries the missing-binary test does not test resolveBinaries ranges a map, so with more than one binary absent the error named whichever it reached first. The test passed locally only because bun and sanderling were on PATH; on CI it was a three-way coin flip. * fix(replay-ui): read data-* attributes by their markup names The web runtime now publishes raw markup attribute names, so attrs["step"] read nothing where the markup writes data-step. Three properties went vacuous and exactlyOneStepIsSelected reported false against a UI that was fine. The test also fails if a dataOf key gains no matching attribute, or if an attribute it derives is rendered nowhere. * fix(web-runtime): focus descends into the shadow root here too The Go driver already descends the boundary; the V8 host did not, so the two enumerations disagreed about focus on any shadow-mounted app. The harness now answers activeElement the way a real root does: a root names a node of its own tree, so only the shadow root itself names the field. * fix(implementation-sweep): name every missing binary, in flag order Ranging a map returned at the first failure, so an operator missing three binaries was told about one, fixed it, reran, and was told about the next. The function exists to stop the sweep once rather than fail per implementation and seed. Two identical runs also printed different errors, which is why this reached master as a flake instead of a clean red. * fix(chrome): focus follows the caret to the field it types into Compose for wasm never focuses the semantics node carrying the testTag. It proxies keystrokes through a hidden 1px backing input that is a sibling of the a11y tree, so the node the runner tapped never held focus and confirmFocus refused to type into every Compose text field. Focus is re-attributed to the smallest editable whose box holds the caret's centre. Centre-point rather than full containment because the caret's height comes from the text style and the field's from its layout box, so a taller font would silently drop back to refusing. * fix(corpus-sweep): name every missing binary, in flag order Same map-ranging bug as the sibling tool, and this copy had no test on the missing-binary path at all. * fix(web-runtime): a handle answers editable for itself, not its container isContentEditable is inherited, so every span inside a contenteditable div called itself typeable. collectTargets and the chrome dump both require the element itself to match; the handle was the one that did not. * test(chrome): a hinted field is not named by its css class The fixture inputs carried no class at all, so the test could not fail the way the bug did. They now carry folio-web-shaped classes, and the test asserts the editable gate the hint is read behind. * test(chrome): the handle and the enumeration agree on editable too The helper compared clickable alone, so the inherited-contenteditable bug was caught by unit test only and never in a real browser. * fix(web-runtime): focus follows the caret to the field it types into Mirrors the driver, so the two hosts agree about focus on a Compose page. The harness inherits custom properties down the parent chain the way CSS does, so an implementation matching the inline style attribute fails. * fix(campaign): name every missing required flag, in flag order Five required flags ranged as a map, so omitting three told the operator about one, chosen at random. * fix(corpus-sweep): name every missing required flag, in flag order * fix(implementation-sweep): name every missing required flag, in flag order * fix(confusion-matrix): name every missing required flag, in flag order * ci: pin the idb-companion tap to the formula the companion is staged from The tap moved to 1.5.0, whose bundle has no top-level Frameworks/, and prepare.sh stages bin/ and Frameworks/ as siblings because the binary resolves through @rpath. Floating on it also made the hard-coded companion-1.1.8 output name a lie. The ios-assets cache does not cover this: it restores and make rebuilds anyway, because checkout stamps prepare.sh newer than the archived tarball. Master was green only because its last run predated the bump. * fix(campaign): refuse to start on a device that is not there A sweep launched at six serials, three of which had been deleted from the host. 19 of 20 runs were lost, and not because half the devices were wrong: a worker on a dead serial fails in about 31 seconds and immediately pulls another seed, so three bad workers drained sixteen seeds while the three good workers were still inside their first run. Fast failure is more dangerous than slow failure, because the fast failure consumes the resource the slow one would have left alone. Preflight names every missing serial before the first seed is dispatched. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(campaign): quarantine a device that keeps failing fast Preflight cannot catch a device that disappears mid-sweep, which is what happened: the serials were alive the previous day. Three consecutive failures under two minutes, with no run that worked in between, is a property of the device and not a coincidence. The manifest records which device was quarantined and which seeds have no result, so an aborted sweep says so in its own artefact. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(trace): record the device a run executed on meta.json carried the host but not the device, so a trace could not say what hardware produced it without the campaign manifest beside it. An experiment splitting cells across api levels could only join them through that manifest. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * ci: let a restored ios bundle survive make's mtime check The cache restored and the build ran anyway: a restored tarball keeps the mtime it was archived with while checkout stamps the sources, so make read every bundle as stale. Both logged Cache hit and rebuilt regardless. Dating the bundles after their sources fixes the lie where it is told. Order-only prerequisites would have fixed it in make, but a laptop has no cache key, so editing prepare.sh would silently embed the previous tarball. The formula version joins the key because a hit now decides what gets embedded, and the key was blind to the brew install: the 1.1.8 and 1.5.0.b2 runs shared a key. * fix(confusion-matrix): a campaign that died is missing data, not a true negative The sweep-level loop excluded a run on launch_error alone, while excludedBecause already checked the campaign process's exit code. An interrupted campaign wrote exit_code -1 with an empty launch_error, so its one completed seed scored the implementation as a clean cell on a tenth of the planned evidence. The fixture builder wrote one exit code into both the sweep record and the campaign run record, which is why no test could tell the two levels apart. * fix(analyze): censor a clean run at the steps it ran, and refuse mismatched budgets A run stops at whichever comes first, the step budget or --duration, so a clean run that reached the wall clock exited with fewer steps than the budget and was still credited with the whole of it. The model arm pays a network call and a screenshot per step, so it reaches the wall sooner and was handed exposure it never had. Nothing checked that two arms shared a budget either. Thirty identical clean runs under budgets of 400 and 100 read a12 0.000 and p 1.685e-14 from the rank-sum while the log-rank in the same report read p 1.0000. groupArms already refused this within one arm. The claims the old convention left in comments and report lines are corrected rather than left standing beside the new behaviour. * fix(runner): a source that was asked and handed nothing says so NextAction returning ErrNoAction left the step with no skip reason, so a run whose every model call failed on transport, a non-2xx, an empty choices array or an echo mismatch printed no violations and exited 0. Only llm-calls.jsonl knew it had never touched the app. The reason now travels the path the other five already take, so it reaches the trace, the summary, and the campaign's dispatched-action exclusion. A held step never asks and keeps carrying nothing. * feat(testrun): refuse a run that dispatched none of its actions Same argument as the zero-property refusal: an instrument that drove nothing must not report a clean result. A first-screen violation still wins under --exit-on-violation, --allow-no-properties exempts the extraction sweeps that measure reach rather than judge, and one dispatched action is enough, so a generator quiet on some screens is untouched. * docs(cli): document --label-source * docs(spec-language): name the hintText selector's host divergence The line said the key matches placeholder alone, which is true of the web runtime and not of the tree, where it resolves against the derived attribute. A spec author reading it wrote a selector that matched on one host and not the other. * feat(bundle-check): --allow-no-properties opts out of the refusal The run path grew the opt-out and the freeze gate did not, so a spec the extraction and portability sweeps register nothing for on purpose could be run but never frozen. The refusal now names the flag the way the runner's does. * test(verifier): an unreadable committed fixture fails, it does not skip The comment said the round trip always runs. A skip on a fixture that is committed turns a missing or truncated file into a green. * fix(testrun): the refusal asks whether the generator drove, not whether anything did A dead provider against folio exited 0 on a real emulator: the login setup dispatched three actions before the generator was consulted, so DispatchedActions was 3 and the gate never fired while the generator drove the app zero times across 83 steps. Any spec with a login setup was immune, which is the normal case. Summary counts generator actions separately and the refusal reads that. NoActionsDispatchedError becomes NoGeneratorActionsError, because a run that dispatched three login taps was lying in the old name. * feat(runner): the summary says how many steps the generator drove A green llm run carried no evidence of how much the generator actually drove: the count was inferable only from llm-calls.jsonl outcomes, and the number the refusal turns on was invisible in the run's own output. * fix(testrun): an ios run records the simulator it executed on Device was read from --device, which only an android run sets, so every ios meta.json left the field empty and the trace could not say what hardware produced it. * fix(campaign): the action count leaves the setup's login out on a model run Defects per thousand actions divided by every dispatched step, so a spec whose setup logs in inflated the denominator by however many steps that took. It is the same error the run gate had, and it does not cancel between arms. A model run is separable because only an llm-selected action stamps next_action.source. A seeded run is not: its setup returns through the same entry with no marker, and 11261 dispatched steps across the 169 recorded runs carry no source at all, so excluding on it blind would report every seeded run as having explored nothing. The seeded arm counts as before and a test pins that. * feat(hierarchy): an element reports whether it masks what is typed into it ios reads it off SecureTextField, which the companion already sent and nothing read; web reads input[type=password]. Android cannot: the native tree mapper drops the password attribute before the sidecar sees it, so the fact is three-valued and null there rather than a false that would read as "not secure". * fix(verifier): a secure field's typed value never reaches the record A folio login run wrote the account email and password in cleartext into llm-calls.jsonl, 166 times in one run, beside screenshots of the same screens. Three sites rendered it: the recent-action memory, the candidate list, and the trace. One helper now covers all three so a fourth cannot bypass it, and the driver still receives the real text. Android redacts every typed value because it cannot tell a secure field from any other. That asymmetry is deliberate and documented: safe by default on the target that cannot tell. * fix(runner): a secure field's value does not reach state.lastAction either folio extracts lastAction, and extractor values are persisted as extractor_changes, so the password still reached the run directory through the spec after the three render sites were closed. The wrap sits in the runner rather than in lastActionFields because the hosts hold the next step's tree, not the one the action was chosen against: a field that stops being secure between the two would publish what the trace withheld. Live and replay now agree byte for byte. * fix(trace): an action names the generator that produced it The setup exclusion landed for the model arm only, because only a model pick stamped a source. A seeded run returned setup's action through the same entry with no marker, so its denominator still counted the login while the model arm's did not, and the two are compared. serializeAction names setup and seeded on the wire, so both arms are counted by one rule. An already-recorded trace names nothing and keeps exactly the count it was reported with; unattributed_actions counts those steps so the old denominator cannot pass as the new one. TraceVersion is deliberately unbumped: oracle-reduction refuses a differing version, and a bump would make all 169 recorded runs unreplayable. * fix(defect-identity): degrade a redacted origin action to its selector The full action key read the typed value straight from the trace, where redaction renders every value typed into one field as the same string, so two runs that typed different values there collapsed into one identity and the report said nothing about it. The key now drops a redacted value, falls back to the selector for that action, and counts the rows it did that to, so the undercount reads as an undercount. * fix(campaign): a record always says how many actions named no producer An omitted count reads the same as a run recorded before actions carried a source, so the two cannot be told apart by anything downstream. * fix(analyze): read how much of a record's action count names no producer A runs.jsonl written before actions named one has no field, and its whole count is of unknown provenance rather than none of it. * fix(analyze): refuse to compare attributed and unattributed denominators One arm's actions may include the login the spec's setup drove and the other's cannot, so a per-action rate over the two divides by different things and the tests rank the bookkeeping. * fix(analyze): mark an action count of unknown provenance in the report * docs(manual): what an action count with no producer means for a rate * fix(folio): install through adb so a remote adb server works Gradle's install task talks to adb through ddmlib, which reads only ANDROID_ADB_SERVER_PORT and dials the loopback address, so ADB_SERVER_SOCKET never reaches it and `just test` could not touch a remote emulator. Gradle now only assembles the APK and adb does the install, which picks up the same server every other call in the run talks to. * docs(folio): say how to point just test at a remote adb server * test(conformance): the g4 fixture holds what a redacted android trace holds Android reports no secure fact for any field, so every InputText it records writes the redaction placeholder rather than the typed value. The fixture still carried the real value, which is the only reason the gate reported itself as catching the doubling. Two more fixtures come with it: a repeated-character corpus value that reads as its own doubling and must not fail, and a backend that does record the typed value. Red at this commit: G4 reports PASS on a doubled field it cannot see. * fix(testrun): a recorded violation outranks the dead-run refusal A campaign never passes --exit-on-violation, so the refusal was discarding runs that had found something: exit_code 1 in the record and the analysis drops them as missing data. A run that recorded a violation holds a verdict, which is the whole reason the refusal exists. * fix(testrun): the dead-run refusal gets its own opt-out --allow-no-properties was waiving two unrelated refusals, so a sweep passing it for the property-free reason silently lost a detector it never asked to disable, and a run with properties could only get the dead-run exemption by claiming one it did not want. * feat(cli): --allow-no-generator-actions The flag the dead-run refusal names, wired through to the pipeline. The property-free flag goes back to meaning what it says. * refactor(analyze): open the log-rank up to a weight on the risk set The log-rank is one member of a family that differs only in how much each event time counts. Nothing else changes: the counts it reports stay counts whatever the weight, and the published-dataset results are unmoved. * feat(analyze): add the gehan generalized wilcoxon test The rank-sum carried over to right-censored samples: every pair of runs is scored by which one outlived the other, and a pair censoring cannot order counts as half rather than as a difference neither run supports. The effect size and the p-value are the same statistic, and with nothing censored both are exactly what the rank-sum reports. * fix(analyze): compare arms on censored runs, not on flattened step counts stepTimes threw the censoring flag away and handed the rank-sum a plain number per run, so a run the wall clock stopped at step 12 was ranked as one that violated at step 12. That was defensible while every clean run sat at the budget, the largest value any run could take, and it stopped being defensible when a clean run started being censored where it stopped. Twenty runs clean at step 12 against twenty violations at step 100 read a12 0.000 and p 4.683e-10 from the rank-sum, in the same report as a log-rank reading p 1.0000. The pairwise comparison is now the Gehan test over the observations themselves, and the report says how many run pairs censoring left with no order between them, which is how much of the effect size is the null value rather than an observation. * fix(conformance): g4 reads a doubling off the observed field value The typed value stopped reaching the trace on any target that reports no secure fact for the field, which on android is every field, so the gate was comparing the redaction placeholder against itself and passing whatever the driver did. The observed value is not redacted, and a field holding one string twice over is the doubling itself. A value that is a single character repeated stays exempt: the corpus types "a" 4096 times and a pair of spaces, and neither can be told apart from its own doubling. The recorded-value check stays for the targets that do record it, where it also catches a doubling appended to content the field already held. * fix(spec): a secure selector names the password field on web secure is derived from the field type, not written by the markup, so matching it as a raw attribute reached nothing: the key is accepted, no unknown-key error fires, and find answered undefined on web for the field it answers with on ios. false is every editable field that is not a password entry, since an element that is no field reports null and answers to neither value. * test(chrome): resolve the secure selector on both matchers the fixture covers the password entry, the three shapes of editable field that are not one, and a checkbox that is no field at all. * test(chrome): compare the secure fact across both producers it is the fourth fact the dump and the web runtime derive independently, and the one that decides whether a typed value is written into the shared record. three-valued, so the fixture guard requires all three states rather than both polarities. * docs(manual): state what a secure selector matches * test(conformance): g4 keeps checking past an input typed at coordinates An InputText that names no field aborts the analyzer, so the gate reports the whole run as failed and checks none of the steps after it. 129 of the 485 recorded traces hold such a step. Red at this commit: jq stops on a null selector and the gate reports FAIL. * fix(conformance): g4 skips an input that names no field jq splits an empty string into no segments, so reading the last one off an action typed at coordinates threw and took the rest of the run's steps with it. Such a step names nothing to check; the gate now passes over it and keeps checking the ones that do. * test(browser): the exit code a dead run and a violated one actually leave Drives the built binary against a page with nothing to tap and reads the process status, then the same run through campaign to pin what lands in runs.jsonl: exit_code 1 there is a detection the analysis drops as missing data. * fix(spec): keep a secure selector valid beside another key a multi-key object selector concatenates its parts into one compound, and a type selector is valid only at the head of one, so {id, secure} built '[id="pwd"]input[type="password"]' and querySelectorAll threw. * fix(analyze): score a seed pair by which run outlived the other The paired path had the same defect as the unpaired one: it subtracted two step counts and handed the differences to the signed-rank test, so a pair holding a run the wall clock stopped at step 12 entered as a difference neither run supports. Twenty seeds where the first arm was still clean at step 12 and the second violated at step 5 in six of them read sign -1 and p 0.0011, pointing at the arm that never violated. A pair is now scored the way the unpaired comparison scores one and tested by the exact sign test over the pairs whose order censoring determines, which is what the log-rank stratified by seed reduces to here. The signed-rank goes with the differences it needed: a magnitude-based paired test wants a difference from every pair, and the arms censor on different clocks. The median difference stays, over the pairs where both runs violated, and says so. * docs(analyze): name the tests the tool actually runs The --paired flag advertised the signed-rank, two comments and a test message still said rank-sum, and nothing said what rankSum is doing in the tree now that no campaign reaches it. * docs(manual): exit 1 also means a run that holds no verdict And the flag the dead-run refusal now names, which --allow-no-properties used to double as. * docs(skills): quote the summary line the runner prints now The setup skill's empty-page claim was the stale one that mattered: that run records no_action_produced on every step and exits 1, it does not sit at exit 0 with no violations. Numbers remeasured against the counter and throwing fixtures. * test(conformance): g4 sees a doubling appended to what the field held Redaction cost the gate this shape on android: the driver typed the value twice onto existing content, so the whole value is not its own doubling and the typed value is not in the trace to compare against. The recorded-value check still catches it on the backends that record one. Red at this commit: G4 reports PASS on a field that grew by one string twice. * refactor(analyze): hoist the sign test's loop bound * fix(conformance): g4 reads a doubling out of what the field grew by The whole-value check misses a driver that typed the value twice onto content the field already held, which is the append-vs-replace shape the recorded value used to catch before it was redacted. What the field grew by over the snapshot the action was chosen against is the same signal and needs no typed value. Checked against every recorded trace under conformance/runs: 485 traces, 299 of them carrying an InputText, none newly failing. * fix(analyze): write an undefined paired p-value as null, not as NaN A paired contrast where censoring orders no pair has no p-value, and JSON has no NaN, so --json failed with 'marshal summary: json: unsupported value: NaN' and wrote no summary at all after printing a complete report. The two fields join the medians and the rates already carried as pointers, undefined reading as null in the summary and n/a in the report. Reachable since a clean run started being censored where it stopped: an arm the wall clock stops before its partner ever violates orders nothing. * fix(spec): a boolean state selector names what the live element reports clickable, enabled, focused, checked and selected are derived from the element rather than written by the markup, so matching them as raw attributes built [clickable="true"] and reached nothing: the keys are accepted, no unknown-key error fires, and the worked example in docs/manual/spec-language.md found no element on web and passed having checked nothing. Each key is answered by the same function elementHandle derives the fact with, since no CSS says what any of them says: :focus names the shadow host of a focused field as well, :checked misses a checked custom element and answers for a selected option besides, and [checked] is the state the page loaded with rather than the one the user left it in. * fix(spec): keep a tag selector valid beside another key a multi-key object selector concatenates its parts into one compound, and a type selector is valid only at the head of one, so {id, tag} built '[id="amount"]input' and querySelectorAll threw. whether a spec got an exception or an element depended on the order its author wrote the keys in. * fix(chrome): state every boolean flag the dump can state internal/hierarchy writes the attribute a selector matches on only where the producer stated the flag, so a state emitted as null is one no selector can ask about: {clickable: false} and {enabled: false} matched nothing at all against a web dump while matching on android, which states every flag both ways. only secure stays three-valued. * test(chrome): resolve the five state selectors on both matchers the fixture differs one state at a time: a disabled button and an aria-disabled role control, a box ticked by script with no checked attribute beside one cleared by script that has it, and a select whose first option is selected without the markup saying so anywhere. half the states are asked inside one container, because a state the whole page has an opinion about answers with most of the document and a want list nobody can check by reading. * test(chrome): compare checked, selected and focused across both producers the target enumeration carries none of the three, so they reach a spec through the ax handle alone, and a selector naming one of them resolves against that same reading. the shadow fixture holds the focused control inside its shadow root, where document.activeElement names the mount element and only a producer that descends finds the field. * fix(spec): keep a selector out of the head subtree the head renders nothing, so the hierarchy dump drops it and so does the enumeration the picker walks, but a selector still resolved into it: a whole-page findAll answered with <head> and <title> here and with neither on the goja host, which is a divergence the moment a state selector asks a question every element has an answer to. * docs(manual): state what the other boolean state selectors match * fix(folio): refuse to install and fuzz a device nobody named adb falls through to the local server when ADB_SERVER_SOCKET is unset, and claims the only device attached there. That could be a personal handset, and a run installs the app, clears its state and fuzzes it. Every recipe that touches a device now resolves the target through _require-device, which only picks on its own when a single local emulator is all adb sees. * docs(folio): state that android recipes need ANDROID_DEVICE * fix(spec): and text with the keys written beside it a compound object selector dropped text and matched on the other keys alone, so {testTag: "Row", text: "Alice"} selected every row carrying the tag where internal/hierarchy selects the one row the author named. matching more than the spec said is silent: the find lands on a row nobody wrote and every property over it still passes. text is answered against the element the way the boolean states are, since css cannot ask what an element's text says and the xpath that can cannot ask about the rest, and the innermost rule now holds over what the whole selector matched, where internal/hierarchy holds it. a text-only selector still compiles to the same innermost xpath. * test(spec): pin text against the key beside it in either order object keys iterate in insertion order, so the order the author wrote them in decided what a compound selector meant. the innermost rule is pinned over the whole selector's matches: a row whose badge carries the class and the text both is dropped, one whose badge carries the text alone is kept, and a state key is anded before either. * test(chrome): compare a compound text selector across both matchers one page, both resolvers, text written before and after the key beside it. the object form now encodes its keys in the order the filters state them rather than the order a map iterates, so both orders are asked. the row and the badge under it share a class so the innermost rule has something to drop, and {text, clickable} pins that text is anded before that rule runs: the innermost element carrying "January" is the option, and the select is the only element that is both. * docs(manual): state how text combines with the key beside it the object selector section said every pair must match without saying where the innermost rule then lands. * fix(hierarchy): reach the class attribute through className className is an accepted selector key that no producer writes: android reports the view class, ios the element type and the chrome dump el.className, all of them under `class`. With no alias onto that key the selector matched NOTHING here on every platform while the web runtime resolved it against the live DOM, so {className: "status"} named the row and the badge on one host and no element at all on the other. The failure is silent: the key is accepted, so no unknown-key error fires, and a property over the element that was never found passes having checked nothing. * test(chrome): compare className across both matchers one page, both resolvers, the two names for the one attribute. class is asked beside className so the pair is pinned to the same elements rather than each to itself: the row and the badge under it both carry it. * test(spec): pin className and class on the same elements this host answers both names against the live DOM and internal/hierarchy now aliases the second onto the first, so a name dropped from the table here would match nothing on web while the dump still answers it. * docs(manual): list className among the cross-platform aliases the key was already typed on the spec surface and already resolved on web, and the alias table said nothing about which attribute it reads. * fix(hierarchy): reach the accessible label through every name for it label and accessibilityLabel aliased onto accessibilityText alone, which only the ios sidecar writes, and alias expansion is ONE level: the hop from accessibilityText to content-desc was never taken, so both keys matched nothing on android and on the chrome dump, which write the fact under content-desc. ariaLabel and contentDescription aliased onto nothing at all and matched nothing anywhere. The web runtime resolves all four against the live DOM, so a selector naming a field this way found it on one host and no element at all on the other. The keys are accepted, so no unknown-key error fires, and a property over the element that was never found passes having checked nothing. Each name lists both keys rather than chaining through accessibilityText: transitive expansion would silently widen every existing key at once. * fix(hierarchy): reach a web test tag through testTag and testID Compose for Web writes a test tag as data-testid, which is what the web runtime resolves both names against. testTag aliased onto the three identifier keys and not that one, and testID aliased onto nothing at all, so a tag the web runtime found on every row of a list named no element here and every property over it passed vacuously. * fix(spec): resolve the identifier, label and class aliases against the DOM identifier, accessibilityIdentifier, accessibilityText and elementType are the names ios writes four facts under, and internal/hierarchy aliases each onto the key the other producers write. This table listed none of them, so each fell through to a raw attribute lookup and built [accessibilityIdentifier="summary_card"], which no element carries. Every one of them resolved against the dump on the goja host and named nothing here. The keys are accepted, so no unknown-key error fires, and a property over the element that was never found passes having checked nothing. * fix(spec): an editable or scrollable selector names what this host derives Both facts are derived from the live element rather than written by the markup, and matching them as attributes built [editable="true"], which no page carries. Both resolve against the dump on the goja host, so a spec naming a field or a scroll container that way found it there and no element at all here, with no unknown-key error to say so. Each reads the same function the fact is derived with, so a selector cannot name an element this host calls something else: the handle, the picker's target list and the editable selector all go through isEditable, and scrollable reads the overflow test collectTargets reads. scrollable false names nothing rather than every element that does not scroll: both producers state the fact only where it holds, the way an element that is no field at all answers to neither value of secure. * test(chrome): compare the alias keys and the two derived facts one page, both resolvers, the ten names that resolved on one host only. each alias is asked beside the key it resolves through, so the pair is pinned to the same elements rather than each to itself. the page grows a container that overflows its box and a neighbour that does not, because scrollable is derived from the box: without one the only scrolling element on the page is the document root, whose answer moves with the window. * fix(hierarchy): bounds is a raw attribute, not a cross-platform key Every native dump writes the rectangle out as a string under bounds, and no DOM element carries an attribute of that name, so the key resolved against the dump and matched nothing on web on every page there is. It is accepted, so no unknown-key error said so, and no mapping can be invented for it: there is no DOM fact to map it to. Off the accepted list the web runtime raises the unknown-key error instead of matching nothing in silence, and the key still resolves wherever a producer writes it, through the escape hatch every other raw attribute already uses: a key some element carries is a key that can match, on both sides. * docs(manual): state which attribute each alias reads, and what bounds is the table listed neither name for the accessible label that a web page writes, nor the key a web test tag lands on, and said nothing about elementType. editable and scrollable are boolean states like the rest, and scrollable is the one of them the platforms state only where it holds. bounds is a raw driver attribute rather than an accepted key. * docs(hierarchy): the package doc names every key an alias reaches it described the alias table as it stood before the label and test-tag names reached the keys android and web write, and said nothing about expansion being one level deep, which is why each name has to list every key rather than hop through another alias. * fix(spec): a hint selector names the ladder both producers derive hintText and placeholderValue are the accessible-name ladder, derived from the live element, and compiling them to [placeholder="..."] made them name the wrong field or none at all. A field labelled by an aria-label or a bound <label> carries no placeholder, so it resolved against the dump on the goja host and reached nothing here; one carrying both answered to its placeholder here where the dump answers to its aria-label, which lands a find on an element nobody named. Both keys read the same fieldHint elementHandle and the hierarchy dump (internal/driver/chrome/driver.go) derive the fact with, so a selector cannot name a field this host calls something else. An empty hint names nothing rather than everything that is no field: both producers write the fact only where the ladder answered. placeholder stays the attribute the markup writes, which is what the dump carries under that name too, so a field whose hint is something else still answers to it on both hosts. * fix(chrome): a hint target is not tapped by the placeholder attribute TapSelector is a third resolver, and it built [placeholder="..."] for hintText and placeholderValue too. Now that both matchers read the accessible-name ladder, that CSS names a field whose hint is its aria-label and whose placeholder happens to carry the value, which is an element neither matcher named. No CSS says what the ladder says, so both keys fall through to a match that reaches nothing and the step fails naming the selector, the way every other derived key in this file already does. A selector reaches here only where the dump resolved it to no coordinates at all. * test(chrome): compare the hint keys and placeholder across both matchers The page gains four fields that differ one rung at a time: a bound label, a placeholder, a placeholder an aria-label outranks, and the name the form gives the field. Only the placeholder rung was reachable before, so hintText and placeholderValue named a field on the goja host and no element at all on web for the other three, and named the field here and nothing there for the rung the ladder passed over. placeholder was measured empty on both hosts because nothing on the page carried the attribute, which said nothing about it. It now names the field the markup wrote it on and not the field whose hint is its aria-label. The third resolver reads the same selectors: what TranslateStringSelector builds for a hint key has to match nothing over CDP rather than the field carrying the value as a placeholder. * docs(manual): both hosts read the hint ladder, placeholder is the attribute The web section said the hintText key does not read the ladder on both hosts and told authors to select such a field by attrs.hintText instead. Both hosts read it now, so that instruction is gone rather than left standing beside a newer sentence. placeholder is stated as the attribute the markup writes and nothing more, the tap path is stated as failing by name where no CSS says what the ladder says, and the alias table gains the row it was missing.	2026-08-19 11:07:49 +05:30
pj	76dce1a75e	experiment instrumentation: step budgets, arm labels, campaign runner (#72) ... * feat(cli): add --max-steps for step-bounded runs runner.Options.MaxSteps already worked but was unreachable from the command line. A step budget is what makes two generators comparable: one making a model call per step and one drawing from a PRNG are not comparable per second. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(trace): record arm membership and host in meta.json meta.json recorded the seed but not which picker ran, how it was configured, what budget it was given, or which machine produced it. A directory of runs cannot be attributed to an experiment cell without those, which makes any factorial computed from such a directory unanalysable after the fact. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(cli): add --arm and populate run meta from it Model and instructions are recorded only when the LLM picker is the one that will actually run, so a spec declaring generator = llm() that is run under the seeded picker does not label its trace with a model it never called. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(campaign): sweep seeds for one experiment cell campaign.json lists the seeds a sweep intended to run and is written before the first run, so a host that dropped runs shows up as missing seeds rather than as a smaller sample. Seed 0 is rejected: sanderling test reads it as "derive a seed from the clock", which is why conformance/gates.sh controls nothing today. Each run contributes one runs.jsonl line carrying steps to first violation by origin step, the step that armed the failed obligation, so the survival analysis never reopens a trace. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(runner): no silent generator fallback, and llm on web --generator llm against a spec declaring no generator = llm(...) logged a warning and ran the seeded picker. For a comparison campaign that is silent arm corruption: the run completes, the directory looks correct, and the wrong policy drove it. It is now fatal. pickSources also returned the V8 source for both action and extractor on web before it looked at the generator, so the llm policy was unreachable there. The two axes are now independent: the driver picks the extractor source, the flag picks the action source, and llmSource composes with either because the runner populates the candidate list and screenshot on every platform. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(chrome): make the hierarchy dump agree with the web runtime Three facts differed between the dump the goja host reads and the DOM the V8 host reads, so the two enumerated different candidates on one page. scrollable was never emitted, and worker.go reads exactly that attribute while targets.ts requires it for scrolls, so the goja host could not offer a single web scroll. clickable tested el.onclick, which React assigns to its root container for event delegation, making the whole viewport a tap target here and in no other enumeration. Both now resolve through the selector sets in pkg/spec/src/web-runtime.ts. The dump also rooted at body while collectTargets walks querySelectorAll("*"), so the goja host never saw html, where page-level scrolling lives. It now roots at documentElement and skips the head subtree, which is all zero-bounds and would otherwise carry script and title text into the trace. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(conformance): give the gate reproducible seeds SEED defaulted to 0 and sanderling test reads --seed 0 as "derive a seed from the clock", so the tunable controlled nothing and a gate failure could not be re-run. SEEDS now takes one explicit non-zero seed per run, recorded in the results table so a failing row names its stream. The five runs stay on five different streams: a gate that scored one path five times would catch less than one that scores five. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(chrome): emit editable as a plain boolean editable was emitted as `isEditable || null`, and an absent field sends internal/hierarchy into the native fallback, which reads any class name containing "EditText" as an Android text widget. On web that is just a CSS class, so a page styling a div with it was editable to the goja host and not to the web runtime, and the model policy could be offered typing into a div. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(spec): leave the head subtree out of the web target walk collectTargets walked querySelectorAll("*") while the hierarchy dump skips head, so the two hosts enumerated different element sets on every page with a <head>. No candidate changes: builtinCandidates pushes only for targets acceptsTarget admits, and head elements have no positive bounds, so the list the draw ranges over is untouched. What changes is that targetIndex now means the same thing on both hosts. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * test(chrome): compare the facts both hosts derive from one DOM The existing parity harness hand-authors the facts on both sides, so it proves that given identical facts both hosts select identical candidates, and says nothing about the two code paths that derive those facts from a real page. Four divergences lived in that blind spot and it passed throughout. This drives one real page and compares clickable, enabled, editable, scrollable and positiveBounds element by element, plus the element sets themselves, which is what catches a host that omits html or includes head. Reverting any of the four fixes makes it fail naming the element and the fact. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * chore(make): run the browser packages one at a time Both launch Chrome and launching two at once has failed with "Launch: context canceled". Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * style: remove every em-dash and en-dash Eighteen occurrences across fourteen files. Each sentence was repunctuated to suit what the dash was doing rather than swapped for a hyphen, which produces comma splices. The minus sign in folio-web's ledger is a minus sign and stays. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(chrome): honor the caller context in Launch Launch and clearState ran against d.tabCtx, so a target that accepts the connection and never answers wedged the process past its own --duration and through SIGTERM, needing SIGKILL. Unattended that is a campaign worker lost for the rest of the sweep with no diagnostic. The browser is still allocated against d.tabCtx first, because chromedp starts Chrome under whichever context calls Run first and allocating under a caller deadline would kill the browser when Launch returns. Everything after allocation goes through runCtx. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * fix(sidecarassets): publish the extracted jar through a rename Extract wrote a 96 MB jar with a plain WriteFile into a temp path every sanderling process on the host shares. On a cold host several concurrent workers all miss the checksum and all write the same path, and O_TRUNC lets one spawn a JVM against another's half-written archive. A fresh experiment host is exactly a cold host. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * feat(campaign): kill a run that outlives --run-timeout A wedged run holds its worker for the rest of the sweep, and on an unattended host nothing else will send it a signal. Defaults to three times --duration and must exceed it. A killed run is recorded as timed_out rather than as a generic failure, so the analysis can tell a lost cell from a real crash. Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX * style(test): gofmt browser_test.go Claude-Session: https://claude.ai/code/session_01A5KmftdEJ49A9z5mF5ESrX	2026-08-12 22:20:31 +05:30
pj	26b49b379a	fix ltl semantics and unify action enumeration (#71) ... * fix(ltl): give every thunk a construction identity Two distinct unnamed predicates both described as "Thunk(...)", so obligation collapse merged their residuals and could drop a live violation. Identity is assigned at construction and the fields are unexported, so a thunk cannot be built without one. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * fix(ltl): reduce a thrown-predicate residual instead of panicking The verifier substitutes an ErrorFormula for the residual of a property whose predicate threw, and that residual is fed back in on the next step. reduce had no case for it, so the run crashed. It re-reports the same failure now. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * fix(ltl): make a bounded always the dual of a bounded eventually G<=n(f) and not F<=n(not f) disagreed on traces where the inner was still pending when the window closed, so nnf's negation normal form was not semantics preserving. Both sides now range over the observations at which their inner can definitely resolve: the eventually keeps a pending inner as a disjunct, and the always discharges vacuously at window close. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * fix(ltl): arm a one-shot root once per run A root that carries its own horizon is one obligation for the whole run, not one per observation. Re-instantiating a top-level eventually monitored G F<=n(p) instead of F<=n(p) and left one live obligation per step behind; a bounded always restarted its window every step and never closed. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * fix(verifier): stop wrapping a top-level eventually in always `eventually(p).within(300, "seconds")` as a property meant "within 300 seconds of every step", which spawned an obligation per step with its own resolved deadline. A 553-step run carried 553 of them and serialized a 75 KB residual. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * fix(ltl): serialize the resolved deadline of a bounded window Two obligations spawned at different steps from one duration-bounded formula differ only in the deadline the evaluator resolved for them, so they serialized identically and the trace erased a distinction the evaluator makes. The authored window stays in amount/unit; the resolved deadline rides alongside. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * fix(verifier): split a witness's origin step from its detection step A deferred obligation spans two steps: the one that armed it and the one whose reduction failed. They were conflated under one index, so the extractor snapshot (which is the detecting step's state) was reported against the origin step. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * fix(runner): record a witness's detection step in the trace Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * feat(replay-ui): show the step a violation was detected at The witness evidence is the detecting step's state, so say which step that is and let a reader jump to it. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * fix(verifier): record the extractor state the predicates actually read On the web path extractor bodies are evaluated in V8 and injected here, but only the goja value was replaced. The trace diff and the violation witness therefore described a state no property ever saw. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * refactor(spec): one candidate producer over one target-eligibility rule Both hosts routed verbs themselves and both policies enumerated their own actions, and all four drifted. Web sent `swipes` to scrollable containers only, so swipe-to-dismiss on a list row was reachable on native and unreachable on web; the model policy folded gestures its own way and could not reach what the seeded picker drew. A host now reports facts about every element and never decides which verb may act on it: targets.ts acceptsTarget owns that for both. pick.ts builtinCandidates is the single enumeration, and the model policy reads it through __sanderlingEnumerateBuiltin__ instead of reimplementing it in Go. Gesture verbs change with it: scrolls stay vertical over scrollable containers, swipes go free-form in all four directions from any element with real bounds. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * fix(runner): name a builtin scroll by its drag origin A builtin gesture carries endpoints and no selector, so every scroll rendered as "Scroll down " in the prompt's recent-action memory and two scrollable regions were indistinguishable. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * fix(chrome): clear storage over cdp instead of scripting an opaque origin Launch runs while the tab is still on about:blank, whose opaque origin denies storage access, so localStorage.clear() threw SecurityError and every web run died at launch. Storage.clearDataForOrigin needs no navigation. The exception helper lands here because "Uncaught" is what hid this for so long. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * fix(chrome): enable the swiftshader webgl fallback Headless Chrome runs with --disable-gpu, and without this flag it refuses the software WebGL backend: getContext returns null, so a canvas-rendered app paints nothing and every screenshot is identical black. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * fix(web): resolve testTag through data-testid or id Compose Multiplatform emits its testTag into the element id, which the native table already accepts via the resource-id alias. The two web selector tables were the only place that rejected it. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * test(spec): type-check the spec api as part of make test The fake runtime in api.test.ts did not return a chainable handle from extract, so the file had not type-checked since named() was added. Wiring the check into make test stops it drifting again. Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J * docs(manual): one-shot eventually and the gesture verbs Claude-Session: https://claude.ai/code/session_01Fj4wJUikdABuMQEETwW55J	2026-08-12 18:06:04 +05:30
pj	7343085614	llm action-selection backend (#68) ... * feat(spec): add llm() action-backend marker * feat(spec): make llm marker inert on the JS picker * feat(spec): expose __sanderlingSampleInput__ corpus draw * feat(openrouter): minimal chat-completions client * test(openrouter): cover request shape, parse, and errors * feat(verifier): thread screenshot + capture corpus sampler * feat(verifier): LLM accessors — candidates, config, sampler * test(verifier): cover AllCandidates, LLMConfig, SampleInput * feat(trace): record action Source and LLMReasoning * feat(runner): thread step screenshot into PushSnapshot * feat(runner): llmSource selects actions via OpenRouter * feat(runner): wire llmSource selection and trace stamping * test(runner): cover llmSource selection, mapping, downscale * docs(folio): add llm action-backend example spec * docs(folio): document the LLM action backend run * feat(llmclient): support OPENAI_API_KEY, openrouter wins * refactor(runner): rename openrouter package to llmclient * docs: both api keys, example model gpt-5.4-nano * docs: add pr style rules to claude.md * fix(runner): explain action kinds in llm prompt to stop swipe loops * feat(trace): record llm ranked list and chosen rank * feat(runner): stamp llm ranked list and chosen rank on trace * fix(runner): tap by selector to survive layout shift after observe * revert(runner): drop selector-first tap; broke path/testTag selectors * feat(spec): llm() accepts optional instructions * feat(verifier): read llm instructions off config * feat(runner): append spec instructions to llm system prompt * docs(folio): describe app in llm spec instructions * feat(bundler): map generator export to globalThis.generator * feat(verifier): read llm config off globalThis.generator * feat(runner): gate llm source on --generator flag * feat(cmd): add --generator llm|seeded flag * test: cover --generator flag parsing and pickSources gating * feat(verifier): enumerate llm candidates by walking actionsRoot collect-walk the weighted action tree: recurse weighted branches accumulating selection probability, call authored leaves once for concrete actions, enumerate builtins per element. label controls by visible text (borrowing descendant text), fold gestures into directional scrolls over scrollable containers, drop disabled, dedup descriptions. * test(verifier): cover candidate enumeration walk * feat(verifier): add SetupAction to walk setup without the seeded root * test(verifier): cover SetupAction setup-only precedence * refactor(llmclient): make JSONSchema.Schema raw json for pinned field order * feat(trace): record llm choice number and chosen_action echo * feat(runner): llm picks one number from weighted candidates drop the seeded-root call for a setup-only precedence path, render a numbered weighted candidate list, pin a reasoning-first choice schema, strict-skip when chosen_action does not echo the numbered entry, and let the model supply typed values (corpus fallback when empty). * test(runner): cover choice schema, strict-skip, and setup precedence * refactor(verifier): drop the superseded AllCandidates enumeration * feat(folio): drive spec.ts under --generator llm; drop spec-llm.ts * fix(verifier): label editable fields by hint, not the typed value an editable field's own text is its transient content; prefer the hint so the field is named by purpose and the label stays stable. * test(runner): cover weight-suffixed echo and stripWeightSuffix * fix(runner): accept chosen_action echo that carries the weight suffix real runs showed the model copies the whole numbered line including the trailing (w34) weight annotation, so strict-skip rejected ~91% of picks and the llm was paralyzed. strip the weight suffix before comparing. also nudge the prompt to stress-test repeated submissions (idempotency). * fix(verifier): skip llm enumeration on cross-fade frames a navhost mid-transition carries >1 route *Screen in a collapsed coordinate space; acting on it taps garbage (soft keyboard). real runs showed the llm acting on 44% of steps being such frames. skip them so the llm re-observes a settled frame next step. * feat(folio): show current balance on the add-transaction screen renders the account's balance (testTag TxnCurrentBalance) below the account name, above the credit/debit toggle, so before/after screenshots carry comparison data. * fix(replay): derive device space from screen extent, not first node the first positive-bounds element is often a short status-bar node (320x24 on android); using it gave a 320/24 aspect ratio that squashed the screenshot overlay into a grey horizontal band. use the max extent across elements (like the runner's screenBounds) instead. * fix(folio): show balance as a compact one-line label per review: one line, account-name-sized, e.g. "Balance: $0.00" instead of a large balance card. * fix(folio): move balance into the header, one compact line under the account name * fix(replay): attribute deferred violations to the causing step, not detection * fix(replay): show a step's own violations in both panels, no next-step bleed * refactor(hierarchy): one Tree.Transitional, drop the duplicated cross-fade check * chore: ignore .playwright-mcp scratch output * docs: document the llm generator and --generator flag * docs(spec): correct the llm() comment; config reads off globalThis.generator * docs: add pr description rules	2026-07-31 21:12:00 +05:30
pj	406b7516b3	iOS simulator driver: Go-native companion-backed backend (#62) ... * perf(ios): use prebuilt XCTest runner to cut startup * chore(ioscompanion): add companion asset prepare script * feat(ioscompanion): embed and extract simulator companion bundle * test(ioscompanion): cover companion stub and embedded extraction * docs: add third party notices for vendored companion * chore: ignore vendored companion bundle artifact * build(proto): pin simulator companion proto v1.1.8 * build(proto): add dedicated buf module and gen template for pinned proto * build(proto): exclude pinned companion proto from root buf workspace * feat(ioscompanion): commit generated companion gRPC stubs * feat(ioscompanion): map flat companion describe dump to TreeNode JSON * test(ioscompanion): add hierarchy-map golden and unit tests * feat(ioscompanion): port screen-settle stability polling to Go * test(ioscompanion): cover settle transitional, hash, streak, and cap rules * feat(ioscompanion): add USB HID keymap module * test(ioscompanion): cover keymap branches and paste-chord constants * build: embed companion assets via withcompanion tag * feat(ioscompanion): add transport companion interface * feat(ioscompanion): add HID event wrapper and builders * feat(ioscompanion): wire gRPC companion client and Dial * test(ioscompanion): cover HID builders and unit conversions * test(ioscompanion): cover Dial, process-state mapping, and install archive * test(ioscompanion): add gated simulator integration smoke test * feat(ioscompanion): text input and gesture HID composition with pasteboard fallback * test(ioscompanion): cover input composers, paste dialog loop, and pure helpers * feat(ioscompanion): add Describe to companion transport * feat(ioscompanion): implement DeviceDriver with companion supervision * test(ioscompanion): unit tests with fake companion transport * test(ioscompanion): gated companion smoke test * feat(ios): add ResolveTarget for simulator vs physical-device routing * feat(testrun): route iOS simulators through the native companion driver * refactor(testrun): defer the java preflight check to the physical-device path * feat(cli): add --ios-app-path flag * feat(doctor): split iOS checks into simulator and physical-device paths * test(folio): add gate-analyzer fixtures for G1-G5 * feat(folio): add iOS conformance gate script * chore(folio): wire gates recipe, app path, and ignore gate output * style: gofmt struct alignment drift * fix(doctor): probe simctl via xcrun instead of PATH lookup * fix(ioscompanion): spawn companion under driver-lifetime context * test(ioscompanion): prove companion child outlives startup context * fix(ioscompanion): chunk install payload under companion message cap * test(ioscompanion): cover install payload chunking * fix(ioscompanion): reinstall via simctl and sanitize companion env * fix(ioscompanion): wait out unresolved accessibility values after launch * perf(ioscompanion): paste long text for atomic landing * test(ioscompanion): cover paste threshold, retry flow, and sentinel detection * fix(ioscompanion): treat unresolved bridge values as transitional, never as content * fix(ioscompanion): accept masked secure-field values as paste landing * test(ioscompanion): cover sentinel mapping and masked-field landing * fix(ioscompanion): atomic erase and single-send paste to prevent doubling * test(ioscompanion): cover atomic erase, single chord, unverifiable field * fix(ioscompanion): verify paste on a time budget that outlasts the bridge blackout * test(ioscompanion): cover bridge-blackout paste verification * fix(ioscompanion): drop unresolved-value settle gate that never let empty-field screens settle * refactor(ioscompanion): name the empty-editable-field sentinel for what it is * perf(ioscompanion): tighten settle streak for the fast companion transport * feat(ioscompanion): pre-grant pasteboard access so unicode input skips the OS prompt * refactor(ioscompanion): drop paste warm-up now that the grant suppresses the prompt * test(ioscompanion): cover pasteboard grant on launch, drop warm-up tests * fix(ioscompanion): retry describe past transient collapsed accessibility dumps * test(ioscompanion): cover collapsed-dump detection * perf(ioscompanion): split raw and retrying describe so settle does not double-wait collapses * perf(ioscompanion): tighten settle now that collapses are handled separately * fix(ioscompanion): replace field content on input so blackout-skipped erase cannot accumulate text * test(ioscompanion): cover replace-on-input and TextReplacer capability * refactor(ioscompanion): neutralize HID events behind the transport seam * feat(companion): add simulator runner project skeleton * feat(companion): serve accessibility snapshots over the wire protocol * feat(companion): synthesize timestamped touch gestures * feat(companion): type text with replace semantics * feat(companion): serve the wire protocol from a parked runner * feat(ioscompanion): add TextEditor capability and unavailable sentinel to the transport seam * feat(ioscompanion): route text input through a text-editing companion when available * fix(companion): bind listener by port and source screen size from snapshot * feat(ioscompanion): add runner companion JSON transport * test(ioscompanion): cover runner transport protocol mapping * fix(companion): synthesize gestures synchronously to avoid the async completion crash * fix(companion): type on the main thread and recover from focus assertions * fix(companion): keep serving after an automation failure * refactor(companion): tidy snapshot serialization * fix(companion): honor sequential tap gaps and survive synthesis exceptions * feat(ioscompanion): expose native typing with an explicit replace flag * chore(companion): add runner asset prepare script * feat(ioscompanion): embed and extract the runner test bundle * test(ioscompanion): cover runner asset extraction * build(ioscompanion): commit runner asset archive * feat(ioscompanion): pair the legacy companion with the in-simulator runner * test(ioscompanion): cover hybrid routing, paste-grant skip, and port binding * fix(ioscompanion): reconnect after interrupted runner calls instead of restarting * fix(ioscompanion): route hybrid lifecycle through the runner and harden restarts * feat(companion): launch and terminate apps through the automation session * build(ioscompanion): refresh runner asset with session lifecycle * fix(ioscompanion): classify connection deadline expiry as caller budget * fix(companion): capture snapshots on the main thread inside the catch bridge * build(ioscompanion): refresh runner asset with main-thread snapshots * perf(ioscompanion): count read spans toward settle and capture snapshots concurrently * feat(ioscompanion): make the hybrid simulator companion the default * test(folio): cover runner-session orphans in the gate harness * test(ioscompanion): pin the child-lifetime test to the legacy path * fix(ioscompanion): keep mappable text on one HID stream and verify unicode clears * fix(ioscompanion): pause the clear chord so selection applies before the delete * fix(companion): prune the keyboard subtree from snapshots * build(ioscompanion): refresh runner asset without keyboard elements * fix(ioscompanion): capture the screenshot transport before a recovery can reassign it * fix(companion): pin the runner listener to loopback * fix(companion): size the replace delete prefix to cover any focused field * build(ioscompanion): refresh runner asset with loopback bind and replace fix * fix(cli): cancel the run context on SIGINT so spawned children are reaped * fix(testrun): point the device java preflight hint at the ios-device doctor * fix(folio): word-bound the G2 ERROR scan and drop the dead objc allowlist glob * test(ioscompanion): cover stopProcess, restart, and failed bring-up supervision * chore: add test-companion target for the withcompanion-tagged suite * chore(ioscompanion): stop tracking the runner archive build artifact * build: produce the runner archive from source like the companion bundle * refactor(conformance): move the gate harness out of examples/folio * chore(folio): drop the gate harness wiring from the example app	2026-06-08 19:10:54 +05:30
pj	94d9511312	test: full test-suite refactor sweep (#61) ... * chore(test): start test-suite refactor sweep * test(ltl): pin exact multi-obligation residual AST * test(ltl): table-test finalize Kleene connective combinations * test(ltl): pin reduce over pending inner for bound, Or, Not * test(ltl): marshal bounded Always steps/duration/deadline * test(verifier): cover LTL combinator verdict transitions and within unit panic * test(verifier): table-test DecodeAction kinds and lastAction field exposure * test(verifier): assert WithPlatform(ios) reaches the picker host and key pool * test(verifier): widen weighted-selection assertion to a 5x skew margin * test(verifier): un-skip ax-find round trip with a committed tree fixture * test(runner): pin isWDADrop to sidecar reconnect-failed message origin * test(runner): assert PressKey/Wait trace encoding records kind-specific fields * test(runner): cover RenderSummary unsupported-verbs surfacing branch * test(trace): set Hierarchy in round-trip and lock lossy Tree contract Also add a -race concurrent WriteStep test that asserts N well-formed JSONL lines, catching torn lines if the writer mutex is dropped. * test(trace): round-trip witnesses/changes/metrics/exceptions, pin step-0 witness * test(trace): document ViolationsAreGreppable grep contract and lock-free WriteScreenshot * test(hierarchy): cover invalid-JSON and malformed-bounds parser paths * test(trace): guard writer mutex via WriteStep/Close race on w.file * test(replay): drop unfailable assets and devproxy assertions * test(replay): cache reuses on equal mtime, reparses after append * test(replay): violation marker falls back to detection step when attributed missing * test(replay): corrupt meta/trace dirs return 500 with error body * test(replay): SSE client receives runs.changed after a broadcast * test(replay): Run coalesces creates, ignores write/chmod, closes subs on cancel * fix(sidecar): synchronize health fixture writes and exercise healthError * test(sidecar): cover swipe/longpress/doubletap/erase/presskey/metrics/logs translations * test(sidecar): cover DoubleTapSelector composition and mid-gesture cancel * test(sidecar): assert gRPC error status surfaces from action RPC * fix(chrome): route action methods through runCtx so caller cancellation aborts CDP * fix(chrome): route hierarchy/screenshot/waitidle/metrics through runCtx * refactor(ios): extract pure simctl JSON parsers * refactor(ios): add command-runner seams for EnsureSimulator * test(ios): table-test simctl parsers and EnsureSimulator seams * test(sidecarassets): cover placeholder build path * test(sidecarassets): assert reuse via sentinel bytes not mtime * test(bundler): cover properties-only spec registration * refactor(testrun): extract prepareBundleInputs from Execute * test(testrun): cover prepareBundleInputs aliases and missing-runtime error * test(testrun): table-test resolveRuntimeSibling search edges * test(testrun): exact-output tests for progressHandler line format * fix(cmd): point bundle-check aliases at pkg/spec/src * test(cmd): smoke-test bundle-check resolves spec aliases * test(cmd): table-test hier-check parse and FindAll on fixture * test(cmd): unit-test buildBrowseURL deep-link vs root * test(cmd): drop flaky TestRun_Doctor that launched real Chromium * test(cmd): pin pipeline error to bundle resolution on web platform * test(replay-ui): add bun test script * ci(replay-ui): run bun test via make web-test target * ci(replay-ui): point bun cache key at replay-ui/bun.lock * test(replay-ui): exercise real URL encoding and non-ok throw in getJson * refactor(replay-ui): extract snapshot flatten/getAtPath into lib module * test(replay-ui): pin snapshot flatten/getAtPath path round-trip * refactor(replay-ui): extract action selector/format into lib module * test(replay-ui): pin action selector parse and row formatting * refactor(replay-ui): share one statusFor between panels * refactor(replay-ui): extract run-history derivation into lib module * test(replay-ui): pin shared statusFor precedence and ordering * test(replay-ui): pin run-history derivation alignment * refactor(replay-ui): export clampIndex for testing * refactor(replay-ui): extract keyboard-nav dispatch into pure module * refactor(replay-ui): extract metrics formatters into lib module * test(replay-ui): pin clampIndex step boundaries * test(replay-ui): pin keyboard-nav ownership and key routing * test(replay-ui): pin metrics formatters and path gap handling * refactor(sidecar): expose device-output parsers as internal for testing * test(sidecar): table-test device-output parsers against malformed input * test(sidecar): cover logcat parsing year inference and line skipping * test(sidecar): pin pressKey keycode mapping and unknown-key rejection * test(sidecar): metrics bundleId falls back to launched app and honors override * test(sidecar): loosen deadline upper bound to tolerate slow CI scheduling * test(web-runtime): export selector builders for unit tests * test(web-runtime): guard sanitize cycle, function, and depth limits * test(web-runtime): table-test selector builder quoting and escaping * test(sidecar): collapse scalar-forwarding RPC tests into a table * test(replay-ui): dedup step/summary fixtures into shared module * test(ios): collapse pickSimulator point-tests into a table	2026-06-06 13:59:08 +05:30
pj	9958b0ddc8	Attribute violations to the causing step and render witness evidence (#59) ... * feat(ltl): attribute violations to the obligation origin step * feat(verifier): label evaluator observations with the runner step index * feat(trace): carry the causing step in violation witnesses and summary * feat(replay): move the violation marker to the causing step * feat(replay-ui): render witness evidence in the violations panel * feat(replay-ui): wire witnesses and step jump into violation panels * fix(ltl): treat next obligations as vacuous at run end * fix(runner): give the finalize trace record its own step index	2026-06-05 23:46:36 +05:30
pj	b44077afde	replay ui fix (#56) ... * refactor: rename inspect to replay across the codebase Renames inspect-ui/ to replay-ui/, internal/inspect/ to internal/replay/, the CLI subcommand from `sanderling inspect` to `sanderling replay`, and updates all references in docs, Makefile, README, and Go comments. * feat(replay-ui): show spec filename with full path on hover RunList and RunDetail now render the basename of spec_path (e.g. login.spec.ts) with the full path available as a title tooltip.	2026-06-03 16:17:26 +05:30
pj	c5bb176be8	UX refactor (#52) ... * feat(ltl): bound fields on AlwaysFormula and named thunks Add StepBound/Duration/Deadline to AlwaysFormula as the dual of bounded Eventually, give ThunkFormula a Name for stable identity, add ThunkNamed, and surface both in describe() and MarshalJSON. * feat(ltl): negation normal form pass nnf/pushNot rewrite a formula so every Not wraps only a Thunk or Error leaf, dualizing Always<->Eventually and preserving bounds. * feat(ltl): NNF in NewEvaluator, bounded-always, Finalize, collapse Apply nnf on construction, reduce bounded Always symmetric to bounded Eventually (vacuous holds once the window closes), add Finalize to resolve undischarged liveness obligations to Violated at run end, and collapse structurally-identical pending obligations. * test(ltl): property-based NNF laws Lock double-negation identity, Always/Eventually duality with bound preservation, leaf pushdown, and not(always true) reaching Violated. * test(ltl): Finalize, bounded eventually, latch, collapse Property tests for monotonic violation latch and eventually-within violating iff n consecutive false, plus Finalize and collapse cases. * feat(inspect): within clause on always residual node A negated bounded eventually serializes as a bounded always; render its bound instead of dropping it. * feat(ltl): witness violations and (bool,error) predicate thunks * test(ltl): migrate thunk call sites to (bool,error) * feat(ltl): flag thrown-predicate witnesses with IsError * refactor(verifier): replace predicate err side-channel with violation witness * test(verifier): witness API for thrown predicates * feat(trace): witnesses map and skipped-verification marker on Step * feat(runner): thread violation witnesses, finalize, skip marker into trace * test(ltl): lock violation witness reason, IsError, and step * test(verifier): finalize surfaces unmet eventually with witness * fix(ltl): eliminate implies and bounded-always false-negatives Rewrite a -> b to (not a) or b in NNF so a pending temporal antecedent can no longer defer the whole implication and drop a consequent that was false at the current step. Carry a pending inner past a bounded-Always window close instead of dropping it to holds, so a deferred obligation is resolved by a later step or Finalize. * test(ltl): lock implies and bounded-always false-negative regressions * fix(web-runtime): seed PRNG for reproducible runs and align weighted pick * feat(testrun): inject seed into web bundle via SANDERLING_SEED define * test: cover web-runtime seeded PRNG, weighted pick, and seed define wiring * test(spec): add Go math/rand/v2 PCG oracle and golden fixture * feat(spec): bit-exact PCG port of Go math/rand/v2 * test(spec): assert pcg.ts matches the PCG golden fixture * feat(spec): shared input corpus and press-key pools * feat(spec): action-tree types and Host interface * feat(spec): verb support matrix and warn-once helper * feat(spec): deterministic shared action picker * test(spec): verb matrix and warn-once semantics * test(spec): picker draw-order and determinism * refactor(spec): actions.ts returns pure GeneratorNode data trees * refactor(spec): wire from() sampling through the picker rng * feat(spec): shared runtime-entry installs next-action over pick.ts * feat(spec): export LongPress/Scroll/longPresses/scrolls factories * test(spec): assert data-tree shapes for action factories * test(spec): runtime-entry serializeAction wire-contract round-trip * refactor(spec): bridge data-tree nodes to the legacy goja picker tags * fix(spec): web runtime walks the spec's globalThis.actions data tree * test(spec): tolerate legacy bridge fields on builtin nodes * refactor(spec): installRuntime accepts a lazy root resolver The web bundle imports the runtime before the spec, so the action root on globalThis.actions only exists after the spec evaluates. Accept a function form so the goja and web hosts resolve the root per tick. * refactor(spec): web-runtime becomes the WEB Host, delegates to shared picker Delete the duplicate picker (resolveGenerator/pickWeighted/randomTap/ randomInput/randomSwipe/randomPressKey/pickFromArray, the mulberry32 PRNG, and the snake_case serializeAction) plus the __sanderling__ action factory binds. web-runtime now implements Host (platform/seedHi/seedLo from the injected 64-bit seed via BigInt, queryCandidates over the live DOM with a per-tick cache, reportUnsupported) and calls installRuntime so both engines run pick.ts over the same Pcg. Swipe/longPress/scroll follow the verbs.ts matrix instead of silently returning null. Keeps the DOM helpers (selector translation, queryElement, elementHandle, buildState, sanitize, extractors) and the global locking. Net -214 lines (741 -> 527). * test(spec): cover the WEB Host surface and seed precision Replace the deleted-picker tests with Host coverage: platform()==web, seedHi() parsing a 64-bit seed without Number precision loss, seedLo()==0, reportUnsupported warning, the installed next-action/extractor globals, and queryCandidates verb routing + per-tick caching over a querySelectorAll stub. * refactor(spec): picker emits native selector + scroll endpoints, setup precedence * feat(spec): goja runtime entry wires the shared picker over the Go host * feat(bundler): optional RuntimeFile prepends a runtime-entry import via stdin * feat(testrun): bundle the goja runtime entry so the verifier runs the shared picker * refactor(spec): drop the legacy goja bridge fields from action factories * feat(spec): serialize selector-only string targets for the runner to re-resolve * refactor(verifier): one DecodeAction reads the unified flat wire contract * refactor(verifier): goja host + shared picker replace the duplicate Go picker * refactor(runner): decode V8 actions via the unified DecodeAction; wire goja runtime * test(verifier): author specs through the shared picker path * test(runner): bundle authored specs with the goja runtime entry * feat(verifier): collect unsupported verbs for the run report * refactor(runner): collapse WebDriver forks behind ActionSource/ExtractorSource * feat(testrun): surface unsupported verbs in run report * test(verifier): cross-runtime goja/node parity gate on the shared picker * test(verifier): unsupported verbs collected deduped in first-seen order * test(runner): summary reports no unsupported verbs on a clean run * test(spec): golden-fixture cross-runtime parity gate for the node picker Replace the env-driven parity harness with a shared scenario module and a committed golden the node picker asserts independently. The goja side asserts the same golden, so neither runtime invokes the other at test time. * test(verifier): assert goja picker against the same cross-runtime golden Drop the node-subprocess coupling: the goja side now installs a stub __sanderlingHost__ with the fixed candidate list and asserts the committed golden, matching pkg/spec/test/parity.test.ts. * refactor(spec): rename pressKey generator export to pressKeys * refactor(spec): update barrel re-exports for pressKeys * test(spec): update pressKeys generator export name * docs(spec): rename pressKey generator to pressKeys * refactor(spec): extract samplerRng into shared sampler-rng module * feat(spec): add fluent seeded value generators (strings/integers/emails/edgeCaseText) * test(spec): cover fluent value generators determinism and chaining * refactor(bundler): inject globalThis trailer from spec named exports * refactor(bundler): reuse registration trailer in web bundler * test(bundler): cover named-export globalThis registration * feat(spec): add named() to Extracted handle type * feat(web-runtime): named() and cross-extractor read guard * feat(verifier): named() and cross-extractor read guard in goja * test(verifier): cross-extractor read guard and named() * test(web-runtime): export runtime and extractors for tests * test(web-runtime): named() and cross-extractor read guard * refactor(folio): drop manual globalThis trailer (bundler injects it) * refactor(folio): seed txn amounts via integers().between(1,500) * refactor(folio-web): drop manual globalThis trailer (bundler injects it) * fix(folio-web): seed card/txn-type selection via from().generate() for reproducible runs * refactor(folio-web): weight valid generators against edgeCaseText for names/amounts * refactor(folio-web): name extractors so violation witnesses are readable * fix(web-runtime): propagate extractor getter throws and unpoison locked global Stop swallowing getter errors in evaluateExtractors so the cross-extractor read guard aborts loudly, matching goja's PushSnapshot. Make the __sanderling__ lock configurable (still non-writable) so a shared test process can reinstall a fake. * test(spec): install fake runtime via defineProperty to survive locked global * test(web-runtime): assert uncaught cross-extractor read aborts evaluateExtractors * feat(runner): add MaxSteps bound to Options * test(runner): MaxSteps stops after exactly N steps * test(driverpb): drop proto getter round-trip tautology * test(sidecar): drop stub-mode placeholder tautology tests * test(mock): drop default-field-value assertion test * test(ltl): drop Verdict.String tautology tests * refactor(runner): extract RenderSummary for snapshot testing * test(runner): golden snapshots for trace stream and violation summary * feat(web-runtime): capture uncaught errors into state.exceptions * test(integration): add throwing and counter web fixtures * test(integration): add specs for the web fixtures * test(integration): drive web fixtures through the real pipeline in headless Chrome * chore(make): add test-browser target for the Chrome-driven suite * ci: run the Chrome-driven browser suite in a separate job * refactor(test): relocate browser suite to test/browser * refactor(permissions): delete dead internal/permissions package * refactor(test): rename package to browser_test * refactor(sidecarassets): rename internal/sidecar to internal/sidecarassets * chore(make): point test-browser at test/browser * docs(decisions): record internal/permissions deletion * refactor(doctor): use sidecarassets package * refactor(testrun): use sidecarassets package * fix(test): resolve testdata relative to browser_test.go * refactor(verifier): remove dead __sanderlingIndex compat alias * refactor(bundler): use encoding/json for JS string literals * docs(action-space): use vendor-neutral native driver wording * refactor(hierarchy): scrub backend tool name from comments * refactor(driver): scrub backend tool name from comments * refactor(driver): add DoubleTap and DoubleTapSelector to DeviceDriver * refactor(sidecar): implement DoubleTap with the sub-100ms inter-tap gap * refactor(chrome): implement DoubleTap as two taps with the gap * refactor(mock): record DoubleTap and DoubleTapSelector actions * refactor(runner): delegate double-tap to driver, drop gesture timing * test(runner): assert double-tap delegates to driver DoubleTap * docs(cmd): add package docs to CLI and developer tools * docs(driver): add package docs to driver interface and chrome backend * docs(driver): add package docs to mock and sidecar backends * docs(platform): add package docs to android and ios device prep * docs: add package docs to bundler and inspect * docs(ltl): add package doc to temporal logic evaluator * docs: add package docs to runner and testrun pipeline * docs: add package docs to trace and verifier * docs(sidecarassets): add package doc for embedded JAR loader * fix(chrome): add disable-dev-shm-usage so Chrome starts in CI * test(chrome): gate real-Chrome driver tests behind the browser tag * chore(make): run chrome driver tests in the browser job * fix(web-runtime): guard global error listeners for non-browser hosts The module registered window error/unhandledrejection listeners at top level, which threw under Node (the spec-api test runner) where globalThis.addEventListener is absent. Register only when the API exists; the real browser run is unaffected. * ci(browser): re-enable unprivileged user namespaces for headless Chrome ubuntu-latest moved to 24.04, whose AppArmor restriction on unprivileged user namespaces stops headless Chrome from opening its DevTools socket even with --no-sandbox, surfacing as the driver's 'websocket url timeout'. Relax the sysctl for the job and add a direct launch check so a future breakage shows Chrome's own stderr rather than an opaque driver timeout. * ci(browser): pin stable Chrome for the driver tests setup-chrome's default latest pulled a dev Chromium (150) whose remote debugging socket never came up under chromedp, while plain --dump-dom worked. Pin the stable channel, which the driver is tested against. * feat(defaults): add scroll and rebalance action weights Use relative-integer weights (taps/typing co-primary 100, scrolls 50, swipes 25, doubleTaps 10); the picker normalizes by their total. Adds scrolls to defaultActions as a first-class reveal behavior. * feat(defaults): trim scroll action weight wiring * fix(build): point sidecar jar ignore and embed paths at sidecarassets * test(defaults): drop stale longPresses re-export assertion longPresses is opt-in vocabulary, no longer re-exported from defaults/actions.ts since e0d3b20; its builtin resolution is already covered by api.test.ts. Trim the defaults test to scrolls, which is an actual default export. * fix(chrome): raise DevTools websocket read timeout to 60s Chrome cold-start on a loaded CI runner can exceed chromedp's 20s default for reading the DevTools websocket URL, flaking the browser tests with "websocket url timeout reached". Give launch more headroom.	2026-06-02 09:52:53 +05:30
pj	88db9653e5	refactoring default action layer (#51) ... * feat(hierarchy): add editable signal with native derivation * feat(chrome): emit editable flag in hierarchy dump * feat(verifier): expose editable on ax element objects * feat(spec): add editable to selector and element types * feat(verifier): register typing builtin generator * feat(verifier): typing builtin types edge-case corpus into editable fields * feat(spec): export typing builtin generator * feat(spec): add defaultActions bundle * feat(spec): export @sanderling/spec/defaults subpath * feat(folio): layer defaultActions breadth over targeted flows * test(verifier): typing builtin targets editable fields, declines otherwise * test(hierarchy): editable derivation and selector matching * test(spec): defaultActions, typing, and defaults barrel resolve * fix(testrun): alias @sanderling/spec/defaults for the bundler * test(chrome): editable flag for inputs, textarea, contenteditable * feat(spec): typing builtin for the web (V8) action path * chore(folio): auto-boot a bootable AVD in just test/install when none connected * feat(driver): add ForegroundChecker optional capability * feat(android): detect foreground package via adb dumpsys * feat(sidecar): implement ForegroundApp via adb for android * feat(runner): relaunch app when foreground escapes during exploration * fix(spec): drop hardware back from defaultActions to stay in-app * feat(spec): add DoubleTap action type and constructor * feat(spec): wire DoubleTap through web-runtime serializer * feat(verifier): bind doubleTap and decode DoubleTap actions * feat(runner): dispatch DoubleTap as two taps inside one step * test(doubleTap): cover constructor, verifier round-trip, and runner dispatch * feat(folio): add noDuplicateTxnPerStep invariant and doubleSubmitTxn action * fix(folio): track ledger row count across non-ledger steps; pin reproducer seed * feat(spec): add doubleTaps random-target builtin to defaultActions * feat(verifier): add doubleTaps random-target generator * refactor(folio): drop doubleSubmitTxn; fuzzer surfaces double-submit via defaultActions * fix(folio): make ledgerRowsSeen monotonic to suppress transient-render false positives * feat(verifier): track newly-violated property set per step Sticky `always(P)` violations re-surfaced on every step after onset, flooding traces and summaries with duplicate records. EvaluateProperties now diffs against the prior verdict map and records the onset set; a new NewlyViolatedProperties accessor exposes it so callers can emit each violation exactly once at its onset step. The verdict-map return is preserved for residual / current-verdict consumers. * refactor(runner): emit onset-only violations to trace and summary Switch the per-step violation list from the sticky verdict map to the verifier's onset set. Each property now appears exactly once across a run: at the step it first violates, not on every subsequent step where the residual stays false. Removes the dead violationNames helper. * style(verifier): use maps.Copy for verdict snapshot * fix(folio): make login spec content-driven (idempotent across re-entries) * fix(verifier): canonicalize selector strings Object/chain JS selectors used to fall through to goja's default stringification, producing "[object Object]" tags that surfaced as garbage in trace.action.selector. Emit canonical "k:v" / " > "-joined strings instead so the tag round-trips back through the hierarchy selector grammar. * refactor(folio): replace txn invariants with balanceMatchesAddedTxn Collapse noDuplicateTxnPerStep and newTxnChangesBalance into a single per-row property: every newly-appearing ledger row's signed amount must match the ledger balance delta. A double-submit lands two rows whose individual amounts cannot both equal the aggregate delta, so each row fires the property, catching both the row-count and balance-math classes of bug under one semantic invariant. * refactor(trace): drop WriteScreenshotAfter Only one screenshot per step is captured now (concurrently with hierarchy after settle), so the -after.png variant is unused. * refactor(runner): one concurrent screenshot per step Move screenshot capture into the post-action errgroup so it observes the same UI moment as the hierarchy fetch. Drop the pre-action and deferred -after captures. Skip WaitForIdle when the action is Wait since the wait itself provides settling time. * refactor(inspect-ui): use next step's screenshot for state after Each step now has one screenshot (the moment of observation). The "state after" view of step N is the same moment as step (N+1)'s observation, so reuse that file rather than expecting a separate -after.png. * feat(sidecar): structural-hash settle poll Add pollUntilStable and structuralHash helpers; wire them into the Stub, Maestro, and iOS backends' waitForIdle. The structural hash ignores bounds-only flicker (measure passes) but trips on any change in resource-id/class/content-desc/text, so a Compose cross-fade where both source and destination composables are momentarily alive no longer slips through Maestro's waitForAppToSettle and contaminates the next hierarchy fetch. * test(sidecar): cover pollUntilStable and structuralHash Verify the poll returns on two equal snapshots, after transient churn, and at the cap when never stable; assert the hash ignores bounds-only flicker and detects content changes. * feat(spec): accept optional name on extract() Add an (name, getter) overload so each extractor handle carries a debuggable label that future trace fields (per-step diffs) can key off. The web-runtime falls back to extractor_\${index} when none is supplied so existing call sites keep working unchanged. * test(spec): cover extract name overload Verify the runtime receives an undefined name in the legacy shape, the supplied name in the (name, getter) shape, and that extract("name") with no getter throws. * feat(verifier): name extractors for diff surfacing bindExtract accepts an optional name argument; falls back to extractor_N when omitted. The name is stored on extractorState alongside prev/curr value caches that the next change will use to emit per-step diffs. * chore(folio): name every extract() call Give each extractor in the Folio spec a debuggable label so the inspect UI can render extractor-value diffs at violation steps keyed by intent (ledgerRows, route, ledgerBalance, ...) rather than by registration index. * feat(verifier): track extractor value transitions Cache each extractor's prior and current JSON-encoded value during PushSnapshot; expose ChangedExtractors to surface per-step diffs the runner can emit into the trace. The first observation flushes every non-null extractor as a change so the inspect UI shows initial state breadcrumbs alongside later transitions. * test(verifier): cover ChangedExtractors diffs Verify initial snapshot reports both named and fallback-named extractors, a subsequent change surfaces prev/curr, and a no-op snapshot leaves the diff empty. * feat(trace): emit extractor_changes per step Add ExtractorChanges to trace.Step and a runner helper that converts the verifier's diff map into the trace shape. The inspect UI keys its violation breadcrumbs off this field. * feat(inspect-ui): render extractor-change breadcrumbs at violations Show prev -> curr for each extractor whose value changed on the selected step, anchored under the violation row in ActionList. Long values collapse into <details> so the inline diff stays readable while the full payload is one click away. * fix(sidecar): cap stability poll independently of settle budget The previous shape halved durationMillis between waitForAppToSettle and the structural poll, then hammered hierarchy() at 80ms intervals - on Maestro this stacked enough RPCs that hierarchy fetches began timing out under load and the run stalled. Pass the full budget to waitForAppToSettle and cap the follow-up structural poll at 600ms with a 120ms interval, so the device sees at most a handful of extra hierarchy reads per step. * feat(cli): default --clear-data on so runs start fresh * feat(sidecar): streak-based settle with route-transition detection Two changes layered into the stability poll: 1. stabilitySnapshot returns null while the tree carries more than one route-level Screen tag (resource-id / testTag / identifier ending in "Screen"), so the poll cannot declare a NavHost cross-fade stable. Apps following the Compose route convention get this detection for free; apps that don't fall through to the generic signal below. 2. pollUntilStable now requires an uninterrupted stable streak of at least MIN_STABLE_STREAK_MILLIS rather than just N consecutive matches. A late transition that fires after a brief calm window breaks the streak instead of slipping past. Interval widened to 250ms so UiAutomation isn't hammered under fuzz load. * test(sidecar): cover streak reset and route-transition rejection Verify the poll honors MIN_STABLE_STREAK_MILLIS, that a transient mid-stream change resets the streak, that null returns block streak progress through a NavHost cross-fade, and that stabilitySnapshot counts only route-level attribute keys when summing Screen tags. * feat(runner): re-fetch on transitional hierarchy capture Some actions trigger async work (DB write, ViewModel coroutine) whose navigation transition begins after the sidecar settle poll has already exited. Without intervention, the next iteration's hierarchy fetch lands mid cross-fade and the verifier observes a partial extractor state which then surfaces as a false-positive violation at the step where the transition completes. fetchSyncedState pairs hierarchy + screenshot in one goroutine and retries the pair (up to 4 times, 200ms apart) while the captured tree contains more than one route-level *Screen tag. Steps that observe no transition get no added cost; steps that catch a transition pay up to ~600ms extra wall time but record a tree that matches the post-transition state the property language expects to compare. * feat(runner): gate first action on app reaching foreground * test(runner): cover startup foreground gate and back-press * feat(verifier): scope random-action targets to app package Random tap/doubleTap/type/swipe candidates now exclude nodes whose package differs from the app under test, so exploration never fuzzes the soft keyboard, system UI, or permission dialogs. An unset app package or an element with no package stays in scope, preserving behavior on iOS. * feat(testrun): pass app package into verifier scope filter * test(verifier): cover package-scoped target selection * feat(hierarchy): derive package from resource-id prefix The Android sidecar omits an explicit package attribute, so the verifier's package scope filter was a no-op and the keyboard still leaked into targets. Native nodes carry their package as the resource-id prefix; derive it there when the attribute is absent. Compose testTags are colon-less and stay empty, keeping them in scope. * test(hierarchy): cover package derivation from resource-id * chore: stop tracking inspect-ui/dist build artifacts * feat(android): detect focused-window package via dumpsys window * feat(driver): add FocusedWindowChecker capability * fix(runner): gate first observe on the app window being drawn, not just resumed * test(mock): add FocusedWindowApp with foreground mirroring * test(runner): cover startup gate waiting for app window to draw * feat(proto): add Snapshot RPC for atomic hierarchy+screenshot Pairs hierarchy and screenshot in a single response so the runner can capture both under a backend mutex, avoiding the cross-fade race where the two reads describe different frames. * feat(sidecar): add snapshot default on DriverBackend Default impl calls hierarchy() then screenshot(). The service layer wraps the call in a mutex so concurrent runners observe a serialized pair. * feat(sidecar): wire Snapshot handler with serialization lock Synchronizes backend.snapshot() so concurrent runners observe a serialized hierarchy+screenshot pair, eliminating the cross-fade race where two parallel reads describe different frames. * test(sidecar): cover Snapshot wire path and serialization lock SnapshotHandlerTest asserts both fields are populated, concurrent calls are serialized, and the default impl runs hierarchy then screenshot. * feat(driver): expose Snapshot on DeviceDriver and sidecar client Snapshot wraps the new atomic-snapshot gRPC: the runner gets hierarchy and screenshot from one round-trip whose two reads are serialized on the sidecar side. * feat(driver): add Snapshot to chrome and mock drivers The chrome tab is single-threaded so its Snapshot pairs the two reads without extra locking. The mock records ActionSnapshot so tests can assert the runner reaches for the paired RPC. * refactor(runner): observe each step via the atomic Snapshot RPC fetchSyncedState now issues one Snapshot per attempt so hierarchy and screenshot describe the same on-device frame. The transitional retry stays: that case handles a fully-captured but mid cross-fade frame, which atomic capture cannot fix. * test(runner): assert step uses Snapshot, not raw hierarchy/screenshot TestRunner_UsesAtomicSnapshot catches regressions to the two-goroutine race, and the existing parallel-fetch test now keys off ActionSnapshot. * test(driver): cover Snapshot in proto descriptor and sidecar client Adds Snapshot to the descriptor allowlist and a sidecar-client test that asserts both fields come back over the wire. * feat(trace): add Transitional flag to Step * fix(runner): skip verifier for transitional trees after retry budget When fetchSyncedState exits its retry loop with a tree that still shows a NavHost cross-fade, the runner now marks the step transitional, writes the step + screenshot to the trace, and skips Verifier.PushSnapshot / EvaluateProperties / ChangedExtractors so the previous-to-current extractor advance is not poisoned by transient state. The next clean step's previous still references the prior clean state. NextAction continues to run so the loop never deadlocks on a never-stabilizing screen. * test(runner): cover transitional step skips verifier and clean control * refactor(trace): rename Step.Action to Step.NextAction The trace step's action field is the action chosen FOR THE NEXT iteration based on observing this step's hierarchy, not the action that produced this step. Rename Step.Action to Step.NextAction and the JSON tag to next_action to make causality explicit at the data level. * refactor(runner): assign trace action to Step.NextAction field Follows the rename of trace.Step.Action to Step.NextAction. The runner already computed the next iteration's action here; only the field name changes. * refactor(inspect): decode trace step's next_action JSON field Mirrors the trace schema rename of action to next_action. The summary shape exposed to the SPA (action_kind/action_label) keeps its current JSON tags since these are derived labels, not the raw next-action. * test(inspect): update fixtures to use next_action trace field Aligns inspect tests with the trace schema rename. Step constructors now set NextAction and the JSONL fixtures use the next_action tag. * refactor(inspect-ui): rename Step.action to Step.next_action Aligns the SPA type and consumers with the trace schema rename. The StepSummary.action_kind/action_label labels stay unchanged since they are derived labels, not the raw next-action. * fix(folio): extract balanceMatchesAddedSum predicate as testable helper Move the ledger-balance-vs-added-rows predicate into a pure helper module so the property's logic is unit-testable in isolation. Marks the sanderling example as an ES module so cross-package ESM imports resolve under node. * fix(folio): use sum-of-added-rows in balanceMatchesAddedTxn The old predicate (every row's signed amount equals delta) silently passed the double-submit bug because two same-amount rows each match the delta in isolation. Switching to the sum check (addedSum === delta) catches both the double-submit case and any future multi-row append whose total drifts from the balance change. * test(spec): cover balanceMatchesAddedSum single, sum-match, over, under cases Pins the sum-based predicate: a single new row matching delta and two new rows summing to delta both hold; two-row over-sum (double-submit) and under-sum cases both violate. * fix(build): rebuild sidecar JAR when Kotlin sources change Without source-file deps on $(SIDECAR_JAR), make never re-ran shadowJar after a Kotlin edit, so a stale embedded JAR shipped on every install and the new sidecar code was silently absent at runtime. * fix(chrome): launch with no-sandbox so headless Chrome starts in CI * fix(sidecar): type text at cursor instead of clearing the field InputText now appends at the focus caret, matching the native driver and the standard mobile-input contract, instead of deleting existing content first. Adds an injectable command runner so the behavior is testable without a device. * test(sidecar): assert InputText types at cursor without clearing Captures the adb command stream and verifies a single input-text call with no preceding delete keyevents, plus the adb escaping cases. * feat(proto): add LongPress RPC * chore(proto): regenerate Go stubs for LongPress * feat(driver): add LongPress to DeviceDriver interface * feat(sidecar): add LongPress client method * feat(mock): record LongPress action * feat(chrome): implement LongPress as press-and-hold * feat(sidecar): implement longPress across backends * feat(sidecar): dispatch LongPress RPC to backend * test(sidecar): cover LongPress dispatch * test(sidecar): implement longPress in snapshot test backend * feat(verifier): add LongPress and Scroll action kinds * feat(folio-spec): predicate that gates balance check on TxnSubmit tap Replaces the row-sum predicate (which always held by construction since balance is derived from rows in Folio) with one that compares the typed amount to the actual balance delta after a tap on TxnSubmit. Catches the planted double-submit bug. * feat(folio-spec): wire submitMovesBalanceByTypedAmount property Adds lastAction and totalBalance extractors and uses them in the new property. Drops ledgerRows/ledgerBalance extractors since nothing else referenced them. * feat(verifier): wire longPresses and scrolls generators * test(verifier): cover longPresses and scrolls generators * test(folio-spec): unit tests for submitChangesBalanceByTypedAmount Covers single vs double submit, the DoubleTap variant, vacuous cases (null action, wrong kind, wrong target, zero typed), and selector-as- object coercion. * feat(spec): add LongPress and Scroll authoring surface * feat(spec): no-op LongPress and Scroll in web runtime * feat(spec): re-export longPresses and scrolls as opt-in generators * test(spec): cover LongPress and Scroll runtime members * test(proto): expect LongPress in service descriptor * feat(runner): dispatch LongPress and Scroll actions * test(runner): cover LongPress and Scroll dispatch * docs(action-space): move LongPress, Scroll, DoubleTap to current actions * fix(runner): mark nil/empty hierarchy as transitional A failed or empty sidecar hierarchy fetch was pushed straight to the verifier, letting spec extractors crash with "Cannot read property 'map' of undefined" when findAll returned null. Treat that case like a transitional capture: skip the verifier push, still record the step, and keep the loop progressing. * fix(verifier): populate Action.On when tap chooser picks an element Coordinate-targeted Taps/DoubleTaps left On empty, so action-gated properties reading lastAction.on couldn't tell which target was hit and were vacuously skipped. Resolve the picked element to a stable key:value selector (resource-id, testTag, text, desc) and validate it resolves back to the same element so we don't accidentally redirect the tap to a sibling that shares the identifier. * fix(folio): add parseTypedAmount helper matching app's parseCents Raw user input like "50" must become 5000 cents, not 50. The existing parseDollarCents helper strips non-digits and so reads "50" as 50 cents, which is correct for formatted balance text but off by 100x for raw input from the amount field. * fix(folio): parse raw amount input as cents in submit predicate txnAmountField holds raw user keystrokes, not formatted balance text. Route it through parseTypedAmount so "50" reads as $50, matching how the app commits the transaction. * fix(folio): carry forward total balance across off-screen transitions AddTransactionScreen shows neither AccountCard nor LedgerBalance, so the extractor used to report 0 at the step before submit. That made every non-zero current balance look like the full delta and tripped the typed amount property on every honest submit. Remember the last-seen sum and return it whenever the current snapshot has no balance signal. * test(folio): cover submit predicate with raw typed-amount inputs Pipes realistic raw keystrokes through parseTypedAmount + the predicate so single submits clear and double submits fire as expected. * feat(folio): add computeHomeTotalBalance helper Pure helper that tracks Home multi-account total only and carries the last Home sum across off-Home steps. Ledger's single-account balance is excluded because mixing it would corrupt cross-screen scale comparisons. * fix(folio): totalBalance carrier tracks only Home, not Ledger Home cardSum is a multi-account total; Ledger's LedgerBalance is a single account on a different scale. Blending them in the carrier produced bogus cross-screen deltas (prev from Ledger, curr from Home), triggering false positives in submitMovesBalanceByTypedAmount. Restrict the carrier to Home AccountCard totals via the computeHomeTotalBalance helper. * test(spec): cover computeHomeTotalBalance carrier behaviour Tests Home sums, carrier passthrough on off-Home steps, the Ledger scale-mismatch case, and a Home > off-Home > Home sequence. * feat(runner): treat transient apply errors as transitional steps Sidecar input RPCs occasionally hang with DEADLINE_EXCEEDED or UNAVAILABLE on long fuzzing runs. The per-step loop previously propagated any applyAction error and killed the run after a single flake. Detect transient gRPC failures via status.FromError, mark the step transitional, skip the post-action idle poll, and continue to the next step. Fatal errors (outer ctx cancellation, non-transient codes, verifier crashes) still propagate. * test(runner): cover transient apply error resilience TestRunner_TransientApplyErrorMarksTransitional drives the runner through a wrapper that fails the first TapSelector with a gRPC DeadlineExceeded then succeeds. Asserts the run does not exit, the failed step is marked transitional with no violations, and the next step runs cleanly. TestIsTransientApplyError_Classification covers the helper's matching rules directly so future code changes don't quietly drop a transient case. * fix(folio): gate submit-balance property on Home route landing totalBalance is only freshly computed when AccountCards are visible on Home; off-Home landings return the carrier and would false-fire the property, latching always(next(F)) to false and masking the real double-submit bug. Skip vacuously when route is not "home". * test(spec): cover route gate in submit-balance predicate Adds route arg to existing cases (all use "home") and adds five new cases: ledger landing with stale carrier, add-transaction with double-insert delta, null route, plus home-landing positive and double-insert negative cases anchoring the gate's allow path.	2026-06-01 12:48:51 +05:30
pj	b23fb0c723	feat: web-native specs + per-platform doctor (#49) ... * feat(doctor): per-platform check sets + --platform flag Replaces unconditional defaultDoctorChecks with doctorChecksFor(platform); web-only users no longer see scary FAILs for adb/emulator/java/sidecar. * feat(testrun): add Preflight() before sidecar/driver setup Mobile platforms get a friendly install hint pointing at `sanderling doctor --platform=<p>` instead of `fork/exec java: not found`. Web is a no-op. * refactor(chrome): split tag (HTML name) from class (CSS classList) Hierarchy attributes now expose HTML tag under 'tag' and CSS classes under 'class', stopping the conflation of the two. * feat(chrome): translate legacy string selectors to CSS/XPath TapSelector now maps id:/desc:/descPrefix:/testTag:/etc. through TranslateStringSelector. Unknown prefixes pass through to a CSS attribute selector so a future Maestro key works without a release. * feat(trace): add WriteHTML + Step.HTMLAvailable Per-step HTML lives in <run>/html/step-NNNNN.html so trace.jsonl stays line-greppable on apps with hundreds-of-KB DOMs. * feat(driver): add WebDriver capability + chrome implementation WebDriver exposes InstallBundle/EvaluateExtractors/NextActionFromV8/Document for the V8-native web tick path. Mobile drivers stay binary-compatible. * feat(verifier): OverrideExtractorValues for V8-driven extractors Web tick path runs extractor bodies in V8 against the real DOM, then overrides goja-side .current slots so LTL predicates evaluate against those values. Mobile callers can pass nil for a no-op. * feat(spec): add WebState + camelCase attribute aliases WebState extends State with live `document`/`window` for V8-side web extractors. KnownAttrSelectors gains camelCase aliases (contentDescription, ariaLabel, testID, etc.) so cross-framework specs autocomplete. * feat(runner): per-tick HTML capture for WebDriver-capable drivers Type-asserts driver.WebDriver and writes <run>/html/step-NNNNN.html in parallel with screenshot/hierarchy/metrics. Step.HTMLAvailable flips so the inspect UI can hide the html tab on mobile runs. * feat(inspect): serveHTML route under /api/runs/<id>/html/<name> Mirrors serveScreenshot path validation; rejects traversal segments and unknown extensions. text/html content-type so the iframe renders cleanly. * feat(bundler): BundleWeb + V8-side runtime shim web-runtime.ts installs globalThis.__sanderling__ with extractor / action registries, plus __sanderlingExtractors__ + __sanderlingNextAction__ globals. BundleWeb composes user spec + runtime under esbuild's PlatformBrowser into one IIFE. * feat(runner): V8 extractor overrides + V8 action source for WebDriver When the driver implements WebDriver, the runner sources extractor values from V8 (real DOM) and the next action from the V8-side action generator. LTL property predicates still run host-side in goja. * feat(testrun): bundle + install web runtime when platform=web BundleWeb composes the user spec with web-runtime.ts; the chrome driver installs the resulting IIFE via Page.AddScriptToEvaluateOnNewDocument post-Launch so the per-tick V8 extractor + action evaluation can begin on step 1. * feat(inspect-ui): hierarchy + html panels in run detail HierarchyPanel renders the captured DOM/AX tree with a filter input. HtmlPanel renders the per-step HTML in an iframe (sandboxed) with a toggle to view source. HTML tab only shows when the step actually has HTML captured. * fix(folio-web): drop aria-label data-carrier abuse Account cards now expose data-account-id + data-balance attrs and use a human-readable aria-label. total-balance / ledger / ledger-balance carry data-cents and data-txn-count instead of stuffing values into title. Spec rewritten to read structured attrs via object-form selectors. * chore: rebuild inspect-ui dist + folio-web .gitignore Embeds the new HierarchyPanel + HtmlPanel into the inspect-ui dist that ships with sanderling. Adds folio-web/.gitignore so generated runs/ don't leak into commits. * revert(trace): drop WriteHTML + Step.HTMLAvailable Screenshots already cover inspection; HTML capture bloats disk by 50-200MB per run with no payoff. * revert(runner): drop per-tick HTML capture Removes captureHTML helper and its three call sites; HTMLAvailable flag no longer set on Step. * revert(driver): drop WebDriver.Document Document was only consumed by the runner's HTML capture which is gone. * revert(inspect): drop /html route Removes htmlPathPattern, serveHTML, and the dispatch block that called it; HTML capture no longer exists on disk. * revert(inspect-ui): drop htmlUrl + html_available type API surface no longer needs the HTML route; Step.html_available has no producer. * revert(inspect-ui): drop HtmlPanel + html tab Removes the iframe-based HTML viewer and its before/after tab wiring from RunDetail. * test(inspect-ui): drop htmlUrl test, add @types/bun Pulls bun-types into tsconfig so api.test.ts (which uses bun:test) typechecks; this was broken from the original feature commit. * chore: rebuild inspect-ui dist without HtmlPanel Embedded SPA bundle no longer ships the iframe HTML viewer. * fix(web-runtime): retry action resolution + implement taps/swipes V8-side runtime previously returned null when weighted picked a generator that returned [] (page-gated), causing 80%+ of ticks on narrow routes to emit no action and no post-screenshot. Now retries up to 16x like goja, and the taps/swipes builtins query the live DOM for clickable elements / dispatch random swipes instead of returning null. * fix(web-runtime): drop swipe, restrict pressKey to browser-meaningful keys Web has no swipe gesture, so swipes dispatched pointer events into empty divs. Make swipe() and the swipes builtin no-op. For PressKey, replace the always-"back" choice with a random pick from {enter, tab, escape, up, down, left, right} - keys that have real semantics in a browser. * chore(folio-web): drop swipes from action root Web runtime no-ops Swipe; remove the import and weighted entry so the spec doesn't request actions that won't fire. * fix(inspect-ui): correct HierarchyPanel CSS variable names Tokens --surface-1/--surface-2/--text-secondary/--border-subtle don't exist in tokens.css, so sticky thead had no background and tag/bounds text fell back to inherited color. Map to the canonical --surface, --surface-elevated, --text-muted, --border that other panels use. * fix(chrome): correct PressKey mappings to chromedp/kb constants Old keyMap had "home":"\x00" (NUL byte) and arrow keys mapped to random punctuation runes (\x25-\x28 = % & ' () instead of arrow keys. "escape" was missing entirely while the V8 runtime emits it. Drop back/home (no browser navigation semantics) and route the remaining keys through chromedp/kb constants so they actually dispatch as the named keys. * fix(cli): -h/--help exits 0 instead of error code parseDoctorArgs hand-rolled its own flag loop and surfaced help text as an error; parseTestArgs used flag.ContinueOnError but propagated flag.ErrHelp to main() which printed "error: flag: help requested" and exited 1. Switch parseDoctorArgs to flag.NewFlagSet matching parseTestArgs, then recognise flag.ErrHelp in main() so all subcommands exit 0 on -h. * fix(chrome): harden cssEscape for control chars + use [class~=] Previous cssEscape only handled " and \, leaving NUL/newlines/control chars to break out of the CSS string literal. Port the CSSOM string serialization rules: NUL becomes U+FFFD, control chars become \HEX, quotes/backslashes get escaped. Class selector switched from `.x` (which would need separate identifier escaping) to `[class~="x"]`, which is also semantically correct for multi-class elements. * fix(web-runtime): use CSS.escape and validate tag-name selectors The previous cssEscape only handled " and \, leaving newlines/control chars to break out of attribute string literals. Delegate to the platform CSS.escape per CSSOM spec. The `tag` selector branch returned the bare value through cssEscape, which doesn't prevent pseudo-classes (`*:hover`) from injecting into the surrounding selector. Add a positive whitelist; values that don't match a tag-name pattern collapse to a never-matching `:not(*)`. Also switch class selectors to `[class~="..."]` to remove the only identifier-context use of cssEscape. * fix(chrome): validate attribute name in unknown-prefix branch A selector like `foo]:has(*),body[x:value` previously produced [foo]:has(*),body[x="..."], a syntactically valid CSS selector that escaped the attribute match and selected `body`. Reject anything that isn't a plain HTML attribute name. * fix(selectors): emit valid XPath 1.0 string literals via concat() Both the Go translator and the V8 runtime escaped " by prepending \, which XPath 1.0 doesn't accept (its string literals have no escape syntax). A `text:` value containing a quote produced malformed XPath that chromedp/document.evaluate rejected. Use the standard concat() composition: when the value contains both ' and ", split on " and join with `, '"', ` so each fragment is wrapped in single or double quotes individually. * fix(runtime): surface unresolved action targets instead of dropping silently serializeAction emitted {x:0,y:0} via `?? 0` whenever a Tap/InputText/Swipe target failed to resolve to coordinates. The runner then collapsed those to ErrNoAction, so every selector typo became a silent no-op tick. Have the runtime return null on unresolved targets and log a console warning (visible via chromedp's runtime listener). Drop the now-redundant {0,0} -> ErrNoAction guard so a deliberate Tap at the origin actually fires. * fix(runner): use errgroup-bound ctx so siblings cancel on failure The errgroup's bound ctx was discarded; goroutines closed over the outer ctx, so neither a sibling failure nor the future ability to propagate per-step cancellation reached the V8 extractor's CDP round-trip. Switch closures to gctx and document why Wait()'s error is intentionally discarded. * fix(chrome): propagate caller ctx cancellation to CDP calls InstallBundle, EvaluateExtractors, NextActionFromV8 ignored the caller ctx and ran chromedp.Run on d.tabCtx alone, so step deadlines and Ctrl-C couldn't interrupt an in-flight CDP round-trip on a hung tab. Add a runCtx helper that derives a chromedp-bound context which also cancels when the caller's ctx cancels, and route the three V8 entry points through it. * fix(verifier): tolerate out-of-range override indices A single stale index from V8 aborted the entire override map, so any valid entries alongside it were dropped and verification ran on stale extractor values. V8 and goja register from the same bundle so a mismatch is unusual but recoverable. Skip out-of-range entries instead of erroring, and return the skipped count so the runner logs the mismatch without losing valid overrides. * test(verifier): cover object-shaped extractor overrides Existing tests only override scalars (777, 200), so a future jsonToJSValue regression around nested object propagation would slip through. Lock down the contract: a JSON object override should make {attrs.testTag, balance} readable from goja predicates. * fix(web-runtime): lock global runtime hooks against page shadowing AddScriptToEvaluateOnNewDocument runs first, but a page script can still delete or replace window.__sanderling{,Extractors__,NextAction__} between install and host invocation. Define them as non-writable, non-configurable properties so any attempt to shadow them throws in strict mode rather than silently breaking the run. * perf(web-runtime): cache randomTap candidate DOM scan per tick The 16-attempt retry loop in __sanderlingNextAction__ called randomTap repeatedly; each call ran querySelectorAll over a-button- input-... and re-flushed layout per match via getBoundingClientRect. On heavy SPA routes that's the per-tick budget gone. Cache the scan in a module-level slot, reset at the top of each __sanderlingNextAction__ invocation so the cache doesn't outlive a tick. * fix(web-runtime): cap sanitize recursion to prevent stack overflow State exposes document and window (per WebState in types.ts). A user extractor returning either crashes the runtime via stack overflow on the circular DOM/Window references. Track seen objects in a WeakSet and bail at depth 32 so the worst case becomes a truncated value, not a process kill. * fix(web-runtime): enforce pressKey allowlist in factory The factory accepted any string while randomPressKey only emitted enter/tab/escape/arrows. A spec emitting pressKey({key:"home"}) would flow through to the chrome driver, which rejects unsupported keys with a runtime error mid-step. Reject at the factory so the spec author sees the failure where it originates. * chore(chrome): drop dead bundleSource/bundleMu bundleSource was written under bundleMu but never read. Either remove it or wire a re-install path; remove until the second is actually needed. * fix(chrome): use strconv.Atoi for extractor key parsing fmt.Sscanf("%d", ...) silently accepts trailing garbage like "3abc" as 3. strconv.Atoi rejects the same input outright, so a malformed key surfaces as an error instead of a wrong-bucket override. * fix(doctor): raise per-check timeout to 15s for chromium launch 5s could time out the headless chromium check on cold CI. Most checks finish in milliseconds, so a longer ceiling doesn't slow real failures. * fix(runner): trust V8 coordinates for InputText, even at origin resolveCoordinates required strict positive X/Y, so a V8-emitted InputText for an element at viewport (0, *) or (*, 0) skipped the focus tap and typed into whatever was focused. Distinguish the selector-driven path (mobile) from the coords-only path (web V8) so edge coordinates are honored without breaking the existing tree-lookup fallback. Add applyAction tests covering both the typical web case and the (0,0) edge case. * test(bundler): lock down deterministic output across builds The review flagged map-iteration nondeterminism as a possible cause of unstable bundle SHAs. Empirically esbuild's Define handling is order- independent (parallel substitution rules), so output is already stable. Add a regression test that builds 10x with multiple Defines and asserts SHA equality so any future change that introduces ordering surfaces.	2026-05-03 11:21:21 +07:00
pj	8ccf95c1cf	refactor: rename project uatu -> sanderling (#24) ... * refactor: rename Go module path uatu -> sanderling Module path github.com/priyanshujain/uatu -> github.com/priyanshujain/sanderling, including all imports and the proto go_package option. Generated .pb.go files rewritten in-place; safe to regenerate with protoc later. * chore(proto): regenerate driverpb after module path rename The previous sed-based module rename corrupted the embedded descriptor byte lengths. buf generate rewrites them cleanly. * refactor: rename CLI binary uatu -> sanderling Updates Makefile target + UATU_BIN var, .goreleaser project/build IDs, .gitignore comment, and all user-facing strings in the CLI help text, error messages, and tests. Binary is now bin/sanderling. * refactor(sdk): rename Kotlin package dev.uatu.sdk -> dev.sanderling.sdk Moves sdk/android/src/{main,test}/kotlin/dev/uatu -> dev/sanderling and rewrites package declarations, imports, and the Gradle namespace. Class names (Uatu, UatuRuntime) are renamed in a follow-up commit. * refactor(sidecar): rename Kotlin package dev.uatu.sidecar -> dev.sanderling.sidecar Moves sidecar/src/{main,test}/kotlin/dev/uatu -> dev/sanderling and rewrites package declarations, imports, and the application mainClass. * refactor: rename Uatu API surface -> Sanderling - Kotlin: Uatu -> Sanderling, UatuRuntime -> SanderlingRuntime (+ files). - JS host binding: globalThis.__uatu__ -> __sanderling__ (Go verifier, spec-api, tests). - TS interface: UatuRuntime -> SanderlingRuntime; internal tags __uatuFormula / __uatuActionGenerator -> __sanderling* variants. - Go trace: UatuVersion field + uatu_version JSON tag renamed. - Socket naming: uatu-agent / uatu-agent-reader -> sanderling-agent*. - Sample app, docs, inline-JS test strings updated to match. * refactor(examples): rename examples/folio/uatu -> examples/folio/sanderling Renames the example spec directory; updates justfile paths + gitignore entries accordingly. Package.json name/description and @uatu/spec dependency are renamed in the npm + docs commits. * chore(build): rename gradle property + rootProject.name uatu -> sanderling - Renames the uatu.version gradle property and all its -P references in Makefile, build.gradle.kts files, and .github/workflows/release.yml. - settings.gradle.kts rootProject.name = "sanderling". - Renames .env.local.example header + release-cli workflow job name. * refactor(proto): rename proto package uatu.driver.v1 -> sanderling.driver.v1 Updates the proto package and java_package, regenerates driver.pb.go + driver_grpc.pb.go, rewrites Kotlin imports and the gRPC ServiceName assertion in driver_test.go. * refactor: rename npm package @uatu/spec -> @sanderling/spec Renames package name in pkg/spec-api/package.json + lockfile, all consumer imports (examples/folio spec, testdata, verifier tests), the esbuild alias in cmd/sanderling/test_run.go, and related doc references. * docs: rename uatu -> sanderling in README, docs, and URLs - README + docs/{manual,development}/*: narrative + GitHub + Pages URLs. - POM + npm package.json repo/homepage/bugs URLs. - .gitignore + embed_stub + Makefile-comment references updated to 'make sanderling'. - Minor narrative comments in cmd/sanderling/test_run.go and internal/inspect/server.go. * refactor: rename remaining internal uatu strings -> sanderling - SANDERLING_TEST_PHONE/OTP env vars (cmd + bundler tests). - sanderling-sidecar runtime tmp dir + extracted JAR filename. - Inspect web UI: @sanderling/inspect-web package, title, theme localStorage key, RunList empty-state copy, uatu_version TS field. - Sample app storage key sanderling.ledger.v1. - Test data: sanderling_test AVD name + com.example.sanderling_test. - Release docs tarball name template.	2026-04-21 11:57:49 +07:00
pj	13bb2feb82	feat: uatu inspect UI (web trace explorer) (#23) ... * feat(trace): extend Step/Action/Meta schema for inspect UI Add Step.Hierarchy, Step.Residuals, Action.Selector/ResolvedBounds/TapPoint, Meta.EndedAt and JSON tags on hierarchy.Element/Bounds/Tree so trace.jsonl can drive the upcoming uatu inspect web UI. * test(trace): cover EndedAt + new step fields round-trip * feat(ltl): MarshalJSON for Formula AST + Evaluator.Residual() Each Formula concrete type now serializes to a closed-set residual node (true/false/not/and/or/implies/always/now/next/eventually/predicate/error) that mirrors the TS spec API surface. Evaluator.Residual() folds pending obligations into a single Formula so the runner can stamp one residual per property per step into trace.jsonl. * feat(runner): stamp residuals, hierarchy, selector targets, ended_at Each Step now carries the captured hierarchy, per-property residual ASTs, and (for Tap/InputText) the selector + resolved bounds + tap point. The test_run command writes meta.ended_at on graceful shutdown so the inspect UI can distinguish completed runs from in-progress ones. * feat(inspect): scaffold embed dist for SPA assets Stage 2 stub for the inspect server. Real web bundle gets wired in Stage 4 (Makefile copies web/dist into internal/inspect/dist). * chore(web): ignore web/ build output in root .gitignore * chore(web): add bun + vite + vitest scaffold config * feat(web): monochrome design tokens, typography, app shell CSS * chore(web): placeholder for self-hosted JetBrains Mono fonts * feat(web): index.html entry with style links and root mount * feat(web): typescript types mirroring run/step trace schema * feat(web): typed fetchers for runs/steps/screenshots * feat(web): App shell with router and run/step routes * feat(inspect): runs scan, lazy step parse, mtime-aware cache * feat(web): RunList route with table, loading, and error states * feat(web): RunDetail route shell with three placeholder panels * feat(inspect): fsnotify-backed runs watcher with debounce * fix(web): use jest-dom/vitest entry so matchers register * test(web): cover listRuns happy path and error response * test(web): render RunList with mocked fetch and assert row * chore(web): commit bun lockfile * feat(inspect): http handlers for runs/steps/screenshots/SSE * test(inspect): cover handlers, screenshot whitelist, SSE, dev proxy * feat(cmd): add 'uatu inspect' subcommand * fix(web): align TS types with snake_case wire format Go inspect server serializes RunSummary, StepSummary, Step, Meta with snake_case JSON tags (matching the on-disk trace.jsonl/meta.json). Update the TS types and consumers to match so API responses parse without runtime undefined fields. Action keeps resolvedBounds/tapPoint as camelCase because those keys were defined that way in the trace schema. * feat(web): add ActionList panel for run-detail step navigation * feat(web): add SnapshotTable panel with diff highlighting Renders snapshots dictionary as a flat sorted dotted-path tree. Changed leaves get data-changed plus a hover title with the previous value. * test(web): cover SnapshotTable rendering and diff behavior Eight cases: empty state, sort order, dotted-path expansion, changed/unchanged/missing-previous flagging, and inline-vs-expanded arrays. * feat(web): add Screenshot panel with bounds and tap overlays Center column of run-detail page. Renders the device screenshot scaled to fit, with an SVG overlay drawing resolvedBounds as a violation-colored rect, tapPoint as a contrast ring, and swipes as an arrow. Falls back to a placeholder when src is missing or the image fails to load. * fix(web): guard scrollIntoView call for jsdom compatibility * test(web): cover Screenshot panel rendering and overlays * test(web): cover ActionList rendering, selection, keyboard, and markers * feat(web): add ExceptionsPanel component * test(web): add ExceptionsPanel tests * feat(web): add Timeline panel with property swimlanes Renders SVG swimlanes per property with violated/pending/holds cells, action-marker dots, click-to-seek, and a selected-step highlight bar. * test(web): cover Timeline empty state, cells, status, click, highlight * feat(web): add ResidualNode recursive AST renderer * test(web): cover ResidualNode operators, predicate, and error chip * feat(web): add ViolationsPanel with status badges and jump button * test(web): cover ViolationsPanel rows, status grouping, and jump button * test(web): register testing-library cleanup globally All six panel test files added local afterEach(cleanup); centralize it in the shared setup so future tests inherit DOM isolation by default. * feat(web): hooks for url/keyboard/theme/sse * feat(web): wire all panels into run-detail with phone-dominant grid ActionList left, Screenshot center, Snapshots/Properties/Exceptions stacked right, Timeline bottom. URL-synced step index (useStep), keyboard shortcuts (j/k/arrows/g/G/.), light+dark theme toggle stored in localStorage, SSE auto-refresh on the run index. * test(web): add three reference run fixtures (clean, violation, exception) * build: web targets in Makefile + bun in CI; docs(inspect) - Makefile: web-build/web-dev/inspect-dev/test-web targets; uatu and install now depend on web-build so the binary embeds the latest SPA. - ci.yml: setup-bun + cache; existing make test now runs web typecheck + vitest as part of the full suite. - docs/manual/inspect.md: panel reference, keyboard shortcuts, URLs. - docs/manual/cli.md: document uatu inspect. - README: link to inspect docs. * feat(runner): capture a screenshot per step The driver already exposes Screenshot(ctx), but the runner never called it. Each step now writes <run>/screenshots/step-NNNNN.png right after the trace line, using the same failure-is-a-warning posture as other best-effort observability hooks. Makes the inspect UI's center panel actually useful. * feat(inspect): include action_label in StepSummary Tap/InputText/Swipe/PressKey/Wait each get a short human-readable label (selector, quoted text, swipe direction, key name, duration) so the action list panel can render readable rows instead of just 'Tap' with no target. * test(inspect): accept either #app or #root in SPA shell fallback * feat(web): render action_label and screen in ActionList rows Step rows now show 'Tap id:save', 'InputText "alice"', 'Swipe up', 'PressKey back', etc. Steps with no action fall back to 'observe @ <screen>' so the list reads as a flow instead of a wall of '--' placeholders. * feat(sidecar): implement screencap for android driver backend Was stubbed to return an empty byte array, which made the runner's per-step screenshot capture a no-op. Shell out to 'adb exec-out screencap -p' and stream the PNG bytes back. Width/height stay zero because the PNG header carries them; the Go side can parse if needed. * feat(proto): add Metrics RPC for per-step CPU and memory capture * feat(driver): Metrics(bundleID) returns cpu_percent + heap/total bytes * feat(sidecar): implement Metrics RPC via adb top + /proc/<pid>/status * feat(runner): capture metrics + before/after screenshots per step Each step now writes step-NNNNN.png (before applyAction) and step-NNNNN-after.png (after the action + wait-for-idle). The runner samples Driver.Metrics(bundleID) before writing the trace line and stamps Step.Metrics with cpu_percent, heap_bytes, total_memory_bytes so the inspect UI can chart CPU and heap over the run. * fix(runner,sidecar): measure CPU across step via /proc stat delta 'top -d 0.3 -n 2' measures CPU in a 300ms window that coincides with the SDK-paused app, always reporting 0%. Switch to reading /proc/<pid>/stat utime+stime and computing the delta between successive calls; the natural step cadence gives a 2-5s measurement window that captures the action response and render cycle. Also moved the sample to before snapshotStep so the delta starts before the SDK pause. * feat(web): add Metrics type for per-step cpu and memory * refactor(web): replace --accent-change with --accent-positive token * refactor(web): recolor chip-progress as neutral outlined chip * refactor(web): use neutral border for changed snapshot rows * feat(web): add MetricsChart panel with HEAP and CPU lanes SVG-based time-series chart rendering heap bytes and CPU percent per step across two stacked lanes, with a shared step axis below. Lines are monochrome; a vertical highlight marks the selected step; per-step hit rects make any click seek to that step. * feat(web): revamp ActionList with tag targets, elapsed time, and expandable rows Render selector-based Tap actions as <tag/> markup, show zero-padded MM:SS.mmm elapsed time per row, and expand the active row with Position/Content sub-rows when a full Step is available. Adds formatActionRow/formatElapsed helpers and covers both with unit tests. * fix(runner): stop copying Tap selector into action.text The 'Content' inspect row should show the user-supplied text for InputText actions and stay empty for Taps. Previously the runner copied action.On into traceAction.Text for both, so the inspect UI showed the selector as the tap's 'Content'. * fix(web): use text-muted for swipe arrow after accent-change removal * fix(web): snapshot values truncate with ellipsis + title tooltip Long JSON values were breaking one character per line due to overflow-wrap:anywhere in a narrow column. Switch to single-line ellipsis with the full value exposed via the title attribute on hover. * feat(web): state-before/after columns + metrics chart at bottom RunDetail now renders a four-column grid: actions | state-before | state-after | side (exceptions + timeline) with MetricsChart spanning the bottom row. Each state column shows its own screenshot (step-NNNNN.png vs step-NNNNN-after.png), snapshot table, and violations panel. ActionList now receives runStartMillis and the selected Step so the active row can expand Position/Content sub-rows. * fix(web): skip zero-value ticks + add exception markers to metrics HEAP '0B' and CPU '100%' labels overlapped at the lane boundary. Drop the bottom-of-range tick on both lanes (baseline is implied) and widen LANE_GAP so the remaining labels have breathing room. Accept an exceptionStepIndices prop and draw a dashed red vertical line at each to surface exception spikes directly on the CPU/heap chart. * fix(web): let action body column shrink below its content Required minmax(0, 1fr) so the row grid honours the column's min-size of 0 instead of the implicit 'auto', preventing the action-list from overflowing its parent when the target string is long. * feat(web): bigger state screenshots + single properties row Collapse snapshots into a summary chip ('SNAPSHOTS · N violations') so the screenshot fills its state card. Deduplicate ViolationsPanel — show it once in a new full-width 'properties' row between the state cards and the timeline. Drop the right sidebar; exceptions now surface as dashed markers on the metrics chart with the ExceptionsPanel only rendering when there are actual exceptions to report. * feat(web): add minimal Tabs component Monochrome tab strip with underline-on-active. Used by state-before and state-after cards to swap between Screenshot, Snapshots, Properties. Pane scrolls internally so the outer grid stays fixed-height. * feat(web): fold timeline into MetricsChart as STEPS lane Adds a thin per-step status row above HEAP showing violated (red), pending (dim gray) or holds (green-tinted). Extends highlight + exception markers to span the status lane. Frees a whole row in the detail grid so the page can fit in 100vh. * refactor(web): tabbed state cards, drop standalone Timeline panel State-before/after now use Tabs (Screenshot / Snapshots / Properties, default Screenshot). Removes the dedicated timeline row; status lane lives on the metrics chart. Banner is gone from the shell. * feat(web): lock app shell to 100vh with no page scroll html/body/#root fill the viewport, body gets overflow:hidden, and the detail grid uses minmax(0, 1fr) rows so inner panels own their scroll. Tightens toolbar + panel padding for a denser feel. * feat(web): arrow-key nav + badges on Tabs (WAI-ARIA tablist) Roving tabindex, ArrowLeft/Right/Up/Down/Home/End navigation, explicit aria-selected/aria-controls/id wiring, and support for an optional badge inside each tab (used for violation counts). * feat(web): ViolationsPanel supports violationsOnly filter * feat(web): ActionList arrow-key nav + listbox semantics + smaller font Promote the list to role=listbox with role=option rows; roving tabindex lets ArrowUp/Down (and Home/End) seek between steps with focus. Font size dropped to 11px and padding tightened so long selector-tag labels fit in the 340px actions column. * fix(web): useKeyboardNav yields arrow keys to tablist/listbox targets Previously pressing ArrowRight on a focused tab switched tabs AND advanced the step. Skip arrow handling when the event target is inside an element with an arrow-owning ARIA role. * feat(web): fourth 'Violations' tab + wider actions + shorter metrics Adds a Violations tab to each state card showing only violated properties (with count badge on the tab label when > 0). Actions column widened from 280px to 340px, bottom metrics strip trimmed from 220px to 140px with tighter lane heights, so the whole page still fits in 100vh with no scrollbar. * feat(web): compact RunDetail layout using 1px borders instead of panel padding * refactor(inspect): simplify MetricsChart to HEAP+CPU with time axis Drop the STEPS status lane and per-sample circle markers, switch the x-axis from step indices to mm:ss clock time, trim y-axis ticks to min/max with compact units, rotate lane labels into the left gutter, and replace the thin playhead line with a wider dotted red band. Traces stay grayscale; red appears only on the playhead pattern. * fix(web): RunList rows no longer stretch to fill viewport height Tables inherited flex: 1 1 auto from .app-main > * and distributed extra vertical space across rows. Override with flex: 0 0 auto + align-self. * misc changes * fix(web): hoist useState above early return in MetricsChart Calling useState after an unconditional early return violates React's Rules of Hooks: the empty-samples branch renders 0 hooks while the populated branch calls 1. On the initial null->loaded transition of history the hook count changes and React throws. * fix(web): subscribe to named SSE event instead of 'message' Server emits 'event: runs.changed' frames; the WHATWG EventSource spec dispatches those as events of type 'runs.changed', not 'message'. The listener registered on 'message' was never fired, so RunList never auto-refreshed on run create/finish/delete. * fix(inspect): unsubscribe SSE clients on disconnect Watcher.Subscribe appended to a slice with no matching removal path, so every closed EventSource connection leaked its channel. Over a long-running server the slice grew unbounded and every fs event paid O(N) iterating dead channels. Add Unsubscribe + defer it in handleEvents. Unsubscribe does not close the channel: broadcast snapshots the slice without holding the mutex, so a concurrent close would race with its non-blocking send. * fix(trace): rename resolvedBounds/tapPoint to snake_case Every other json tag in the trace schema (from_x, duration_millis, bundle_sha256, etc.) uses snake_case. The two new Action fields introduced with the inspect UI broke that pattern. Rename them before the format ships to external consumers. * chore(web): drop vitest and remove UI tests from CI No UI tests wanted in web. Removes vitest, jsdom, testing-library devDeps and the vitest.setup.ts + vite.config.ts test block. Makefile test-web becomes web-typecheck (typecheck only). Fixes CI failure where `vitest run` exits 1 with no test files. * chore(make): dedupe sidecar embed and drop recursive make Make $(SIDECAR_JAR) the real recipe and $(SIDECAR_EMBED) a file target, so uatu/install/inspect-dev share one copy step and sidecar/release-cli just depend on the jar instead of re-invoking make.	2026-04-21 11:32:17 +07:00
pj	a2e96af1af	WIP: rename sample app to Folio (#20) ... * refactor: rename examples/sample-app to examples/folio Directory-level rename and path references in Go tests, bundle-check, top-level README, and getting-started docs. Package declarations, Gradle config, iOS bundle IDs, and class names follow in later commits. * refactor(folio): rename Kotlin package dev.uatu.sample to app.folio Moves source dirs and sqldelight schema from dev/uatu/sample to app/folio, updates package declarations and imports, and switches Android namespace/applicationId, iOS binaryOption bundleId, and sqldelight database packageName to the new identifier. * refactor(folio): rename SampleApplication to FolioApplication Android manifest now points at .FolioApplication with label 'Folio' instead of 'Uatu Sample'. * refactor(folio): set iOS bundle id and display name to Folio bundleIdPrefix + PRODUCT_BUNDLE_IDENTIFIER -> app.folio. CFBundleName + CFBundleDisplayName -> 'Folio'. * refactor(folio): update demo email to [email protected] * refactor(folio): point justfile at app.folio bundle id Updates xcrun simctl launch target, uatu test --bundle-id, and the build/uninstall comments to reference folio instead of sample. * test: update fixture package ids to app.folio Sidecar activity-resolver test and verifier spec-integration XML fixtures referenced the old dev.uatu.sample Android package. Updates them to match the folio app's real package id so the tests stay representative of what the CLI sees on-device. * test(verifier): rename SampleApp identifiers to Folio Renames TestSampleAppSpec* functions, bundleSampleAppSpec helper, and sampleAppHierarchyXML const (now loginHierarchyXML for consistency with the other per-screen fixtures). Updates trailing sample-app mentions in comments and assertion messages. * refactor(folio): rename Gradle/npm/wasm project identifiers to folio settings.gradle.kts rootProject.name, package.json + package-lock.json name, and the WasmJS index.html <title> all still read 'uatu-sample' / 'Uatu Sample'. Realigns them with the Folio brand. * docs(folio): rewrite README title + getting-started bundle id examples/folio/README.md is now titled 'Folio' with the Kotlin source paths corrected to app/folio. Getting-started example uses --bundle-id app.folio. Harness launch message is now generic ('app under test') since uatu-sample-harness is not specific to folio. * chore(folio): drop trailing 'sample' reference in gradle.properties * refactor(folio): rename LoginPage composable to LoginScreen Align with KMP/Android industry convention (NowInAndroid, Cash App, JetBrains samples use Screen, not Page). * refactor(folio): rename HomePage composable to HomeScreen * refactor(folio): rename AddAccountPage composable to AddAccountScreen * refactor(folio): rename LedgerPage composable to LedgerScreen * refactor(folio): rename AddTransactionPage composable to AddTransactionScreen * refactor(folio): split Models.kt into app.folio.data package Account, Transaction (with TxnType), and Session move into their own files under app.folio.data, matching NowInAndroid-style per-type organization. * refactor(folio): move data layer into app.folio.data package Repository, LedgerStore (expect + interface), SqlLedgerStore, WebLedgerStore, DriverFactory (expect + actuals), AndroidLedgerContext, and Snapshot move into app.folio.data. Update all consumer imports. * refactor(folio): move Navigation into app.folio.navigation package Split the former Navigation.kt into Route.kt (sealed interface) and Navigator.kt (singleton). Update consumer imports across screens, App.kt, and FolioApplication. * refactor(folio): move Platform and Format into app.folio.platform Both files carry expect declarations (Platform object, formatDate); grouping them into a dedicated platform package makes the KMP seam obvious and mirrors the structure used by JetBrains samples. * refactor(folio): move login into feature/auth package Create app.folio.feature.auth with LoginScreen + LoginUiState. Inline the former Auth.kt (DEMO_EMAIL, DEMO_PASSWORD, checkCredentials) into LoginScreen since it is the sole caller. * refactor(folio): move HomeScreen into feature/home package * refactor(folio): move account creation into feature/account package AddAccountScreen gets its own AddAccountUiState colocated with the screen, replacing the shared UiState.addAccountError. * refactor(folio): move ledger screens into feature/ledger package LedgerScreen and AddTransactionScreen move into app.folio.feature.ledger with AddTransactionUiState (txnError, txnFormType) colocated. The former catch-all UiState.kt is removed now that each screen owns its state alongside its UI. * refactor(folio): split Theme.kt; move theme and icons to subpackages Theme split into Theme.kt (tokens, layout dims, LedgerTheme) and Type.kt (typography) under app.folio.ui.theme. Icons moves to app.folio.ui.icon. Update every consumer's imports to match. * refactor(folio): split ui components into per-file under ui/component Former Widgets.kt and Components.kt become 10 focused files: AppButton, Card, EmptyState, ErrorText, FieldLabel, Header, IconButton (w/ BackButton), Screen, Segmented, TextInput. Matches NowInAndroid style of one composable per file in a designsystem/component package. * chore(folio): consolidate uatu testing files under uatu/ folder Move spec.ts, package.json, package-lock.json into examples/folio/uatu so all uatu-specific testing artifacts live in one place. runs/ and node_modules/ follow the same convention (both remain gitignored). Update justfile, README, and the two Go consumers (bundle-check tool + verifier/trace tests) that referenced the old path. * refactor(trace): drop folio path in writer test Round-trip only needs a non-empty string; neutralize to keep the library free of folio references. * refactor(sidecar): neutralize ResolveActivity test fixtures Swap app.folio for com.example.app in the fixture strings so the sidecar tests don't reference the example app by name. * refactor(bundle-check): take spec path as argument Previously the tool hardcoded examples/folio/uatu/spec.ts. Accept a positional spec path instead so the tool works for any example and leaves no folio reference in the library surface. * test(verifier): add neutral integration spec and hierarchy fixtures Adds testdata/integration_spec.ts with two routes ("list", "form"), an InputText on text_field, a Tap on primary/secondary_action, a safety property (itemCountNonNegative), and a liveness property (submitEventually). Adds hierarchies_test.go with matching XML fixtures. Constants intentionally go in a _test.go at package root rather than testdata/hierarchies.go because go skips .go files under testdata/. * refactor(verifier): replace folio integration tests with neutral ones Renames bundleFolioSpec -> bundleIntegrationSpec and the three Test* entry points to TestIntegrationSpec*. Uses the synthetic spec and hierarchies added in the previous commit so the library's test suite no longer references examples/folio at all. Folio-specific coverage remains covered by examples/folio/justfile's 'just test' which exercises the real spec on device/emulator. * chore: remove cmd/uatu-sample-harness Not referenced by Makefile, docs, CI, or any script. Duplicates the adb reverse helpers already in cmd/uatu/test_run.go, and its name implies ownership by the sample app which violates the library/ example decoupling. If a bare-protocol debugging tool is later needed it belongs inside cmd/uatu/. * docs(folio): drop Layout section and KMP layout paragraph; fix AVD override syntax The directory-tree Layout section rots faster than the code and duplicates what ls shows for free. The expect/actual paragraph in Stack was the same kind of filler. The README also claimed 'just AVD=Pixel_7 test' but the justfile reads AVD as an env var via env_var_or_default, so the correct invocation is 'AVD=Pixel_7 just test'.	2026-04-20 16:04:20 +07:00
pj	7493945251	feat: LTL operators, sampling, and default generators (#17) ... * feat(ltl): add Now/Next/Eventually/Implies/Or/And/Not formulas Replace the fold-with-latch evaluator with a residual-formula reducer. Each Observe() instantiates a fresh obligation from the root (stripping an outer Always), reduces each pending obligation against current state, latches Violated on first failure, and surfaces Pending verdicts for deferred obligations. Existing Always/Pure/Thunk tests continue to pass. * feat(ltl): support relative duration for eventually().within() * feat(proto): add Swipe, PressKey, RecentLogs RPCs * feat(verifier,runner): formula handles, new action kinds, rich state - verifier: add formula-spec registry; bindNow/bindNext/bindEventually with chainable .implies/.or/.and/.not and .within(n,unit) on eventually; bindFrom for uniform sampling. bindAlways keeps accepting plain predicates. - verifier: store lastTree, lastAction, step time, logs, exceptions on the Verifier; SnapshotInput replaces the (snapshots, tree) pair. stateObject now produces state.lastAction/time/logs/exceptions matching the TS State type. - verifier: make taps/swipes/waitOnce/pressKey built-in generators actually fire; taps picks a clickable, enabled element from the last hierarchy. - agent: add exceptions field to Message wire format. - driver: add Swipe/PressKey/RecentLogs to Driver interface; wire maestro client and mock driver. LogEntry exposed for runner consumption. - runner: apply Swipe/PressKey/Wait actions; collect logcat and exceptions; pass lastAction and step time into PushSnapshot. * feat(spec-api): LTL operators, new actions, richer State - ltl.ts exports now/next/eventually; always overload accepts a Formula - types.ts: Formula gains implies/or/and/not; EventuallyFormula adds .within; State gains lastAction/time/logs/exceptions; Swipe/PressKey/Wait action types - actions.ts: Swipe/PressKey/Wait/from constructors; waitOnce + pressKey default generators - tests exercise the chaining, sampling, and new actions through a recorded fake runtime * feat(sidecar): add swipe, pressKey, recentLogs RPC handlers * feat(sdk-android): capture uncaught exceptions Install a default uncaught handler on Uatu.start, chained with any existing handler so Android's crash reporter still runs. Expose Uatu.reportError for callers to forward caught throwables. A bounded circular buffer (default 50) drains into each STATE message's new exceptions field. Protocol.kt serializes/deserializes the field, matching the Go wire format added to internal/agent/protocol.go. * feat(spec-api): add @uatu/spec/defaults/properties bundle * feat(sample-app): exercise new LTL operators + defaults spec.ts now imports eventually/next/now/from from @uatu/spec and noUncaughtExceptions from @uatu/spec/defaults/properties. It declares three properties that exercise the new surface: - accountCountNonNegative: plain always() safety - addAccountAdvances: always(now(x).implies(next(y))) - eventuallyLoggedIn: eventually(p).within(30, "seconds") - noUncaughtExceptions: imported default The weighted actions root uses from() for random phone/name sampling and entries for taps/swipes/waitOnce/pressKey built-ins. SampleApplication gains a debug hook gated on the system property uatu.inject_error so the e2e run can synthesize an Uatu.reportError and verify noUncaughtExceptions violates. cmd/uatu/test_run.go adds a subpath alias so specs importing "@uatu/spec/defaults/properties" resolve against the in-tree source when running from the uatu checkout. The spec-integration tests swap the old click-counter fixtures for the new login hierarchy. * feat(trace): record swipe/key/wait details + exceptions trace.Step gains an Exceptions array so the trace captures the class/message/stackTrace for each SDK-reported throwable in a step. trace.Action gains FromX/FromY/ToX/ToY/Key/DurationMillis so the full payload of Swipe/PressKey/Wait actions is visible in trace.jsonl. sample-app's debug error hook now gates on ApplicationInfo.DEBUGGABLE instead of a system property (adb setprop fails on non-rooted emulators).	2026-04-20 02:19:39 +07:00
pj	0c775c199d	feat(examples): add sample-app spec ... Introduce examples/sample-app/spec.ts — a minimal property-based spec that taps the sample app's "Click me" button and asserts click_count is monotonic. Bundle-check and the trace writer test now reference the new path.	2026-04-18 10:33:35 +07:00
pj	72c4e4c4d3	feat(trace): JSONL writer for steps + meta + screenshots ... Each WriteStep appends one JSON object per line so the trace can be filtered with jq directly. Screenshots land under screenshots/ with zero-padded indexes. Writer is concurrency-safe; Close is idempotent and a write after Close errors loudly rather than silently dropping.	2026-04-17 23:43:45 +07:00