4 Commits

Include renames

Author	SHA1	Message	Date
pj	6b0d6cb971	WIP: Drive physical Android devices over USB (#67) ... * feat(sidecar): reach USB devices via the adb server by serial * feat(test): add --device flag to target a specific Android device by serial * feat(folio): select Android device via ANDROID_DEVICE in justfile * feat(conformance): add android backend to the gate suite * feat(android): keep device awake and unlocked so the app stays foreground * feat(conformance): prep physical android device (autofill/verifier/stayon) * fix(android): make device prep best-effort so OEM-blocked commands don't abort the run * fix(verifier): require positive bounds for swipe candidates A zero-bounds element centers at (0,0); a downward swipe from the top-left corner is the system gesture that pulls down the notification shade, dragging the fuzzer out of the app. Swipes now require positive bounds like every other verb. * fix(runner): harden app-scope guard against launcher and overlays The per-step guard now relaunches and waits until the app window is actually drawn before proceeding, so a slow physical-device relaunch no longer lets an observe or action land on the launcher. It also detects a system overlay (notification shade) stealing window focus while the app stays resumed, and dismisses it with back. * feat(android): harden physical-device runs in device prep Device prep now disables the AOSP cached-app freezer, phantom-process killer, and Doze (and exempts the driver) so OEM background management stops suspending the driver mid-run. Adds ReinstallApp for clear-state on ROMs that deny pm clear, and teaches focus detection to report the notification shade as systemui so the scope guard can dismiss it. * feat(driver): clear-state via APK reinstall when pm clear is blocked When an APK path is set, Android clear-state resets the app by uninstalling and reinstalling instead of asking the sidecar to pm clear, which hardened OEM builds (ColorOS) deny even to the adb shell user. Falls back to the sidecar clear path when no APK path is provided. * feat(cli): add --android-app-path for clear-state reinstall Wires the APK path from the test command through to the sidecar client so Android clear-state can reset apps on OEM builds that deny pm clear. * chore(folio): pass --android-app-path in just test * fix(runner): clamp swipe/scroll origin out of edge gesture zones A gesture starting in the top status-bar strip pulls down the notification shade; the bottom and side strips are the home and back gestures. Any of them drags the fuzzer out of the app. Swipe and scroll origins are now clamped into a safe inner area sized from the maximum element extent (the Android hierarchy root reports zero bounds, so the extent is the reliable screen size). Calibrated on device: origins below ~7% of height no longer open the shade. * perf(sidecar): faster Android text input and drop redundant settle poll inputText now uses adb `input text` for short shell-safe ASCII (~5x faster than the driver's per-character path) and falls back to the driver for unicode, injection payloads, and overflow-length strings. waitForIdle drops the structural-hash poll that followed waitForAppToSettle: each hierarchy fetch is ~500ms on a physical device, so it cost ~2.8s per mutating step for marginal benefit, and the runner already re-fetches transitional frames. Cuts p95 step latency from ~6.5s to ~5.1s; G1-G4 still pass. * fix(verifier): exclude soft-keyboard region from action candidates The fuzzer was tapping Gboard's "Settings" key, navigating out of the app. That key is a bare FrameLayout with a content-desc and no package or resource-id, so the package-based scope filter missed it. Candidates whose center falls in the keyboard region (derived from the IME elements' bounds) are now dropped, so no tap or long-press lands on a key. Opt-in with app scoping; unscoped runs keep every node. * perf(runner): replace focus-tap settle with a brief wait The full WaitForIdle after a field-focus tap cost ~0.5-1s per InputText step on a physical device while the keyboard animated in. The tap registers focus immediately and text is injected into the focused view, so a short fixed wait suffices. Drops p95 step latency ~5.1s to ~4.0s; G1-G4 stay green. * chore(conformance): platform-aware G5 p95 budget for android The 2500ms ceiling was calibrated on the iOS simulator. A physical Android device drives every step over USB (snapshot + settle + adb round-trips), so its per-step floor is several times higher; holding it to 2500ms would force removing the settle/retry logic the correctness gates depend on. The android backend now defaults to 4500ms (override with P95_LIMIT_MS); iOS stays 2500. * fix(sidecar): retry maestro android driver startup The maestro Android driver's dadb.open() occasionally misses its startup deadline (its instrumentation host is slow to come up right after a reboot or per-run reinstall), which aborted the whole run. Retry the open a few times with a short backoff so a transient timeout recovers. * chore(conformance): widen android G5 budget to 5500ms Physical-device p95 swung 3209-4612ms across sessions (cold runs right after a reboot are slower). 4500ms was too tight for that jitter; 5500ms covers the observed ceiling with headroom. * web replay fix * feat(android): force 3-button nav during runs to prevent app drift On gesture navigation a fuzzer swipe can trigger swipe-up-home or edge-back and fling the app off screen. Device-prep now switches to 3-button navigation for the run (no edge gestures; the nav bar's buttons are systemui-owned and already excluded from action candidates) and restores the original navigation mode when the run ends. Best effort: leaves nav untouched if the overlay command is unavailable. * fix(android): target the selected device in adb reads; don't strand nav mode Review fixes: - ForegroundPackage/FocusedWindowPackage now take a serial and pass -s, so the foreground/scope guard works when several devices are attached (the --device path). Previously they ran bare `adb shell`, which errors with multiple devices, silently disabling app-scope enforcement. The sidecar client passes its serial through. - Extract an adbArgs helper and route every adb call through it, removing four duplicated serial-arg builders. - ForceThreeButtonNav now decides what to restore before changing anything: if the current mode is unknown or already 3-button it leaves nav untouched, instead of switching and then stranding the device in 3-button. Logic split into the pure navModeToRestore, now unit tested. * fix(runner): restore scrollBounds doc; cover destination clamp and screenBounds Review fixes: move the scrollBounds doc comment back onto scrollBounds (it was stranded above screenBounds by an insertion). Extend the clamp test to assert an off-screen destination is clamped onto the screen and that the origin lands exactly on the margin. * test(verifier): cover keyboardRegionTop, including the decor-view guard The full-screen IME decor view rejection had no test; removing it left the suite green. Add direct cases: no keyboard -> sentinel, decor view ignored in favor of the real keyboard line, and decor-only -> sentinel. * style(cli): gofmt testOptions field alignment * fix(sidecar): keep a leading dash off the fast input path A value starting with '-' could be read as an option by `adb input text`, so the fast-path regex now requires a non-dash first character; such values fall back to the driver. Also cover the dadb-target branch where a colon precedes a non-numeric port (a USB serial, not host:port). * refactor(verifier): scope action candidates by window ownership Replaces the leaky per-element package check and the keyboard-region Y heuristic with one rule: walk the window tree propagating each node's owning package (empty and the neutral android framework package are transparent); a node is in scope only when no concrete foreign package owns it (the app's own window carries no package on Compose apps) or the owner is the app package. This drops whole foreign windows (soft keyboard, system UI, launcher) AND their empty-package child wrappers -- e.g. a keyboard's 'Settings' key, which the old empty-package-is-in-scope rule admitted and which navigated out of the app. Deletes keyboardRegionTop/isInputMethodElement. * fix(runner): re-check foreground at apply time, skip stale actions ensureForeground runs before observe, but the app can leave between observe and apply (a prior gesture settling late); swipes/keys then fire stale coordinates onto whatever screen is now up. Re-check foreground immediately before applying and, when the app is gone, skip the action and log it (making the escape visible) so the next step's guard relaunches instead. * fix(android): type long ASCII via fast guarded path to stop keystroke escape A 4096-char corpus string exceeded the fast input cap and fell to the per-character driver path, which takes ~120s. During that uninterruptible window focus could leave the app and the remaining keystrokes sprayed into the launcher search box. Route shell-safe ASCII of any length through adb input text, chunked, re-checking the foreground app between chunks and stopping if it changed. * chore: ignore gate artifacts and local scratch files * refactor(runner): narrow gesture clamp to the top shade strip 3-button nav (forced for every run) disables the side back and bottom home gestures at the OS level. On-device probing confirmed side and bottom swipe origins no longer drift, leaving the notification shade as the only edge gesture a swipe can trigger. Clamp only the top strip; keep origin and destination on screen otherwise. * chore(format): add .editorconfig enforcing 80-column limit * chore(format): add prettier config with 80-char printWidth * chore(deps): add prettier devDependency to replay-ui * chore(deps): add prettier devDependency to folio-web * chore(deps): add prettier devDependency to spec package * chore(format): add swift-format config with 80-char lineLength * feat(format): add make fmt targets for per-language 80-col formatting * fix(runner): translate gesture to safe area so near-top scrolls keep direction Clamping the swipe origin to the top margin while leaving the destination on the full screen used two reference frames: a scrollable container pinned in the top strip had its origin pushed past the destination, reversing the gesture. Translate the whole from->to segment down by the same delta so the origin clears the shade strip without flipping direction. Adds a scroll-near-top test that fails under the old origin-only clamp. * fix(runner): apply-time guard consults focused window, not just resumed activity ensureForeground detects a system overlay (notification shade) owning the focused window while the app stays the resumed activity, but appIsForeground only queried ForegroundApp. A swipe that pulls the shade over the app between observe and apply then fired onto the shade. Mirror the focus check at apply time so the action skips and the next step dismisses the overlay. * test(runner): cover apply-time foreground skip and appIsForeground table Adds a Run-level test asserting no tap reaches the driver while a system overlay holds focus (guards against the skip branch being dead-coded), plus a decision-table test for appIsForeground. Adds ForegroundErr/FocusedWindowErr to the mock driver so the guard's transient-read paths are exercised. * fix(sidecar): harden android driver open, input guard, pressKey, foreground marker - openWithRetry rebuilt a closed AndroidDriver, whose gRPC channel is final and shut down by close(); the retry then ran against a dead channel. Build a fresh driver per attempt and extract a unit-tested retryOpen helper (named DRIVER_OPEN_ATTEMPTS/BACKOFF). - pressKey on the Maestro backend did KEY_MAP[key] (no lowercase, no throw), silently dropping unknown or wrong-case keys; route through a pure maestroKeyFor that lowercases and rejects unknown keys like the Stub contract. - the mid-type foreground guard (typeShellSafe) was untested; extract a pure typeChunks and cover stop-on-foreground-change, always-send-first-chunk, and unknown-owner. - foreground detection required the literal topResumedActivity=ActivityRecord; align parseResumedPackage to the same *ResumedActivity marker set Go reads so OEM wording does not disable the guard. * fix(conformance): pin self-test p95 budget and score install failures as run failures self_test reused the backend-dependent P95_LIMIT_MS, so under BACKEND=android the 4000ms slow fixture rated PASS and the offline analyzer check failed from an env var; pin it to 2500. A per-run adb install failure ran unguarded under set -e and aborted the whole harness; guard it, record the run as a G1 failure, and continue. * fix(android): require --device when several devices are connected With no serial requested and more than one device online, pickDevice silently returned connected[0], but that serial is never threaded into the per-step adb calls, so every later bare adb command failed with "more than one device". Error instead and ask for --device, mirroring pickAVD; a single device stays unambiguous. * refactor(android): move PrepareDevice doc onto it; extract tested wakeCommands The PrepareDevice doc block was stranded above adbArgs, leaving the exported function undocumented under godoc. Move it back and split the wake/keyguard tuples into wakeCommands so they have a unit test. * perf(verifier): memoize scopedElements per tree scopedElements rebuilt a full tree walk plus map on every candidatesForVerb call (~16 per step). Cache the result keyed on lastTree and invalidate it in PushSnapshot. * fix(sidecar): default reinstallApp in SetClearStateReinstall; cover non-android clear Only Dial set reinstallApp, so a Client built another way would nil-deref on Android clear-state. Default it in SetClearStateReinstall too. Add a non-android test so the platform guard has negative coverage: dropping the android check would now fail. * test(runner): make focusTapSettle injectable so apply tests don't sleep 250ms The focus-tap settle was a const, so five InputText apply tests each blocked the full 250ms. Make it a package var and shorten it per-test with cleanup. * refactor(runner,android): drop unused bringToForeground return; grep no-match yields empty bringToForeground's bool return was read by no caller. FocusedWindowPackage's on-device grep exited 1 on no match, surfacing as an error instead of the documented ""; add || true. * perf(sidecar): reuse a single Jackson ObjectMapper structuralHash, countRouteScreens, and hierarchy each built a fresh ObjectMapper per call inside the stability poll; the instance is thread-safe and meant to be reused. Hoist one shared val. * refactor(android): remove unused AdbReverse/AdbReverseRemove No callers anywhere in the tree; they were also the only adb calls bypassing adbArgs. Dead code, removed. * style(runner): trim non-load-bearing comments from this PR's runner code and tests * style(sidecar): trim non-load-bearing comments from this PR's driver code and tests	2026-06-11 10:10:05 +05:30
pj	c5bb176be8	UX refactor (#52) ... * feat(ltl): bound fields on AlwaysFormula and named thunks Add StepBound/Duration/Deadline to AlwaysFormula as the dual of bounded Eventually, give ThunkFormula a Name for stable identity, add ThunkNamed, and surface both in describe() and MarshalJSON. * feat(ltl): negation normal form pass nnf/pushNot rewrite a formula so every Not wraps only a Thunk or Error leaf, dualizing Always<->Eventually and preserving bounds. * feat(ltl): NNF in NewEvaluator, bounded-always, Finalize, collapse Apply nnf on construction, reduce bounded Always symmetric to bounded Eventually (vacuous holds once the window closes), add Finalize to resolve undischarged liveness obligations to Violated at run end, and collapse structurally-identical pending obligations. * test(ltl): property-based NNF laws Lock double-negation identity, Always/Eventually duality with bound preservation, leaf pushdown, and not(always true) reaching Violated. * test(ltl): Finalize, bounded eventually, latch, collapse Property tests for monotonic violation latch and eventually-within violating iff n consecutive false, plus Finalize and collapse cases. * feat(inspect): within clause on always residual node A negated bounded eventually serializes as a bounded always; render its bound instead of dropping it. * feat(ltl): witness violations and (bool,error) predicate thunks * test(ltl): migrate thunk call sites to (bool,error) * feat(ltl): flag thrown-predicate witnesses with IsError * refactor(verifier): replace predicate err side-channel with violation witness * test(verifier): witness API for thrown predicates * feat(trace): witnesses map and skipped-verification marker on Step * feat(runner): thread violation witnesses, finalize, skip marker into trace * test(ltl): lock violation witness reason, IsError, and step * test(verifier): finalize surfaces unmet eventually with witness * fix(ltl): eliminate implies and bounded-always false-negatives Rewrite a -> b to (not a) or b in NNF so a pending temporal antecedent can no longer defer the whole implication and drop a consequent that was false at the current step. Carry a pending inner past a bounded-Always window close instead of dropping it to holds, so a deferred obligation is resolved by a later step or Finalize. * test(ltl): lock implies and bounded-always false-negative regressions * fix(web-runtime): seed PRNG for reproducible runs and align weighted pick * feat(testrun): inject seed into web bundle via SANDERLING_SEED define * test: cover web-runtime seeded PRNG, weighted pick, and seed define wiring * test(spec): add Go math/rand/v2 PCG oracle and golden fixture * feat(spec): bit-exact PCG port of Go math/rand/v2 * test(spec): assert pcg.ts matches the PCG golden fixture * feat(spec): shared input corpus and press-key pools * feat(spec): action-tree types and Host interface * feat(spec): verb support matrix and warn-once helper * feat(spec): deterministic shared action picker * test(spec): verb matrix and warn-once semantics * test(spec): picker draw-order and determinism * refactor(spec): actions.ts returns pure GeneratorNode data trees * refactor(spec): wire from() sampling through the picker rng * feat(spec): shared runtime-entry installs next-action over pick.ts * feat(spec): export LongPress/Scroll/longPresses/scrolls factories * test(spec): assert data-tree shapes for action factories * test(spec): runtime-entry serializeAction wire-contract round-trip * refactor(spec): bridge data-tree nodes to the legacy goja picker tags * fix(spec): web runtime walks the spec's globalThis.actions data tree * test(spec): tolerate legacy bridge fields on builtin nodes * refactor(spec): installRuntime accepts a lazy root resolver The web bundle imports the runtime before the spec, so the action root on globalThis.actions only exists after the spec evaluates. Accept a function form so the goja and web hosts resolve the root per tick. * refactor(spec): web-runtime becomes the WEB Host, delegates to shared picker Delete the duplicate picker (resolveGenerator/pickWeighted/randomTap/ randomInput/randomSwipe/randomPressKey/pickFromArray, the mulberry32 PRNG, and the snake_case serializeAction) plus the __sanderling__ action factory binds. web-runtime now implements Host (platform/seedHi/seedLo from the injected 64-bit seed via BigInt, queryCandidates over the live DOM with a per-tick cache, reportUnsupported) and calls installRuntime so both engines run pick.ts over the same Pcg. Swipe/longPress/scroll follow the verbs.ts matrix instead of silently returning null. Keeps the DOM helpers (selector translation, queryElement, elementHandle, buildState, sanitize, extractors) and the global locking. Net -214 lines (741 -> 527). * test(spec): cover the WEB Host surface and seed precision Replace the deleted-picker tests with Host coverage: platform()==web, seedHi() parsing a 64-bit seed without Number precision loss, seedLo()==0, reportUnsupported warning, the installed next-action/extractor globals, and queryCandidates verb routing + per-tick caching over a querySelectorAll stub. * refactor(spec): picker emits native selector + scroll endpoints, setup precedence * feat(spec): goja runtime entry wires the shared picker over the Go host * feat(bundler): optional RuntimeFile prepends a runtime-entry import via stdin * feat(testrun): bundle the goja runtime entry so the verifier runs the shared picker * refactor(spec): drop the legacy goja bridge fields from action factories * feat(spec): serialize selector-only string targets for the runner to re-resolve * refactor(verifier): one DecodeAction reads the unified flat wire contract * refactor(verifier): goja host + shared picker replace the duplicate Go picker * refactor(runner): decode V8 actions via the unified DecodeAction; wire goja runtime * test(verifier): author specs through the shared picker path * test(runner): bundle authored specs with the goja runtime entry * feat(verifier): collect unsupported verbs for the run report * refactor(runner): collapse WebDriver forks behind ActionSource/ExtractorSource * feat(testrun): surface unsupported verbs in run report * test(verifier): cross-runtime goja/node parity gate on the shared picker * test(verifier): unsupported verbs collected deduped in first-seen order * test(runner): summary reports no unsupported verbs on a clean run * test(spec): golden-fixture cross-runtime parity gate for the node picker Replace the env-driven parity harness with a shared scenario module and a committed golden the node picker asserts independently. The goja side asserts the same golden, so neither runtime invokes the other at test time. * test(verifier): assert goja picker against the same cross-runtime golden Drop the node-subprocess coupling: the goja side now installs a stub __sanderlingHost__ with the fixed candidate list and asserts the committed golden, matching pkg/spec/test/parity.test.ts. * refactor(spec): rename pressKey generator export to pressKeys * refactor(spec): update barrel re-exports for pressKeys * test(spec): update pressKeys generator export name * docs(spec): rename pressKey generator to pressKeys * refactor(spec): extract samplerRng into shared sampler-rng module * feat(spec): add fluent seeded value generators (strings/integers/emails/edgeCaseText) * test(spec): cover fluent value generators determinism and chaining * refactor(bundler): inject globalThis trailer from spec named exports * refactor(bundler): reuse registration trailer in web bundler * test(bundler): cover named-export globalThis registration * feat(spec): add named() to Extracted handle type * feat(web-runtime): named() and cross-extractor read guard * feat(verifier): named() and cross-extractor read guard in goja * test(verifier): cross-extractor read guard and named() * test(web-runtime): export runtime and extractors for tests * test(web-runtime): named() and cross-extractor read guard * refactor(folio): drop manual globalThis trailer (bundler injects it) * refactor(folio): seed txn amounts via integers().between(1,500) * refactor(folio-web): drop manual globalThis trailer (bundler injects it) * fix(folio-web): seed card/txn-type selection via from().generate() for reproducible runs * refactor(folio-web): weight valid generators against edgeCaseText for names/amounts * refactor(folio-web): name extractors so violation witnesses are readable * fix(web-runtime): propagate extractor getter throws and unpoison locked global Stop swallowing getter errors in evaluateExtractors so the cross-extractor read guard aborts loudly, matching goja's PushSnapshot. Make the __sanderling__ lock configurable (still non-writable) so a shared test process can reinstall a fake. * test(spec): install fake runtime via defineProperty to survive locked global * test(web-runtime): assert uncaught cross-extractor read aborts evaluateExtractors * feat(runner): add MaxSteps bound to Options * test(runner): MaxSteps stops after exactly N steps * test(driverpb): drop proto getter round-trip tautology * test(sidecar): drop stub-mode placeholder tautology tests * test(mock): drop default-field-value assertion test * test(ltl): drop Verdict.String tautology tests * refactor(runner): extract RenderSummary for snapshot testing * test(runner): golden snapshots for trace stream and violation summary * feat(web-runtime): capture uncaught errors into state.exceptions * test(integration): add throwing and counter web fixtures * test(integration): add specs for the web fixtures * test(integration): drive web fixtures through the real pipeline in headless Chrome * chore(make): add test-browser target for the Chrome-driven suite * ci: run the Chrome-driven browser suite in a separate job * refactor(test): relocate browser suite to test/browser * refactor(permissions): delete dead internal/permissions package * refactor(test): rename package to browser_test * refactor(sidecarassets): rename internal/sidecar to internal/sidecarassets * chore(make): point test-browser at test/browser * docs(decisions): record internal/permissions deletion * refactor(doctor): use sidecarassets package * refactor(testrun): use sidecarassets package * fix(test): resolve testdata relative to browser_test.go * refactor(verifier): remove dead __sanderlingIndex compat alias * refactor(bundler): use encoding/json for JS string literals * docs(action-space): use vendor-neutral native driver wording * refactor(hierarchy): scrub backend tool name from comments * refactor(driver): scrub backend tool name from comments * refactor(driver): add DoubleTap and DoubleTapSelector to DeviceDriver * refactor(sidecar): implement DoubleTap with the sub-100ms inter-tap gap * refactor(chrome): implement DoubleTap as two taps with the gap * refactor(mock): record DoubleTap and DoubleTapSelector actions * refactor(runner): delegate double-tap to driver, drop gesture timing * test(runner): assert double-tap delegates to driver DoubleTap * docs(cmd): add package docs to CLI and developer tools * docs(driver): add package docs to driver interface and chrome backend * docs(driver): add package docs to mock and sidecar backends * docs(platform): add package docs to android and ios device prep * docs: add package docs to bundler and inspect * docs(ltl): add package doc to temporal logic evaluator * docs: add package docs to runner and testrun pipeline * docs: add package docs to trace and verifier * docs(sidecarassets): add package doc for embedded JAR loader * fix(chrome): add disable-dev-shm-usage so Chrome starts in CI * test(chrome): gate real-Chrome driver tests behind the browser tag * chore(make): run chrome driver tests in the browser job * fix(web-runtime): guard global error listeners for non-browser hosts The module registered window error/unhandledrejection listeners at top level, which threw under Node (the spec-api test runner) where globalThis.addEventListener is absent. Register only when the API exists; the real browser run is unaffected. * ci(browser): re-enable unprivileged user namespaces for headless Chrome ubuntu-latest moved to 24.04, whose AppArmor restriction on unprivileged user namespaces stops headless Chrome from opening its DevTools socket even with --no-sandbox, surfacing as the driver's 'websocket url timeout'. Relax the sysctl for the job and add a direct launch check so a future breakage shows Chrome's own stderr rather than an opaque driver timeout. * ci(browser): pin stable Chrome for the driver tests setup-chrome's default latest pulled a dev Chromium (150) whose remote debugging socket never came up under chromedp, while plain --dump-dom worked. Pin the stable channel, which the driver is tested against. * feat(defaults): add scroll and rebalance action weights Use relative-integer weights (taps/typing co-primary 100, scrolls 50, swipes 25, doubleTaps 10); the picker normalizes by their total. Adds scrolls to defaultActions as a first-class reveal behavior. * feat(defaults): trim scroll action weight wiring * fix(build): point sidecar jar ignore and embed paths at sidecarassets * test(defaults): drop stale longPresses re-export assertion longPresses is opt-in vocabulary, no longer re-exported from defaults/actions.ts since e0d3b20; its builtin resolution is already covered by api.test.ts. Trim the defaults test to scrolls, which is an actual default export. * fix(chrome): raise DevTools websocket read timeout to 60s Chrome cold-start on a loaded CI runner can exceed chromedp's 20s default for reading the DevTools websocket URL, flaking the browser tests with "websocket url timeout reached". Give launch more headroom.	2026-06-02 09:52:53 +05:30
pj	88db9653e5	refactoring default action layer (#51) ... * feat(hierarchy): add editable signal with native derivation * feat(chrome): emit editable flag in hierarchy dump * feat(verifier): expose editable on ax element objects * feat(spec): add editable to selector and element types * feat(verifier): register typing builtin generator * feat(verifier): typing builtin types edge-case corpus into editable fields * feat(spec): export typing builtin generator * feat(spec): add defaultActions bundle * feat(spec): export @sanderling/spec/defaults subpath * feat(folio): layer defaultActions breadth over targeted flows * test(verifier): typing builtin targets editable fields, declines otherwise * test(hierarchy): editable derivation and selector matching * test(spec): defaultActions, typing, and defaults barrel resolve * fix(testrun): alias @sanderling/spec/defaults for the bundler * test(chrome): editable flag for inputs, textarea, contenteditable * feat(spec): typing builtin for the web (V8) action path * chore(folio): auto-boot a bootable AVD in just test/install when none connected * feat(driver): add ForegroundChecker optional capability * feat(android): detect foreground package via adb dumpsys * feat(sidecar): implement ForegroundApp via adb for android * feat(runner): relaunch app when foreground escapes during exploration * fix(spec): drop hardware back from defaultActions to stay in-app * feat(spec): add DoubleTap action type and constructor * feat(spec): wire DoubleTap through web-runtime serializer * feat(verifier): bind doubleTap and decode DoubleTap actions * feat(runner): dispatch DoubleTap as two taps inside one step * test(doubleTap): cover constructor, verifier round-trip, and runner dispatch * feat(folio): add noDuplicateTxnPerStep invariant and doubleSubmitTxn action * fix(folio): track ledger row count across non-ledger steps; pin reproducer seed * feat(spec): add doubleTaps random-target builtin to defaultActions * feat(verifier): add doubleTaps random-target generator * refactor(folio): drop doubleSubmitTxn; fuzzer surfaces double-submit via defaultActions * fix(folio): make ledgerRowsSeen monotonic to suppress transient-render false positives * feat(verifier): track newly-violated property set per step Sticky `always(P)` violations re-surfaced on every step after onset, flooding traces and summaries with duplicate records. EvaluateProperties now diffs against the prior verdict map and records the onset set; a new NewlyViolatedProperties accessor exposes it so callers can emit each violation exactly once at its onset step. The verdict-map return is preserved for residual / current-verdict consumers. * refactor(runner): emit onset-only violations to trace and summary Switch the per-step violation list from the sticky verdict map to the verifier's onset set. Each property now appears exactly once across a run: at the step it first violates, not on every subsequent step where the residual stays false. Removes the dead violationNames helper. * style(verifier): use maps.Copy for verdict snapshot * fix(folio): make login spec content-driven (idempotent across re-entries) * fix(verifier): canonicalize selector strings Object/chain JS selectors used to fall through to goja's default stringification, producing "[object Object]" tags that surfaced as garbage in trace.action.selector. Emit canonical "k:v" / " > "-joined strings instead so the tag round-trips back through the hierarchy selector grammar. * refactor(folio): replace txn invariants with balanceMatchesAddedTxn Collapse noDuplicateTxnPerStep and newTxnChangesBalance into a single per-row property: every newly-appearing ledger row's signed amount must match the ledger balance delta. A double-submit lands two rows whose individual amounts cannot both equal the aggregate delta, so each row fires the property, catching both the row-count and balance-math classes of bug under one semantic invariant. * refactor(trace): drop WriteScreenshotAfter Only one screenshot per step is captured now (concurrently with hierarchy after settle), so the -after.png variant is unused. * refactor(runner): one concurrent screenshot per step Move screenshot capture into the post-action errgroup so it observes the same UI moment as the hierarchy fetch. Drop the pre-action and deferred -after captures. Skip WaitForIdle when the action is Wait since the wait itself provides settling time. * refactor(inspect-ui): use next step's screenshot for state after Each step now has one screenshot (the moment of observation). The "state after" view of step N is the same moment as step (N+1)'s observation, so reuse that file rather than expecting a separate -after.png. * feat(sidecar): structural-hash settle poll Add pollUntilStable and structuralHash helpers; wire them into the Stub, Maestro, and iOS backends' waitForIdle. The structural hash ignores bounds-only flicker (measure passes) but trips on any change in resource-id/class/content-desc/text, so a Compose cross-fade where both source and destination composables are momentarily alive no longer slips through Maestro's waitForAppToSettle and contaminates the next hierarchy fetch. * test(sidecar): cover pollUntilStable and structuralHash Verify the poll returns on two equal snapshots, after transient churn, and at the cap when never stable; assert the hash ignores bounds-only flicker and detects content changes. * feat(spec): accept optional name on extract() Add an (name, getter) overload so each extractor handle carries a debuggable label that future trace fields (per-step diffs) can key off. The web-runtime falls back to extractor_\${index} when none is supplied so existing call sites keep working unchanged. * test(spec): cover extract name overload Verify the runtime receives an undefined name in the legacy shape, the supplied name in the (name, getter) shape, and that extract("name") with no getter throws. * feat(verifier): name extractors for diff surfacing bindExtract accepts an optional name argument; falls back to extractor_N when omitted. The name is stored on extractorState alongside prev/curr value caches that the next change will use to emit per-step diffs. * chore(folio): name every extract() call Give each extractor in the Folio spec a debuggable label so the inspect UI can render extractor-value diffs at violation steps keyed by intent (ledgerRows, route, ledgerBalance, ...) rather than by registration index. * feat(verifier): track extractor value transitions Cache each extractor's prior and current JSON-encoded value during PushSnapshot; expose ChangedExtractors to surface per-step diffs the runner can emit into the trace. The first observation flushes every non-null extractor as a change so the inspect UI shows initial state breadcrumbs alongside later transitions. * test(verifier): cover ChangedExtractors diffs Verify initial snapshot reports both named and fallback-named extractors, a subsequent change surfaces prev/curr, and a no-op snapshot leaves the diff empty. * feat(trace): emit extractor_changes per step Add ExtractorChanges to trace.Step and a runner helper that converts the verifier's diff map into the trace shape. The inspect UI keys its violation breadcrumbs off this field. * feat(inspect-ui): render extractor-change breadcrumbs at violations Show prev -> curr for each extractor whose value changed on the selected step, anchored under the violation row in ActionList. Long values collapse into <details> so the inline diff stays readable while the full payload is one click away. * fix(sidecar): cap stability poll independently of settle budget The previous shape halved durationMillis between waitForAppToSettle and the structural poll, then hammered hierarchy() at 80ms intervals - on Maestro this stacked enough RPCs that hierarchy fetches began timing out under load and the run stalled. Pass the full budget to waitForAppToSettle and cap the follow-up structural poll at 600ms with a 120ms interval, so the device sees at most a handful of extra hierarchy reads per step. * feat(cli): default --clear-data on so runs start fresh * feat(sidecar): streak-based settle with route-transition detection Two changes layered into the stability poll: 1. stabilitySnapshot returns null while the tree carries more than one route-level Screen tag (resource-id / testTag / identifier ending in "Screen"), so the poll cannot declare a NavHost cross-fade stable. Apps following the Compose route convention get this detection for free; apps that don't fall through to the generic signal below. 2. pollUntilStable now requires an uninterrupted stable streak of at least MIN_STABLE_STREAK_MILLIS rather than just N consecutive matches. A late transition that fires after a brief calm window breaks the streak instead of slipping past. Interval widened to 250ms so UiAutomation isn't hammered under fuzz load. * test(sidecar): cover streak reset and route-transition rejection Verify the poll honors MIN_STABLE_STREAK_MILLIS, that a transient mid-stream change resets the streak, that null returns block streak progress through a NavHost cross-fade, and that stabilitySnapshot counts only route-level attribute keys when summing Screen tags. * feat(runner): re-fetch on transitional hierarchy capture Some actions trigger async work (DB write, ViewModel coroutine) whose navigation transition begins after the sidecar settle poll has already exited. Without intervention, the next iteration's hierarchy fetch lands mid cross-fade and the verifier observes a partial extractor state which then surfaces as a false-positive violation at the step where the transition completes. fetchSyncedState pairs hierarchy + screenshot in one goroutine and retries the pair (up to 4 times, 200ms apart) while the captured tree contains more than one route-level *Screen tag. Steps that observe no transition get no added cost; steps that catch a transition pay up to ~600ms extra wall time but record a tree that matches the post-transition state the property language expects to compare. * feat(runner): gate first action on app reaching foreground * test(runner): cover startup foreground gate and back-press * feat(verifier): scope random-action targets to app package Random tap/doubleTap/type/swipe candidates now exclude nodes whose package differs from the app under test, so exploration never fuzzes the soft keyboard, system UI, or permission dialogs. An unset app package or an element with no package stays in scope, preserving behavior on iOS. * feat(testrun): pass app package into verifier scope filter * test(verifier): cover package-scoped target selection * feat(hierarchy): derive package from resource-id prefix The Android sidecar omits an explicit package attribute, so the verifier's package scope filter was a no-op and the keyboard still leaked into targets. Native nodes carry their package as the resource-id prefix; derive it there when the attribute is absent. Compose testTags are colon-less and stay empty, keeping them in scope. * test(hierarchy): cover package derivation from resource-id * chore: stop tracking inspect-ui/dist build artifacts * feat(android): detect focused-window package via dumpsys window * feat(driver): add FocusedWindowChecker capability * fix(runner): gate first observe on the app window being drawn, not just resumed * test(mock): add FocusedWindowApp with foreground mirroring * test(runner): cover startup gate waiting for app window to draw * feat(proto): add Snapshot RPC for atomic hierarchy+screenshot Pairs hierarchy and screenshot in a single response so the runner can capture both under a backend mutex, avoiding the cross-fade race where the two reads describe different frames. * feat(sidecar): add snapshot default on DriverBackend Default impl calls hierarchy() then screenshot(). The service layer wraps the call in a mutex so concurrent runners observe a serialized pair. * feat(sidecar): wire Snapshot handler with serialization lock Synchronizes backend.snapshot() so concurrent runners observe a serialized hierarchy+screenshot pair, eliminating the cross-fade race where two parallel reads describe different frames. * test(sidecar): cover Snapshot wire path and serialization lock SnapshotHandlerTest asserts both fields are populated, concurrent calls are serialized, and the default impl runs hierarchy then screenshot. * feat(driver): expose Snapshot on DeviceDriver and sidecar client Snapshot wraps the new atomic-snapshot gRPC: the runner gets hierarchy and screenshot from one round-trip whose two reads are serialized on the sidecar side. * feat(driver): add Snapshot to chrome and mock drivers The chrome tab is single-threaded so its Snapshot pairs the two reads without extra locking. The mock records ActionSnapshot so tests can assert the runner reaches for the paired RPC. * refactor(runner): observe each step via the atomic Snapshot RPC fetchSyncedState now issues one Snapshot per attempt so hierarchy and screenshot describe the same on-device frame. The transitional retry stays: that case handles a fully-captured but mid cross-fade frame, which atomic capture cannot fix. * test(runner): assert step uses Snapshot, not raw hierarchy/screenshot TestRunner_UsesAtomicSnapshot catches regressions to the two-goroutine race, and the existing parallel-fetch test now keys off ActionSnapshot. * test(driver): cover Snapshot in proto descriptor and sidecar client Adds Snapshot to the descriptor allowlist and a sidecar-client test that asserts both fields come back over the wire. * feat(trace): add Transitional flag to Step * fix(runner): skip verifier for transitional trees after retry budget When fetchSyncedState exits its retry loop with a tree that still shows a NavHost cross-fade, the runner now marks the step transitional, writes the step + screenshot to the trace, and skips Verifier.PushSnapshot / EvaluateProperties / ChangedExtractors so the previous-to-current extractor advance is not poisoned by transient state. The next clean step's previous still references the prior clean state. NextAction continues to run so the loop never deadlocks on a never-stabilizing screen. * test(runner): cover transitional step skips verifier and clean control * refactor(trace): rename Step.Action to Step.NextAction The trace step's action field is the action chosen FOR THE NEXT iteration based on observing this step's hierarchy, not the action that produced this step. Rename Step.Action to Step.NextAction and the JSON tag to next_action to make causality explicit at the data level. * refactor(runner): assign trace action to Step.NextAction field Follows the rename of trace.Step.Action to Step.NextAction. The runner already computed the next iteration's action here; only the field name changes. * refactor(inspect): decode trace step's next_action JSON field Mirrors the trace schema rename of action to next_action. The summary shape exposed to the SPA (action_kind/action_label) keeps its current JSON tags since these are derived labels, not the raw next-action. * test(inspect): update fixtures to use next_action trace field Aligns inspect tests with the trace schema rename. Step constructors now set NextAction and the JSONL fixtures use the next_action tag. * refactor(inspect-ui): rename Step.action to Step.next_action Aligns the SPA type and consumers with the trace schema rename. The StepSummary.action_kind/action_label labels stay unchanged since they are derived labels, not the raw next-action. * fix(folio): extract balanceMatchesAddedSum predicate as testable helper Move the ledger-balance-vs-added-rows predicate into a pure helper module so the property's logic is unit-testable in isolation. Marks the sanderling example as an ES module so cross-package ESM imports resolve under node. * fix(folio): use sum-of-added-rows in balanceMatchesAddedTxn The old predicate (every row's signed amount equals delta) silently passed the double-submit bug because two same-amount rows each match the delta in isolation. Switching to the sum check (addedSum === delta) catches both the double-submit case and any future multi-row append whose total drifts from the balance change. * test(spec): cover balanceMatchesAddedSum single, sum-match, over, under cases Pins the sum-based predicate: a single new row matching delta and two new rows summing to delta both hold; two-row over-sum (double-submit) and under-sum cases both violate. * fix(build): rebuild sidecar JAR when Kotlin sources change Without source-file deps on $(SIDECAR_JAR), make never re-ran shadowJar after a Kotlin edit, so a stale embedded JAR shipped on every install and the new sidecar code was silently absent at runtime. * fix(chrome): launch with no-sandbox so headless Chrome starts in CI * fix(sidecar): type text at cursor instead of clearing the field InputText now appends at the focus caret, matching the native driver and the standard mobile-input contract, instead of deleting existing content first. Adds an injectable command runner so the behavior is testable without a device. * test(sidecar): assert InputText types at cursor without clearing Captures the adb command stream and verifies a single input-text call with no preceding delete keyevents, plus the adb escaping cases. * feat(proto): add LongPress RPC * chore(proto): regenerate Go stubs for LongPress * feat(driver): add LongPress to DeviceDriver interface * feat(sidecar): add LongPress client method * feat(mock): record LongPress action * feat(chrome): implement LongPress as press-and-hold * feat(sidecar): implement longPress across backends * feat(sidecar): dispatch LongPress RPC to backend * test(sidecar): cover LongPress dispatch * test(sidecar): implement longPress in snapshot test backend * feat(verifier): add LongPress and Scroll action kinds * feat(folio-spec): predicate that gates balance check on TxnSubmit tap Replaces the row-sum predicate (which always held by construction since balance is derived from rows in Folio) with one that compares the typed amount to the actual balance delta after a tap on TxnSubmit. Catches the planted double-submit bug. * feat(folio-spec): wire submitMovesBalanceByTypedAmount property Adds lastAction and totalBalance extractors and uses them in the new property. Drops ledgerRows/ledgerBalance extractors since nothing else referenced them. * feat(verifier): wire longPresses and scrolls generators * test(verifier): cover longPresses and scrolls generators * test(folio-spec): unit tests for submitChangesBalanceByTypedAmount Covers single vs double submit, the DoubleTap variant, vacuous cases (null action, wrong kind, wrong target, zero typed), and selector-as- object coercion. * feat(spec): add LongPress and Scroll authoring surface * feat(spec): no-op LongPress and Scroll in web runtime * feat(spec): re-export longPresses and scrolls as opt-in generators * test(spec): cover LongPress and Scroll runtime members * test(proto): expect LongPress in service descriptor * feat(runner): dispatch LongPress and Scroll actions * test(runner): cover LongPress and Scroll dispatch * docs(action-space): move LongPress, Scroll, DoubleTap to current actions * fix(runner): mark nil/empty hierarchy as transitional A failed or empty sidecar hierarchy fetch was pushed straight to the verifier, letting spec extractors crash with "Cannot read property 'map' of undefined" when findAll returned null. Treat that case like a transitional capture: skip the verifier push, still record the step, and keep the loop progressing. * fix(verifier): populate Action.On when tap chooser picks an element Coordinate-targeted Taps/DoubleTaps left On empty, so action-gated properties reading lastAction.on couldn't tell which target was hit and were vacuously skipped. Resolve the picked element to a stable key:value selector (resource-id, testTag, text, desc) and validate it resolves back to the same element so we don't accidentally redirect the tap to a sibling that shares the identifier. * fix(folio): add parseTypedAmount helper matching app's parseCents Raw user input like "50" must become 5000 cents, not 50. The existing parseDollarCents helper strips non-digits and so reads "50" as 50 cents, which is correct for formatted balance text but off by 100x for raw input from the amount field. * fix(folio): parse raw amount input as cents in submit predicate txnAmountField holds raw user keystrokes, not formatted balance text. Route it through parseTypedAmount so "50" reads as $50, matching how the app commits the transaction. * fix(folio): carry forward total balance across off-screen transitions AddTransactionScreen shows neither AccountCard nor LedgerBalance, so the extractor used to report 0 at the step before submit. That made every non-zero current balance look like the full delta and tripped the typed amount property on every honest submit. Remember the last-seen sum and return it whenever the current snapshot has no balance signal. * test(folio): cover submit predicate with raw typed-amount inputs Pipes realistic raw keystrokes through parseTypedAmount + the predicate so single submits clear and double submits fire as expected. * feat(folio): add computeHomeTotalBalance helper Pure helper that tracks Home multi-account total only and carries the last Home sum across off-Home steps. Ledger's single-account balance is excluded because mixing it would corrupt cross-screen scale comparisons. * fix(folio): totalBalance carrier tracks only Home, not Ledger Home cardSum is a multi-account total; Ledger's LedgerBalance is a single account on a different scale. Blending them in the carrier produced bogus cross-screen deltas (prev from Ledger, curr from Home), triggering false positives in submitMovesBalanceByTypedAmount. Restrict the carrier to Home AccountCard totals via the computeHomeTotalBalance helper. * test(spec): cover computeHomeTotalBalance carrier behaviour Tests Home sums, carrier passthrough on off-Home steps, the Ledger scale-mismatch case, and a Home > off-Home > Home sequence. * feat(runner): treat transient apply errors as transitional steps Sidecar input RPCs occasionally hang with DEADLINE_EXCEEDED or UNAVAILABLE on long fuzzing runs. The per-step loop previously propagated any applyAction error and killed the run after a single flake. Detect transient gRPC failures via status.FromError, mark the step transitional, skip the post-action idle poll, and continue to the next step. Fatal errors (outer ctx cancellation, non-transient codes, verifier crashes) still propagate. * test(runner): cover transient apply error resilience TestRunner_TransientApplyErrorMarksTransitional drives the runner through a wrapper that fails the first TapSelector with a gRPC DeadlineExceeded then succeeds. Asserts the run does not exit, the failed step is marked transitional with no violations, and the next step runs cleanly. TestIsTransientApplyError_Classification covers the helper's matching rules directly so future code changes don't quietly drop a transient case. * fix(folio): gate submit-balance property on Home route landing totalBalance is only freshly computed when AccountCards are visible on Home; off-Home landings return the carrier and would false-fire the property, latching always(next(F)) to false and masking the real double-submit bug. Skip vacuously when route is not "home". * test(spec): cover route gate in submit-balance predicate Adds route arg to existing cases (all use "home") and adds five new cases: ledger landing with stale carrier, add-transaction with double-insert delta, null route, plus home-landing positive and double-insert negative cases anchoring the gate's allow path.	2026-06-01 12:48:51 +05:30
pj	eed99e58aa	refactor: code organization cleanup (#35) ... * chore: fix gitignore + decisions doc after web->inspect-ui rename Update web/ references to inspect-ui/ in .gitignore and Makefile. Add decisions.md tracking architectural decisions from code-org discussion. * refactor: rename pkg/spec-api to pkg/spec Aligns the directory name with the npm package name @sanderling/spec. Updates Makefile, package.json directory field, and resolveSpecAPIPath. * refactor(verifier): split bindings.go into types.go + bindings.go Move shared public types (Action, ActionKind, LogEntry, Exception) to types.go. bindings.go retains internal JS runtime wiring only. * refactor(inspect): split runs.go into runs.go, runs_cache.go, runs_decode.go runs.go: types (RunSummary, StepSummary, RunDetail, Run) and Scan. runs_cache.go: Cache type, Open/Step/Detail methods, parseRun, scanSteps. runs_decode.go: readMeta, tallyTrace, decodeStepSummary, validRunID. * refactor: move android_env.go to internal/android/ Extracts Android device/AVD/adb logic into internal/android package. Exports EnsureDevice, AdbReverse, AdbReverseRemove, EnvWithAndroidPlatformTools, AdbBinary. Moves tests to internal/android/android_test.go. cmd/sanderling becomes a thin caller. * refactor: extract test pipeline to internal/testrun/ runTestPipeline logic moves to testrun.Execute. buildDriver, resolveSpecAPIPath, pickFreePort, and the progress logger move to internal/testrun/. cmd/sanderling/test_run.go becomes a thin adapter. Tests follow their code. * ci: update workflow paths after pkg/spec-api -> pkg/spec rename	2026-04-22 20:35:34 +07:00