sanderling

pj/sanderling

0 Fork 0

mirror of https://github.com/priyanshujain/sanderling.git synced 2026-10-02 11:07:10 +00:00

Author	SHA1	Message	Date
pj	9fb121e9d0	correctness fixes from the first real folio dispatch, and spec authoring skills (#82 ) * docs: add the apache 2.0 license text package.json has declared Apache-2.0 since the first release and .goreleaser.yaml globs LICENSE* into the archives, so that glob has been matching nothing. npm only picks up a license from the package directory, hence the copy under pkg/spec. * fix(sidecar): close the soft keyboard after typing on android * test(sidecar): pin the guarded ime dismissal * fix(sidecar): treat a failed ime probe as no keyboard open * ci(folio): let the ios leg clear state for itself * ci(folio): drop the stale frontboard note from the ios job * docs(ci): record what the ios calibration assumes and where it was measured * fix(ios): replace the session when a launch blows its bound a launch the simulator refuses is never reported: xctest records it as a test failure the runner cannot see, then holds the session's main thread for about four minutes on a diagnostic chain. so the only signal is the expired bound, and every later call queues behind the same wedge. restart the session once and launch again, bounded so the launch path stays inside testrun's backstop. * test(ios): cover the session replacement a wedged launch needs * fix(ios): share one deadline across the restart and the second launch the recovery a blown bound triggers now costs at most launchRecoveryTimeout whatever it spends it on, so the launch path tops out at 150s and testrun's three minute backstop stays a backstop. * test(ios): the restart a blown launch triggers has to be bounded * docs(ci): the ios leg does not convict on the runner, and a seed cannot fix it seed 28 reproduced its walk on macos-15 and reached the bug at the step it convicts at locally. it still could not be judged: the run did not return home between step 19 and step 136, so the counting invariant saw a rise of 15 against a window of 37 submits. * docs(sidecar): record why the stale ime flag stays out of reach * test(folio): add commonTest source sets to core and shared * fix(folio): reject amounts parseCents cannot represent * fix(folio): cap a transaction at one million dollars * test(spec): give the fake dom a real tree and a walking querySelectorAll * style(sidecar): make ktlint clean, formatting only ktlint -F over every kotlin file except DriverBackend.kt, then hand fixes where the reflow read worse and for the long lines ktlint cannot break. No behaviour changes. DriverBackend.kt is left untouched to avoid a conflict with concurrent work; its three over-long lines still fail fmt-kotlin. * fix(spec): deepQueryAll returns matches in document order * ci(folio): say what the android gate found, not why The gate proves only that AddTransactionScreen is absent from the trace. Claiming the run never got past login was an inference it cannot make: the run that produced it had logged in and was stuck on the new account screen. Name the routes the trace does record instead. * test(runner): a relaunch must not convict the submit counting property * test(browser): compare ax.find across both hosts on one page * test(spec): name the shadow match in the grammar both hosts parse * ci: move every action off the node20 runtime checkout v4->v7, setup-go v5->v7, setup-node v4->v7, setup-java v4->v5, upload-artifact v4->v7, cache v4->v6, upload-pages-artifact v3->v5, deploy-pages v4->v5, setup-chrome v1->v2, setup-android v3->v4, goreleaser-action v6->v7. setup-bun and android-emulator-runner are already node24; buf-setup-action stays on its deliberate SHA pin. setup-chrome v2 resolves stable from Chrome for Testing rather than the official installer, so the ci.yml comment about the action's default no longer held. * feat(verifier): report a relaunch on state.lastAction * test(verifier): pin the relaunch field on both hosts * fix(runner): keep the action the app was relaunched after * test: pin that a nested undefined does not survive the wire * fix(folio): uninstall before installing in just ios folio's signed-in session lives in the data container, which an install over the top keeps, so a local run started right after just ios opened on the previous run's Home screen and diverged at step 1. On CI's fresh simulator the uninstall is a no-op, so the ios leg is unchanged. * style(sidecar): bring the last three lines under the line limit * fix(ios): the runner must not answer ok for a launch that failed XCTest records a refused launch as a test issue that never throws, so the companion returned ok for an app that never started. Check the state the app actually reached and report the refusal instead. * test(ios): a refusal the runner names costs no session restart The session restart is for a launch that never answers. A launch that reports the app's state has already said what a fresh session would. * fix(folio): attribute a created account by its whole key, not a suffix createdAccountHasNonZeroBalance matched the created card with endsWith, so an older account whose name ends with the typed one ("Emergency Fund" for a typed "Fund") was judged instead whenever the new card was clipped out of the reading. Build both keys the card can carry, the plain name and web's initials + name, and compare them whole. * fix(hierarchy): object selectors resolve by the same rule as string ones * test(verifier): both ax.find selector forms resolve the same element * test(browser): the cross-host fixture uses the object selector form * fix(replay-ui): wrap the tab strip so its last tabs stay clickable * test(replay-ui): drive the fuzzer onto a violating step with a panel * feat(folio): judge a submit against the account's own balance The counting invariant can only close its window on Home, and the iOS run in #78 went 117 steps between two Home readings: 37 submits against a rise of 15 transactions is no evidence about the double tap sitting inside it. The ledger and the add-transaction screen both show the account's own balance, and an accepted submit pops back to the ledger, so a window bounded by those readings holds one action. The bound is an upper one: a balance that has not moved is a commit still in flight, a rejected submit or a tap that never landed, and none of those is a violation. Moving by more than the one submit in the window typed is. * test(runner): an overlay dismissal must not convict the counting property * docs(runner): point the guard comment at the renamed test * docs(replay-ui): name the viewport the tab overflow was measured at * feat(folio): close the submit window on the account's own screens submitCommitsOneTransactionPerAction now states its rule over two windows: the Home counts it already compared, and the account balance the ledger and the add-transaction screen redraw on nearly every frame of the transaction flow. Same rule, and the second window is usually one action wide. * fix(sidecar): reach the adb server the environment names buildDadb hardcoded localhost:5037, so a serial-addressed device always resolved through this machine's adb server and ADB_SERVER_SOCKET was ignored. Read the endpoint the way the adb CLI does instead. Fixes #79 * test(sidecar): pin the adb server endpoint parsing * fix(sidecar): close a keyboard standing in the snapshot A tap on a text field raises the keyboard and nothing closed it, so the tree the picker chooses from was missing every app node underneath it, the submit control included. Close it before the read rather than after the tap: the picker only ever sees snapshots, and the keyboard is still on its way up when the tap returns. Fixes #78 * test(sidecar): pin the tree-guarded keyboard dismissal * chore(make): a target that runs folio's unit tests * chore(folio): a just recipe for the unit tests * ci: run folio's unit tests on every pr * ci: switch to jdk 21 only for the folio step * docs(sidecar): put the measured read cost in the dismissal bound * fix(folio): decline the two demanding properties across a relaunch The runner now keeps lastAction and marks it relaunched: true where it used to report nothing at all, so the two properties that demand an effect judge a step whose process may have died before the write landed. submitChangesBalanceByTypedAmount and createdAccountHasNonZeroBalance both decline there. The counting bound does not: a relaunch cannot manufacture a transaction, and the submit is counted, so declining would throw away the detection the runner fix restored. * docs(folio): say why the merged card key cannot be made injective Folio rejects a duplicate account name, so the twin the drop rule guards against is two names the web key cannot tell apart, not two accounts sharing a name. State what closing the rest would cost and what the tree would have to carry to close it properly. * test(folio): pin that two accounts can render the same card text The proof behind the comment: "Travel1" holding 25 transactions and "Travel12" holding 5 merge to the same string, so no identity key read off a web card can tell them apart. * fix(sidecar): bound the diagnostic adb reads adbOutput and readLogcat read to EOF and then waited with no timeout, so a wedged adb held the step for as long as it liked; one stall over a remote adb server measured ~100s. The bound has to sit on the read, not on waitFor: a wedged adb never reaches EOF, so a bounded waitFor after the read is a line that never runs. * test(sidecar): pin the bound on a wedged adb read * fix(sidecar): an unreadable animation count is not idle Defaulting the count to zero made a dumpsys that said nothing mean nothing is animating, so a degraded link broke out of the settle early and handed the runner a frame caught mid-animation. Unknown now waits, inside the deadline waitForIdle already holds. * test(sidecar): unknown animation state must not read as idle * fix(folio): stop spending the submit budget on taps the app refused The window is an upper bound on the transactions an interval could hold, and a bound inflated by taps that commit nothing is a bound the app can never exceed: #78 read a rise of 15 transactions against 37 submits. TxnSubmit is clickable(enabled = amount.isNotBlank()) and parseCents refuses anything its regex misses, so a tap whose landing frame shows a refused amount cannot have committed. Over four recorded android runs that is 19, 11, 25 and 25 of 35, 26, 42 and 42 submit taps. A relaunch is excepted: a fresh process draws an empty field whatever was submitted. * feat(folio): read the amount field into every submit window Each of the three windows asks whether the tap could have committed, off the field as the landing frame shows it. * fix(sidecar): a foreground read that fails degrades the typing guard An unreadable dumpsys passed a null owner to typeChunks, which switches the mid-type focus guard off outright and lets the rest of the string spray into whatever holds the foreground. Fall back to the launched bundle instead: the guard stays armed, typing still happens, and the degradation is said out loud rather than assumed away. * test(sidecar): pin the degraded typing guard both ways * test(runner): answer Snapshot and Hierarchy off one tree in the fakes * feat(runner): skip a step whose tree changed between two reads * test(runner): cover the reread's cost to the existing snapshot rules * fix(ios): clear app state before the automation session attaches New performs the clear-state reset, so the uninstall and reinstall no longer land underneath a live XCTest session that is already bound to the app. Launch refuses a clear-state request the driver was not built for rather than reinstalling under its own session. * fix(ios): the device path clears before its runner session too * fix(testrun): thread clear-data into the ios drivers * test(runner): a skipped step must not swallow the action before it * fix(runner): hold the action back on a step nothing verified * refactor(runner): drop the empty branch from the hold path * docs(runner): describe both modes of the composing test driver * fix(sidecar): erase a field by selecting it, not one delete per character maestro's eraseText sends one delete per character through its instrumentation, measured 29.6 ms/char on the API 34 emulator. The 4096-character string the corpus types cost ~121s to clear, a fifth of a 20 minute run spent on one step, and it recurred every time that field was typed into again. Select the content and delete the selection instead: two key events at any length, measured 0.15s to 1.16s for 4096 characters across API 34, 35 and 36. The result is read back off the tree, and a field that is not empty, or that the tree cannot report on, is finished off per character in batches rather than assumed clear. Fixes #80 * test(sidecar): pin the constant-cost erase and its residue check * fix(sidecar): find the erased field by class, past the keyboard's own focus The check that decides whether the select-all worked looked for an "editable" attribute maestro's tree does not carry, so it answered "cannot tell" every time and every erase paid the per-character fallback. Worse, an open keyboard puts a second focused node in the tree, one of the IME's own keys, carrying no text: taking the first focused node would read a field still holding 4096 characters as empty, which is the one answer that stops the erase early. Match the text field by class instead. Measured against the real backend, 4096 characters now clear in 385ms on API 34, 409ms on API 35 and 870ms on API 36, verified empty, where the fallback took ~4s. * test(sidecar): use the tree the device really returns * docs(ci): the android step number describes a local emulator, not ci the leg disables animations and the number was measured with them on. the first real dispatch carries 4 transitional steps over 200, so the cross-fade wait does still fire in ci, just far less often. * fix(android): say what the sdk lookup checked, not just to set ANDROID_HOME * fix(doctor): resolve adb and emulator the way a run does * docs(cli): the android doctor checks are not path-only * fix(testrun): preflight resolves adb through the sdk, not just PATH * docs(skills): add a spec review skill and the skills index * fix(testrun): report a sidecar that dies at startup as the exit it was * test(testrun): cover the sidecar shutdown path after an early exit * docs(skills): add a property patterns catalogue skill * fix(folio): bound the total-balance move instead of demanding it exactly The write finishes before AddTransactionViewModel navigates, but nothing establishes that Home's total has re-rendered before the frame is read, and an equality convicts a healthy app for a total one frame behind. A delta of zero is exactly the shape nine of the eleven measured android false convictions had. 2x still exceeds x, so all four recorded convictions survive, checked against the traces. The trade is real: a balance that moves by LESS than the amount typed is no longer judged anywhere in this spec. * docs(folio): say what property 2 demands now that it is a bound * docs(skills): add a spec authoring skill covers hooks, extractors, selectors, properties, actions and the order to write them in, with a complete sample spec that typechecks against the real export surface. * docs(skills): ground the property patterns catalogue in the merged specs * docs(skills): name the selector keys that still substring match * docs(skills): add a setup skill for adopting sanderling * docs(skills): add a run triage skill * docs(skills): point the setup skill at its siblings * docs(manual): correct the flags the cli reference gets wrong --launcher-activity does not exist in cmd/sanderling/main.go. --device, --android-app-path and --arm do and were undocumented. runs.md still listed --max-steps and --exit-on-violation as unshipped, and described --clear-data as opt-in when the default is already true, contradicting itself ten lines on. * test(folio): pin that a commit stays in the window until Home reads it The interaction that keeps a stale Home card list from ever banking counts the budget has already forgotten: a submit lands on the ledger, so the reading that resets the window is a whole action later and the submit is still in it. Characterization, not a regression: no code changed and it cannot go red first. * docs(folio): record why a banked card reading can be trusted as current The freshness rule rests on the app popping one entry back to the ledger, not on anything the frame carries, so the assumption and the measurements behind it belong next to it. * refactor(folio): name the balance property for the bound it asserts it stopped being an equality and became \|delta\| <= typed, so the old name demanded more than the property does. renamed with the ci gate's GATED_PROPERTIES in the same commit so the gate never sees a name it does not know. * fix(android): a refused uninstall must not pass for clear-state adb uninstall answers Failure [DELETE_FAILED_INTERNAL_ERROR] both when the package was never installed and when it refuses to remove one, so the failure text cannot say which happened and the old code installed over the top either way, keeping the data clear-state was asked to drop. Ask pm path instead, and fall back to pm clear when the app is still there. * fix(ios): a failed simctl uninstall must fail the reinstall simctl install over an installed app carries its data container across, so discarding the uninstall error reported a clear-state that never happened. Uninstalling an app that is not installed exits 0 on a booted simulator, so every failure here is a real one. * fix(ios): a failed devicectl uninstall must fail the reinstall same hole as the simulator path: devicectl install over an app keeps its data, and the discarded uninstall error hid it. Uninstalling a bundle id that is not installed exits 0 with 'App uninstalled.' on a paired iPhone, so a failure here is always real. * docs(android): say why the uninstall text cannot be read * test(android): name the uninstall failure for what it says, not why * docs(ci): the ios leg convicts on the runner now, and why it did not before * docs(ci): the cross-fade wait does not fire on ci, say so * fix(runner): a bounded hold puts the swallow back one step later the hold carries one action; letting the runner act again while the verifier is still skipped overwrites it, so the carried action reaches no spec. hold for as long as the verifier is skipped, and settle on a held step so the reread pair is not tighter than the window the detector was measured over. * test(runner): pin what the two reads are compared on structuralShape excluding text and bounds is the decision separating this feature from a run that verifies nothing, and only prose held it. adding either field back now turns a case red. * fix(ci): close shell injection into the npm publish job A refname is attacker-controlled and git permits backtick, $, (, ; and \| in it. Three sites substituted it into a run: block, and NODE_AUTH_TOKEN sat at job level, so a pushed tag ran arbitrary commands with the publish credential in reach. The tag now goes through env:, is validated against an anchored version pattern before anything consumes it, and reaches the other jobs as a job output. The token is scoped to the publish step. release-npm declares contents: read instead of inheriting the repo default. * fix(ios): recognise every shape a blown launch bound arrives in The runner transport reports a blown budget two ways, its own comment says so: the context's error once cancellation has landed, and the connection's i/o timeout when the deadline armed from that context fires first. The legacy transport reports it as a gRPC status. errors.Is against context.DeadlineExceeded only matches the first, so the session restart never fired for the other two and a wedged session stayed wedged. * test(ios): drive the launch recovery with what the transports return The wedged-session fake answered with ctx.Err() raw, which is the one shape the guard already matched. The recovery now runs against the error each transport really produces for the same expiry, taken from a runner and a legacy companion that never answer. * test(runner): pin both guard writes to what the spec reads deleting lastAction.Relaunched or lastAction.Applied left the whole suite green, so the only producer of the two fields every spec-side guard reads had nothing holding it. both now assert the value out of the trace. * fix(testrun): a run that judged nothing is not a green run every step skipped means no property ever evaluated, so no violations is the absence of a verdict rather than a clean one. the hold makes that reachable now, so the run says it instead of exiting 0. * fix(ios): stop the app before clearing its state Launch terminated and then cleared; the clear moved to construction and left nothing stopping the app first. The container wipe deletes files a live app still holds open, and the CI ios leg passes no app path so the wipe is the path it takes. simctl stops it, since the clear now runs before any automation session exists. On a device the uninstall that is its only clear takes the running app with it. * test(ios): pin the stop that has to precede a clear The ordering probe now records the stop, and a scripted xcrun holds what reaches the tool: terminate before get_app_container, with the previous run's files gone after. A simctl terminate that finds nothing to stop still leaves the clear a success. * docs(spec): an unbounded eventually is violated at run end * docs(skills): an unreached eventually convicts at run end * docs(skills): noUncaughtExceptions only fires on web * fix(ios): a device clear-state that cannot happen must fail --clear-data on a physical device with no --ios-app-path warned and then ran anyway, so the run started on the previous run's data while the flag said it started clean. There is no data-container wipe on a device, so there is nothing to fall back to. * test(ios): a device clear-state without an app path ends the run * docs(skills): the stock properties each cover one platform * docs(ci): the balance property demands a bound, not an equality * fix(ios): the clear-state guard checks the bundle that was cleared A bool only said that something was cleared, so Launch(ctx, otherBundle, clearState=true) passed the guard and reported a reset that had reached a different app. Record what was cleared and compare against the bundle being launched. * test(ios): a clear-state launch for an uncleared bundle is refused * docs(manual): the flagship property is a bound, and say what that costs * fix(ios): one address picker for every bring-up bringUpRunner reads the picker from a field, and NewDevice only ever set the device one, so a device driver that reached bringUpRunner would call nil. The two fields held the same function; keeping one leaves no path that can be wired without it. * test(ios): a device driver can bring a runner up * docs(skills): both shipped balance forms are bounds now * docs(skills): name the balance predicate that still exists * docs(skills): quote the doctor the binary actually prints * docs(skills): screen= is the chrome driver's url, web only * docs(skills): substring selector matching is native only * docs(skills): web selectors are exact, native ones are substrings * fix(ci): a run that wrote no trace is not evidence about folio run_dir is empty when the run produced no output directory, and the fallback made trace ./trace.jsonl. A stray trace in the working directory was then read as this run's, so a run that wrote nothing reported 'found the submit bug' and exited 0, defeating the missing-trace check below it. * fix(ci): fail folio when a gated property is not in the spec Nothing tied GATED_PROPERTIES to the spec it gates. Renaming a property left the classifier matching nothing: ios and web blamed the spec for finding a different bug, and android silently reclassified a real conviction as 'judging health only' and stayed green. replay-ui-summary.sh already makes this check for its own list. The spec path becomes SPEC-overridable the same way, so the check is testable. * test(ci): cover the folio classifier's verdicts 21 cases through a stubbed sanderling: every exit path, the drift check, a missing trace, a zero-byte trace, an empty glob and a truncated line. Asserts the flags that reached the binary, not just the exit code. Invoked as bash -eo pipefail -c, which is what a run: block does. Running folio-run.sh itself under -e would kill it at the first non-zero sanderling test, which is the exit code it exists to read. * docs(skills): defaultActions bundles five of the eight generators * test(ios): name the picker test for what it covers * docs(skills): three of the replay-ui properties are cross-panel * docs(manual): state.exceptions is web only and reportError does not exist * fix(folio): the bound carries no unconfirmed-submit guard deleting confirmedApplied here broke 0 of 355 tests: under a bound a submit that may not have landed moves the balance by 0, which the bound already permits, so the guard could only ever drop the double commit it exists to catch. the relaunch guard stays for a reason the bound does not cover, and both tests now assert a verdict that changes when their guard does. * docs(ci): three of the replay-ui properties are cross-panel * docs(manual): the starter property only fires on web * test(folio): judge the conjunct on the landings a real run produces three of the 18 frames the recorded ios run drove it down, each with the second commit the bound is there to catch. neutering the comparison reddens it: a second commit on 357900 went unjudged. * test(folio): the walk drives the composition the spec runs countSubmitsInWindow never saw an amountText here, so every walk test counted submits the app must have refused. with the field passed, a refused submit no longer buys a later double tap an alibi: without it the window reads 3, not 1. * docs(folio): say which double submit the conjunct can see, and which it cannot the home landing is the counting invariant's, three of three in the recorded ios run; this one gets the interleaving whose second pop is cancelled. it is still the only judge on the 18 ledger landings that run produced. * docs(folio): the narrow window is not where the detection comes from the double taps land on home, so the counting form convicts them; what turned 0 convictions into 4 on the recorded ios run is submitCouldCommit, which drops the windows at those three steps from 5/4/7 to 2/1/2. * docs(skills): folio drives three platforms from one spec * fix(folio): an amount over the app's cap spends no window budget the corpus reaches TxnSubmit with 999999999999999999999, AMOUNT_REGEX takes it and AddTransactionViewModel refuses it against MAX_TRANSACTION_AMOUNT_CENTS, so counting it was budget a double submit could hide behind. * docs(folio): say which form judged one step, not which node was read once * docs: a bound still needs the relaunch guard, and eventually does convict * fix(sidecar): the hierarchy rpc serves the tree the snapshot reads the runner compares the two per step, but snapshot settles and closes a keyboard while hierarchy was a bare contentDescriptor. measured on emulator -5556 (api 34) with an ime open: 489 nodes against the snapshot's 134. both now come off snapshotTree under the same lock; the reread still costs ~75ms when no keyboard is up. * docs(runner): say what makes the two reads comparable the reread's comment claimed the round trip was the only interval between them; what it left out is that the two rpcs have to read the same way, which the repo's own android backend did not do. * refactor(runner): name the settle predicate for what it means * ci: add a headless-chrome composite action The setup-chrome / apparmor sysctl / launch-check trio is copied across three jobs. The old comment described setup-chrome v1 semantics: under v2 stable is the default and the alternative is Chrome for Testing latest, not a dev Chromium, so it is restated for what the pin actually does. * ci(examples): add the folio setup actions folio-app holds the per-platform toolchain and app build, so a caller guards one step instead of eight. folio-simulator boots the simulator, installs folio and leaves the app stopped. * ci(examples): add the replay-ui fixture action Records a trace and serves it with sanderling replay. The step page URL is a composite output rather than GITHUB_ENV, so it is scoped to the one step that drives it. * ci(examples): one dispatch workflow for every example folio.yml and replay-ui.yml ran the same operation: build sanderling for a platform, bring a target up, run a spec against it, classify the trace, upload the run. They are now one matrix over four examples, each naming its own runner. The job is named for what it fuzzes. 'dogfood' named why we run it, not what runs, the same error as a diagnostic that reports a motivation instead of an observation. The matrix is computed by a plan job because jobs.<id>.if cannot read the matrix context, so a static matrix has no way to leave a leg out. Seeds, budgets, timeouts, runners and artifact names are unchanged. * ci: reuse the headless-chrome action in the browser job Same three steps the examples workflow needs, and the comment explaining the AppArmor sysctl now lives in one place. * ci: move the folio jdk step to setup-java v5 The only setup-java left on v4; every other one moved. * ci: pin third-party actions to commit shas buf-setup-action was already pinned with a comment saying why; the other five rode mutable major tags, so a tag move is an unreviewed change to what runs. Each major currently resolves to the release named in the comment, so this freezes today's behaviour rather than changing it. actions/* stay on major tags: they are first-party to the runner. * ci(replay-ui): name the run directory for what it fuzzes runs/dogfood and the '### replay-ui dogfood' heading carried the same naming error as the job name: dogfooding is why the run exists, not what it fuzzes. * docs(driver): state the log level scale on LogEntry Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * docs(sidecar): name the device the node counts came off * fix(chrome): keep a log entry the level scale cannot rank Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * fix(chrome): record console levels on the logcat scale Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * docs(ci): say why upload-pages-artifact needs no include-hidden-files v3 to v5 crossed v4's change to exclude dot-files. build/site has none, so nothing was dropped, and the underscore directory is not hidden. * test(browser): drive a console error through to the spec Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * docs(ioscompanion): name the vacuity behind the empty log slice Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * ci: run the folio classifier's test in make test-ci-scripts * fix(chrome): keep the message of an object console argument Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * test(browser): cover console.error with an error object Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * feat(spec): let the runner install state.logs in the page Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * feat(verifier): encode state.logs for the web host Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * feat(chrome): install the step's logs in the page Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * test(ci): pin three real folio traces from run 31902501859 ios convicted on submitCommitsOneTransactionPerAction, web on both gated properties, android ran its full 200 steps healthy. Every step is kept; of each step only step, violations, witnesses and residuals survive. hierarchy is replaced by the quoted "...Screen" resource ids it held, in order. It cannot just be dropped: it is 95% of the bytes and also the only place the android route gate's grep can match, so dropping it flips that leg from healthy to 'never reached'. 8.4MB to 59KB. * test(ci): drive the classifier over the real traces Four cases on real data: each leg's real verdict, plus the android trace cut before it reached the transaction screen, which is what proves the route gate reads a real hierarchy dump. Also corrects the hand-written fixtures. They set is_error to false on a plain violation; internal/trace/writer.go tags that field omitempty, so a real trace omits it entirely. Harmless to the classifier, but a fixture that does not look like reality is the thing that hides drift. * test(runner): teach the web fakes to take the step's logs Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * fix(runner): install the step's logs before the page extracts On web every extractor reading is replaced by the one the page computed, and the page answered logs: [], so noLogcatErrors counted an empty array however full of errors the console was. Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * test(runner): cover the logs reaching the page and failing to Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * test(spec): cover the host pushing state.logs into the page Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * test(browser): drive console.error through to a fired property Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * fix(runner): report a log fetch the driver could not make The comment claimed the failure was warned about; nothing warned, so a device whose log fetch failed every step held noLogcatErrors on evidence nobody collected. Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * test(runner): cover the silently dropped log fetch Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * fix(ci): read the route the spec reports, not the hierarchy dump The android health gate grepped the trace for "AddTransactionScreen", which occurs in exactly one place: the hierarchy dump, as a resource-id. That is a debug artifact standing in for a fact the spec already reports, and it was wrong in both directions. Against the real 8.4MB trace with hierarchy stripped, the old gate failed a healthy 200-step run; against a trace carrying the marker on a transition frame without the route ever being reported, it passed and called it healthy. It reads extractor_changes.route now, whose values come from SCREENS in the spec, so the gate and the app agree on what being on a screen means. routeOf answers null on a frame showing two screens, which is exactly the frame the marker was matching. The drift check grows to cover both new names: extract("route") and the SCREENS key. The fixtures are re-derived keeping the route entry and no hierarchy at all; the full artifacts and the fixtures give byte-identical verdicts, which is what proves the coupling is gone. Script and fixtures move together: either alone leaves the suite red. * ci: check that the workflow references resolve actionlint reads a local action's inputs but never checks its path exists: uses: ./.github/actions/typo lints clean and fails only when the job runs. Covers composite action paths, make targets including the ones the examples matrix builds from $SANDERLING, and the scripts a run: step invokes plus their executable bit. Fails when it parses fewer references out of a file than that file mentions, because a checker that matches nothing reports a safety it never looked for. * ci: lint the workflows on every pr The workflows that fuzz the examples are dispatch-only, and GitHub will not dispatch a workflow that is not on the default branch, so their first real run is after merge. actionlint and the reference checker are the only things that can fail before that. actionlint is pinned by commit, and its tool version is pinned too so a new release cannot change what CI enforces. * ci: collapse the four workflows into one Nine jobs written out one by one, each with its own steps and its own calibrated seed and budget as literals. Triggers are pull requests, master and v* tags, and a dispatch with no inputs. Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * ci: inline the two composite actions with one caller each Both existed to give the matrix a per-target hook. folio-app and headless-chrome stay: three and three callers. Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * ci: check that no run: block interpolates an expression A ${{ }} lands in the script text before bash reads the line, and actionlint only flags the contexts it already knows are attacker controlled. Nothing enforced the rule the workflow follows. Also drops the matrix table lookup, which has no table to read now. Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * ci(folio): name the run, not the fuzzer, in the clean-run message Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * docs: point at the workflow that holds the release secrets now Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * ci: name every job Category (variant), and gate the lot on one check Follows the convention in antithesishq/bombadil: the display name is what groups a run in the Actions UI, so Check (tests), Check (browser), Check (workflows), Folio (android), Folio (ios), Folio (web), Replay UI, Release and Docs. Every job carries a name, so none of them falls back to its kebab-case id. All checks passed needs all nine and runs with if: always(), so branch protection has one check to point at and a skipped job cannot read as a pass. Release and docs now gate on startsWith(github.ref, 'refs/tags/v') alongside master, which is the form the trigger filter already uses. Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * ci: split the release job back in two Collapsing them left the npm publish steps in a job holding contents: write, because GoReleaser needs it, so npm ci ran its dependency lifecycle scripts with a write-capable GITHUB_TOKEN in reach of the same job as a live NPM_TOKEN. Release (npm) is back on contents: read and Release (cli) keeps contents: write, which is what they each had before. Each validates the tag from its own copy of the pattern rather than waiting on a job that exists only to pass a string. Release (cli) is tags only: there is no CLI to cut on a merge. Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * ci: run folio on pull requests folio was skipped on pull requests, so ios, android and web only ever ran after a merge. The three legs are 3 to 19 minutes and run in parallel, and a superseded pull request run already cancels itself. Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ * ci: draw each group as its own box in the run graph The run graph boxes jobs together when they share the same dependencies and the same dependents. All ten jobs fed only all-checks-passed, so all ten drew as one pile. A gate per group gives each group a dependent that is exactly that group. Release and docs now need the checks, which they should have all along: npm publish and the pages deploy ran on a merge without waiting for the test job. Folio stays unblocked so a 20 minute leg does not wait on a 3 minute one. Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ	2026-08-16 13:23:53 +05:30
pj	6b0d6cb971	WIP: Drive physical Android devices over USB (#67 ) * feat(sidecar): reach USB devices via the adb server by serial * feat(test): add --device flag to target a specific Android device by serial * feat(folio): select Android device via ANDROID_DEVICE in justfile * feat(conformance): add android backend to the gate suite * feat(android): keep device awake and unlocked so the app stays foreground * feat(conformance): prep physical android device (autofill/verifier/stayon) * fix(android): make device prep best-effort so OEM-blocked commands don't abort the run * fix(verifier): require positive bounds for swipe candidates A zero-bounds element centers at (0,0); a downward swipe from the top-left corner is the system gesture that pulls down the notification shade, dragging the fuzzer out of the app. Swipes now require positive bounds like every other verb. * fix(runner): harden app-scope guard against launcher and overlays The per-step guard now relaunches and waits until the app window is actually drawn before proceeding, so a slow physical-device relaunch no longer lets an observe or action land on the launcher. It also detects a system overlay (notification shade) stealing window focus while the app stays resumed, and dismisses it with back. * feat(android): harden physical-device runs in device prep Device prep now disables the AOSP cached-app freezer, phantom-process killer, and Doze (and exempts the driver) so OEM background management stops suspending the driver mid-run. Adds ReinstallApp for clear-state on ROMs that deny pm clear, and teaches focus detection to report the notification shade as systemui so the scope guard can dismiss it. * feat(driver): clear-state via APK reinstall when pm clear is blocked When an APK path is set, Android clear-state resets the app by uninstalling and reinstalling instead of asking the sidecar to pm clear, which hardened OEM builds (ColorOS) deny even to the adb shell user. Falls back to the sidecar clear path when no APK path is provided. * feat(cli): add --android-app-path for clear-state reinstall Wires the APK path from the test command through to the sidecar client so Android clear-state can reset apps on OEM builds that deny pm clear. * chore(folio): pass --android-app-path in just test * fix(runner): clamp swipe/scroll origin out of edge gesture zones A gesture starting in the top status-bar strip pulls down the notification shade; the bottom and side strips are the home and back gestures. Any of them drags the fuzzer out of the app. Swipe and scroll origins are now clamped into a safe inner area sized from the maximum element extent (the Android hierarchy root reports zero bounds, so the extent is the reliable screen size). Calibrated on device: origins below ~7% of height no longer open the shade. * perf(sidecar): faster Android text input and drop redundant settle poll inputText now uses adb `input text` for short shell-safe ASCII (~5x faster than the driver's per-character path) and falls back to the driver for unicode, injection payloads, and overflow-length strings. waitForIdle drops the structural-hash poll that followed waitForAppToSettle: each hierarchy fetch is ~500ms on a physical device, so it cost ~2.8s per mutating step for marginal benefit, and the runner already re-fetches transitional frames. Cuts p95 step latency from ~6.5s to ~5.1s; G1-G4 still pass. * fix(verifier): exclude soft-keyboard region from action candidates The fuzzer was tapping Gboard's "Settings" key, navigating out of the app. That key is a bare FrameLayout with a content-desc and no package or resource-id, so the package-based scope filter missed it. Candidates whose center falls in the keyboard region (derived from the IME elements' bounds) are now dropped, so no tap or long-press lands on a key. Opt-in with app scoping; unscoped runs keep every node. * perf(runner): replace focus-tap settle with a brief wait The full WaitForIdle after a field-focus tap cost ~0.5-1s per InputText step on a physical device while the keyboard animated in. The tap registers focus immediately and text is injected into the focused view, so a short fixed wait suffices. Drops p95 step latency ~5.1s to ~4.0s; G1-G4 stay green. * chore(conformance): platform-aware G5 p95 budget for android The 2500ms ceiling was calibrated on the iOS simulator. A physical Android device drives every step over USB (snapshot + settle + adb round-trips), so its per-step floor is several times higher; holding it to 2500ms would force removing the settle/retry logic the correctness gates depend on. The android backend now defaults to 4500ms (override with P95_LIMIT_MS); iOS stays 2500. * fix(sidecar): retry maestro android driver startup The maestro Android driver's dadb.open() occasionally misses its startup deadline (its instrumentation host is slow to come up right after a reboot or per-run reinstall), which aborted the whole run. Retry the open a few times with a short backoff so a transient timeout recovers. * chore(conformance): widen android G5 budget to 5500ms Physical-device p95 swung 3209-4612ms across sessions (cold runs right after a reboot are slower). 4500ms was too tight for that jitter; 5500ms covers the observed ceiling with headroom. * web replay fix * feat(android): force 3-button nav during runs to prevent app drift On gesture navigation a fuzzer swipe can trigger swipe-up-home or edge-back and fling the app off screen. Device-prep now switches to 3-button navigation for the run (no edge gestures; the nav bar's buttons are systemui-owned and already excluded from action candidates) and restores the original navigation mode when the run ends. Best effort: leaves nav untouched if the overlay command is unavailable. * fix(android): target the selected device in adb reads; don't strand nav mode Review fixes: - ForegroundPackage/FocusedWindowPackage now take a serial and pass -s, so the foreground/scope guard works when several devices are attached (the --device path). Previously they ran bare `adb shell`, which errors with multiple devices, silently disabling app-scope enforcement. The sidecar client passes its serial through. - Extract an adbArgs helper and route every adb call through it, removing four duplicated serial-arg builders. - ForceThreeButtonNav now decides what to restore before changing anything: if the current mode is unknown or already 3-button it leaves nav untouched, instead of switching and then stranding the device in 3-button. Logic split into the pure navModeToRestore, now unit tested. * fix(runner): restore scrollBounds doc; cover destination clamp and screenBounds Review fixes: move the scrollBounds doc comment back onto scrollBounds (it was stranded above screenBounds by an insertion). Extend the clamp test to assert an off-screen destination is clamped onto the screen and that the origin lands exactly on the margin. * test(verifier): cover keyboardRegionTop, including the decor-view guard The full-screen IME decor view rejection had no test; removing it left the suite green. Add direct cases: no keyboard -> sentinel, decor view ignored in favor of the real keyboard line, and decor-only -> sentinel. * style(cli): gofmt testOptions field alignment * fix(sidecar): keep a leading dash off the fast input path A value starting with '-' could be read as an option by `adb input text`, so the fast-path regex now requires a non-dash first character; such values fall back to the driver. Also cover the dadb-target branch where a colon precedes a non-numeric port (a USB serial, not host:port). * refactor(verifier): scope action candidates by window ownership Replaces the leaky per-element package check and the keyboard-region Y heuristic with one rule: walk the window tree propagating each node's owning package (empty and the neutral android framework package are transparent); a node is in scope only when no concrete foreign package owns it (the app's own window carries no package on Compose apps) or the owner is the app package. This drops whole foreign windows (soft keyboard, system UI, launcher) AND their empty-package child wrappers -- e.g. a keyboard's 'Settings' key, which the old empty-package-is-in-scope rule admitted and which navigated out of the app. Deletes keyboardRegionTop/isInputMethodElement. * fix(runner): re-check foreground at apply time, skip stale actions ensureForeground runs before observe, but the app can leave between observe and apply (a prior gesture settling late); swipes/keys then fire stale coordinates onto whatever screen is now up. Re-check foreground immediately before applying and, when the app is gone, skip the action and log it (making the escape visible) so the next step's guard relaunches instead. * fix(android): type long ASCII via fast guarded path to stop keystroke escape A 4096-char corpus string exceeded the fast input cap and fell to the per-character driver path, which takes ~120s. During that uninterruptible window focus could leave the app and the remaining keystrokes sprayed into the launcher search box. Route shell-safe ASCII of any length through adb input text, chunked, re-checking the foreground app between chunks and stopping if it changed. * chore: ignore gate artifacts and local scratch files * refactor(runner): narrow gesture clamp to the top shade strip 3-button nav (forced for every run) disables the side back and bottom home gestures at the OS level. On-device probing confirmed side and bottom swipe origins no longer drift, leaving the notification shade as the only edge gesture a swipe can trigger. Clamp only the top strip; keep origin and destination on screen otherwise. * chore(format): add .editorconfig enforcing 80-column limit * chore(format): add prettier config with 80-char printWidth * chore(deps): add prettier devDependency to replay-ui * chore(deps): add prettier devDependency to folio-web * chore(deps): add prettier devDependency to spec package * chore(format): add swift-format config with 80-char lineLength * feat(format): add make fmt targets for per-language 80-col formatting * fix(runner): translate gesture to safe area so near-top scrolls keep direction Clamping the swipe origin to the top margin while leaving the destination on the full screen used two reference frames: a scrollable container pinned in the top strip had its origin pushed past the destination, reversing the gesture. Translate the whole from->to segment down by the same delta so the origin clears the shade strip without flipping direction. Adds a scroll-near-top test that fails under the old origin-only clamp. * fix(runner): apply-time guard consults focused window, not just resumed activity ensureForeground detects a system overlay (notification shade) owning the focused window while the app stays the resumed activity, but appIsForeground only queried ForegroundApp. A swipe that pulls the shade over the app between observe and apply then fired onto the shade. Mirror the focus check at apply time so the action skips and the next step dismisses the overlay. * test(runner): cover apply-time foreground skip and appIsForeground table Adds a Run-level test asserting no tap reaches the driver while a system overlay holds focus (guards against the skip branch being dead-coded), plus a decision-table test for appIsForeground. Adds ForegroundErr/FocusedWindowErr to the mock driver so the guard's transient-read paths are exercised. * fix(sidecar): harden android driver open, input guard, pressKey, foreground marker - openWithRetry rebuilt a closed AndroidDriver, whose gRPC channel is final and shut down by close(); the retry then ran against a dead channel. Build a fresh driver per attempt and extract a unit-tested retryOpen helper (named DRIVER_OPEN_ATTEMPTS/BACKOFF). - pressKey on the Maestro backend did KEY_MAP[key] (no lowercase, no throw), silently dropping unknown or wrong-case keys; route through a pure maestroKeyFor that lowercases and rejects unknown keys like the Stub contract. - the mid-type foreground guard (typeShellSafe) was untested; extract a pure typeChunks and cover stop-on-foreground-change, always-send-first-chunk, and unknown-owner. - foreground detection required the literal topResumedActivity=ActivityRecord; align parseResumedPackage to the same ResumedActivity marker set Go reads so OEM wording does not disable the guard. fix(conformance): pin self-test p95 budget and score install failures as run failures self_test reused the backend-dependent P95_LIMIT_MS, so under BACKEND=android the 4000ms slow fixture rated PASS and the offline analyzer check failed from an env var; pin it to 2500. A per-run adb install failure ran unguarded under set -e and aborted the whole harness; guard it, record the run as a G1 failure, and continue. * fix(android): require --device when several devices are connected With no serial requested and more than one device online, pickDevice silently returned connected[0], but that serial is never threaded into the per-step adb calls, so every later bare adb command failed with "more than one device". Error instead and ask for --device, mirroring pickAVD; a single device stays unambiguous. * refactor(android): move PrepareDevice doc onto it; extract tested wakeCommands The PrepareDevice doc block was stranded above adbArgs, leaving the exported function undocumented under godoc. Move it back and split the wake/keyguard tuples into wakeCommands so they have a unit test. * perf(verifier): memoize scopedElements per tree scopedElements rebuilt a full tree walk plus map on every candidatesForVerb call (~16 per step). Cache the result keyed on lastTree and invalidate it in PushSnapshot. * fix(sidecar): default reinstallApp in SetClearStateReinstall; cover non-android clear Only Dial set reinstallApp, so a Client built another way would nil-deref on Android clear-state. Default it in SetClearStateReinstall too. Add a non-android test so the platform guard has negative coverage: dropping the android check would now fail. * test(runner): make focusTapSettle injectable so apply tests don't sleep 250ms The focus-tap settle was a const, so five InputText apply tests each blocked the full 250ms. Make it a package var and shorten it per-test with cleanup. * refactor(runner,android): drop unused bringToForeground return; grep no-match yields empty bringToForeground's bool return was read by no caller. FocusedWindowPackage's on-device grep exited 1 on no match, surfacing as an error instead of the documented ""; add \|\| true. * perf(sidecar): reuse a single Jackson ObjectMapper structuralHash, countRouteScreens, and hierarchy each built a fresh ObjectMapper per call inside the stability poll; the instance is thread-safe and meant to be reused. Hoist one shared val. * refactor(android): remove unused AdbReverse/AdbReverseRemove No callers anywhere in the tree; they were also the only adb calls bypassing adbArgs. Dead code, removed. * style(runner): trim non-load-bearing comments from this PR's runner code and tests * style(sidecar): trim non-load-bearing comments from this PR's driver code and tests	2026-06-11 10:10:05 +05:30
pj	88db9653e5	refactoring default action layer (#51 ) * feat(hierarchy): add editable signal with native derivation * feat(chrome): emit editable flag in hierarchy dump * feat(verifier): expose editable on ax element objects * feat(spec): add editable to selector and element types * feat(verifier): register typing builtin generator * feat(verifier): typing builtin types edge-case corpus into editable fields * feat(spec): export typing builtin generator * feat(spec): add defaultActions bundle * feat(spec): export @sanderling/spec/defaults subpath * feat(folio): layer defaultActions breadth over targeted flows * test(verifier): typing builtin targets editable fields, declines otherwise * test(hierarchy): editable derivation and selector matching * test(spec): defaultActions, typing, and defaults barrel resolve * fix(testrun): alias @sanderling/spec/defaults for the bundler * test(chrome): editable flag for inputs, textarea, contenteditable * feat(spec): typing builtin for the web (V8) action path * chore(folio): auto-boot a bootable AVD in just test/install when none connected * feat(driver): add ForegroundChecker optional capability * feat(android): detect foreground package via adb dumpsys * feat(sidecar): implement ForegroundApp via adb for android * feat(runner): relaunch app when foreground escapes during exploration * fix(spec): drop hardware back from defaultActions to stay in-app * feat(spec): add DoubleTap action type and constructor * feat(spec): wire DoubleTap through web-runtime serializer * feat(verifier): bind doubleTap and decode DoubleTap actions * feat(runner): dispatch DoubleTap as two taps inside one step * test(doubleTap): cover constructor, verifier round-trip, and runner dispatch * feat(folio): add noDuplicateTxnPerStep invariant and doubleSubmitTxn action * fix(folio): track ledger row count across non-ledger steps; pin reproducer seed * feat(spec): add doubleTaps random-target builtin to defaultActions * feat(verifier): add doubleTaps random-target generator * refactor(folio): drop doubleSubmitTxn; fuzzer surfaces double-submit via defaultActions * fix(folio): make ledgerRowsSeen monotonic to suppress transient-render false positives * feat(verifier): track newly-violated property set per step Sticky `always(P)` violations re-surfaced on every step after onset, flooding traces and summaries with duplicate records. EvaluateProperties now diffs against the prior verdict map and records the onset set; a new NewlyViolatedProperties accessor exposes it so callers can emit each violation exactly once at its onset step. The verdict-map return is preserved for residual / current-verdict consumers. * refactor(runner): emit onset-only violations to trace and summary Switch the per-step violation list from the sticky verdict map to the verifier's onset set. Each property now appears exactly once across a run: at the step it first violates, not on every subsequent step where the residual stays false. Removes the dead violationNames helper. * style(verifier): use maps.Copy for verdict snapshot * fix(folio): make login spec content-driven (idempotent across re-entries) * fix(verifier): canonicalize selector strings Object/chain JS selectors used to fall through to goja's default stringification, producing "[object Object]" tags that surfaced as garbage in trace.action.selector. Emit canonical "k:v" / " > "-joined strings instead so the tag round-trips back through the hierarchy selector grammar. * refactor(folio): replace txn invariants with balanceMatchesAddedTxn Collapse noDuplicateTxnPerStep and newTxnChangesBalance into a single per-row property: every newly-appearing ledger row's signed amount must match the ledger balance delta. A double-submit lands two rows whose individual amounts cannot both equal the aggregate delta, so each row fires the property, catching both the row-count and balance-math classes of bug under one semantic invariant. * refactor(trace): drop WriteScreenshotAfter Only one screenshot per step is captured now (concurrently with hierarchy after settle), so the -after.png variant is unused. * refactor(runner): one concurrent screenshot per step Move screenshot capture into the post-action errgroup so it observes the same UI moment as the hierarchy fetch. Drop the pre-action and deferred -after captures. Skip WaitForIdle when the action is Wait since the wait itself provides settling time. * refactor(inspect-ui): use next step's screenshot for state after Each step now has one screenshot (the moment of observation). The "state after" view of step N is the same moment as step (N+1)'s observation, so reuse that file rather than expecting a separate -after.png. * feat(sidecar): structural-hash settle poll Add pollUntilStable and structuralHash helpers; wire them into the Stub, Maestro, and iOS backends' waitForIdle. The structural hash ignores bounds-only flicker (measure passes) but trips on any change in resource-id/class/content-desc/text, so a Compose cross-fade where both source and destination composables are momentarily alive no longer slips through Maestro's waitForAppToSettle and contaminates the next hierarchy fetch. * test(sidecar): cover pollUntilStable and structuralHash Verify the poll returns on two equal snapshots, after transient churn, and at the cap when never stable; assert the hash ignores bounds-only flicker and detects content changes. * feat(spec): accept optional name on extract() Add an (name, getter) overload so each extractor handle carries a debuggable label that future trace fields (per-step diffs) can key off. The web-runtime falls back to extractor_\${index} when none is supplied so existing call sites keep working unchanged. * test(spec): cover extract name overload Verify the runtime receives an undefined name in the legacy shape, the supplied name in the (name, getter) shape, and that extract("name") with no getter throws. * feat(verifier): name extractors for diff surfacing bindExtract accepts an optional name argument; falls back to extractor_N when omitted. The name is stored on extractorState alongside prev/curr value caches that the next change will use to emit per-step diffs. * chore(folio): name every extract() call Give each extractor in the Folio spec a debuggable label so the inspect UI can render extractor-value diffs at violation steps keyed by intent (ledgerRows, route, ledgerBalance, ...) rather than by registration index. * feat(verifier): track extractor value transitions Cache each extractor's prior and current JSON-encoded value during PushSnapshot; expose ChangedExtractors to surface per-step diffs the runner can emit into the trace. The first observation flushes every non-null extractor as a change so the inspect UI shows initial state breadcrumbs alongside later transitions. * test(verifier): cover ChangedExtractors diffs Verify initial snapshot reports both named and fallback-named extractors, a subsequent change surfaces prev/curr, and a no-op snapshot leaves the diff empty. * feat(trace): emit extractor_changes per step Add ExtractorChanges to trace.Step and a runner helper that converts the verifier's diff map into the trace shape. The inspect UI keys its violation breadcrumbs off this field. * feat(inspect-ui): render extractor-change breadcrumbs at violations Show prev -> curr for each extractor whose value changed on the selected step, anchored under the violation row in ActionList. Long values collapse into <details> so the inline diff stays readable while the full payload is one click away. * fix(sidecar): cap stability poll independently of settle budget The previous shape halved durationMillis between waitForAppToSettle and the structural poll, then hammered hierarchy() at 80ms intervals - on Maestro this stacked enough RPCs that hierarchy fetches began timing out under load and the run stalled. Pass the full budget to waitForAppToSettle and cap the follow-up structural poll at 600ms with a 120ms interval, so the device sees at most a handful of extra hierarchy reads per step. * feat(cli): default --clear-data on so runs start fresh * feat(sidecar): streak-based settle with route-transition detection Two changes layered into the stability poll: 1. stabilitySnapshot returns null while the tree carries more than one route-level Screen tag (resource-id / testTag / identifier ending in "Screen"), so the poll cannot declare a NavHost cross-fade stable. Apps following the Compose route convention get this detection for free; apps that don't fall through to the generic signal below. 2. pollUntilStable now requires an uninterrupted stable streak of at least MIN_STABLE_STREAK_MILLIS rather than just N consecutive matches. A late transition that fires after a brief calm window breaks the streak instead of slipping past. Interval widened to 250ms so UiAutomation isn't hammered under fuzz load. * test(sidecar): cover streak reset and route-transition rejection Verify the poll honors MIN_STABLE_STREAK_MILLIS, that a transient mid-stream change resets the streak, that null returns block streak progress through a NavHost cross-fade, and that stabilitySnapshot counts only route-level attribute keys when summing Screen tags. * feat(runner): re-fetch on transitional hierarchy capture Some actions trigger async work (DB write, ViewModel coroutine) whose navigation transition begins after the sidecar settle poll has already exited. Without intervention, the next iteration's hierarchy fetch lands mid cross-fade and the verifier observes a partial extractor state which then surfaces as a false-positive violation at the step where the transition completes. fetchSyncedState pairs hierarchy + screenshot in one goroutine and retries the pair (up to 4 times, 200ms apart) while the captured tree contains more than one route-level Screen tag. Steps that observe no transition get no added cost; steps that catch a transition pay up to ~600ms extra wall time but record a tree that matches the post-transition state the property language expects to compare. feat(runner): gate first action on app reaching foreground * test(runner): cover startup foreground gate and back-press * feat(verifier): scope random-action targets to app package Random tap/doubleTap/type/swipe candidates now exclude nodes whose package differs from the app under test, so exploration never fuzzes the soft keyboard, system UI, or permission dialogs. An unset app package or an element with no package stays in scope, preserving behavior on iOS. * feat(testrun): pass app package into verifier scope filter * test(verifier): cover package-scoped target selection * feat(hierarchy): derive package from resource-id prefix The Android sidecar omits an explicit package attribute, so the verifier's package scope filter was a no-op and the keyboard still leaked into targets. Native nodes carry their package as the resource-id prefix; derive it there when the attribute is absent. Compose testTags are colon-less and stay empty, keeping them in scope. * test(hierarchy): cover package derivation from resource-id * chore: stop tracking inspect-ui/dist build artifacts * feat(android): detect focused-window package via dumpsys window * feat(driver): add FocusedWindowChecker capability * fix(runner): gate first observe on the app window being drawn, not just resumed * test(mock): add FocusedWindowApp with foreground mirroring * test(runner): cover startup gate waiting for app window to draw * feat(proto): add Snapshot RPC for atomic hierarchy+screenshot Pairs hierarchy and screenshot in a single response so the runner can capture both under a backend mutex, avoiding the cross-fade race where the two reads describe different frames. * feat(sidecar): add snapshot default on DriverBackend Default impl calls hierarchy() then screenshot(). The service layer wraps the call in a mutex so concurrent runners observe a serialized pair. * feat(sidecar): wire Snapshot handler with serialization lock Synchronizes backend.snapshot() so concurrent runners observe a serialized hierarchy+screenshot pair, eliminating the cross-fade race where two parallel reads describe different frames. * test(sidecar): cover Snapshot wire path and serialization lock SnapshotHandlerTest asserts both fields are populated, concurrent calls are serialized, and the default impl runs hierarchy then screenshot. * feat(driver): expose Snapshot on DeviceDriver and sidecar client Snapshot wraps the new atomic-snapshot gRPC: the runner gets hierarchy and screenshot from one round-trip whose two reads are serialized on the sidecar side. * feat(driver): add Snapshot to chrome and mock drivers The chrome tab is single-threaded so its Snapshot pairs the two reads without extra locking. The mock records ActionSnapshot so tests can assert the runner reaches for the paired RPC. * refactor(runner): observe each step via the atomic Snapshot RPC fetchSyncedState now issues one Snapshot per attempt so hierarchy and screenshot describe the same on-device frame. The transitional retry stays: that case handles a fully-captured but mid cross-fade frame, which atomic capture cannot fix. * test(runner): assert step uses Snapshot, not raw hierarchy/screenshot TestRunner_UsesAtomicSnapshot catches regressions to the two-goroutine race, and the existing parallel-fetch test now keys off ActionSnapshot. * test(driver): cover Snapshot in proto descriptor and sidecar client Adds Snapshot to the descriptor allowlist and a sidecar-client test that asserts both fields come back over the wire. * feat(trace): add Transitional flag to Step * fix(runner): skip verifier for transitional trees after retry budget When fetchSyncedState exits its retry loop with a tree that still shows a NavHost cross-fade, the runner now marks the step transitional, writes the step + screenshot to the trace, and skips Verifier.PushSnapshot / EvaluateProperties / ChangedExtractors so the previous-to-current extractor advance is not poisoned by transient state. The next clean step's previous still references the prior clean state. NextAction continues to run so the loop never deadlocks on a never-stabilizing screen. * test(runner): cover transitional step skips verifier and clean control * refactor(trace): rename Step.Action to Step.NextAction The trace step's action field is the action chosen FOR THE NEXT iteration based on observing this step's hierarchy, not the action that produced this step. Rename Step.Action to Step.NextAction and the JSON tag to next_action to make causality explicit at the data level. * refactor(runner): assign trace action to Step.NextAction field Follows the rename of trace.Step.Action to Step.NextAction. The runner already computed the next iteration's action here; only the field name changes. * refactor(inspect): decode trace step's next_action JSON field Mirrors the trace schema rename of action to next_action. The summary shape exposed to the SPA (action_kind/action_label) keeps its current JSON tags since these are derived labels, not the raw next-action. * test(inspect): update fixtures to use next_action trace field Aligns inspect tests with the trace schema rename. Step constructors now set NextAction and the JSONL fixtures use the next_action tag. * refactor(inspect-ui): rename Step.action to Step.next_action Aligns the SPA type and consumers with the trace schema rename. The StepSummary.action_kind/action_label labels stay unchanged since they are derived labels, not the raw next-action. * fix(folio): extract balanceMatchesAddedSum predicate as testable helper Move the ledger-balance-vs-added-rows predicate into a pure helper module so the property's logic is unit-testable in isolation. Marks the sanderling example as an ES module so cross-package ESM imports resolve under node. * fix(folio): use sum-of-added-rows in balanceMatchesAddedTxn The old predicate (every row's signed amount equals delta) silently passed the double-submit bug because two same-amount rows each match the delta in isolation. Switching to the sum check (addedSum === delta) catches both the double-submit case and any future multi-row append whose total drifts from the balance change. * test(spec): cover balanceMatchesAddedSum single, sum-match, over, under cases Pins the sum-based predicate: a single new row matching delta and two new rows summing to delta both hold; two-row over-sum (double-submit) and under-sum cases both violate. * fix(build): rebuild sidecar JAR when Kotlin sources change Without source-file deps on $(SIDECAR_JAR), make never re-ran shadowJar after a Kotlin edit, so a stale embedded JAR shipped on every install and the new sidecar code was silently absent at runtime. * fix(chrome): launch with no-sandbox so headless Chrome starts in CI * fix(sidecar): type text at cursor instead of clearing the field InputText now appends at the focus caret, matching the native driver and the standard mobile-input contract, instead of deleting existing content first. Adds an injectable command runner so the behavior is testable without a device. * test(sidecar): assert InputText types at cursor without clearing Captures the adb command stream and verifies a single input-text call with no preceding delete keyevents, plus the adb escaping cases. * feat(proto): add LongPress RPC * chore(proto): regenerate Go stubs for LongPress * feat(driver): add LongPress to DeviceDriver interface * feat(sidecar): add LongPress client method * feat(mock): record LongPress action * feat(chrome): implement LongPress as press-and-hold * feat(sidecar): implement longPress across backends * feat(sidecar): dispatch LongPress RPC to backend * test(sidecar): cover LongPress dispatch * test(sidecar): implement longPress in snapshot test backend * feat(verifier): add LongPress and Scroll action kinds * feat(folio-spec): predicate that gates balance check on TxnSubmit tap Replaces the row-sum predicate (which always held by construction since balance is derived from rows in Folio) with one that compares the typed amount to the actual balance delta after a tap on TxnSubmit. Catches the planted double-submit bug. * feat(folio-spec): wire submitMovesBalanceByTypedAmount property Adds lastAction and totalBalance extractors and uses them in the new property. Drops ledgerRows/ledgerBalance extractors since nothing else referenced them. * feat(verifier): wire longPresses and scrolls generators * test(verifier): cover longPresses and scrolls generators * test(folio-spec): unit tests for submitChangesBalanceByTypedAmount Covers single vs double submit, the DoubleTap variant, vacuous cases (null action, wrong kind, wrong target, zero typed), and selector-as- object coercion. * feat(spec): add LongPress and Scroll authoring surface * feat(spec): no-op LongPress and Scroll in web runtime * feat(spec): re-export longPresses and scrolls as opt-in generators * test(spec): cover LongPress and Scroll runtime members * test(proto): expect LongPress in service descriptor * feat(runner): dispatch LongPress and Scroll actions * test(runner): cover LongPress and Scroll dispatch * docs(action-space): move LongPress, Scroll, DoubleTap to current actions * fix(runner): mark nil/empty hierarchy as transitional A failed or empty sidecar hierarchy fetch was pushed straight to the verifier, letting spec extractors crash with "Cannot read property 'map' of undefined" when findAll returned null. Treat that case like a transitional capture: skip the verifier push, still record the step, and keep the loop progressing. * fix(verifier): populate Action.On when tap chooser picks an element Coordinate-targeted Taps/DoubleTaps left On empty, so action-gated properties reading lastAction.on couldn't tell which target was hit and were vacuously skipped. Resolve the picked element to a stable key:value selector (resource-id, testTag, text, desc) and validate it resolves back to the same element so we don't accidentally redirect the tap to a sibling that shares the identifier. * fix(folio): add parseTypedAmount helper matching app's parseCents Raw user input like "50" must become 5000 cents, not 50. The existing parseDollarCents helper strips non-digits and so reads "50" as 50 cents, which is correct for formatted balance text but off by 100x for raw input from the amount field. * fix(folio): parse raw amount input as cents in submit predicate txnAmountField holds raw user keystrokes, not formatted balance text. Route it through parseTypedAmount so "50" reads as $50, matching how the app commits the transaction. * fix(folio): carry forward total balance across off-screen transitions AddTransactionScreen shows neither AccountCard nor LedgerBalance, so the extractor used to report 0 at the step before submit. That made every non-zero current balance look like the full delta and tripped the typed amount property on every honest submit. Remember the last-seen sum and return it whenever the current snapshot has no balance signal. * test(folio): cover submit predicate with raw typed-amount inputs Pipes realistic raw keystrokes through parseTypedAmount + the predicate so single submits clear and double submits fire as expected. * feat(folio): add computeHomeTotalBalance helper Pure helper that tracks Home multi-account total only and carries the last Home sum across off-Home steps. Ledger's single-account balance is excluded because mixing it would corrupt cross-screen scale comparisons. * fix(folio): totalBalance carrier tracks only Home, not Ledger Home cardSum is a multi-account total; Ledger's LedgerBalance is a single account on a different scale. Blending them in the carrier produced bogus cross-screen deltas (prev from Ledger, curr from Home), triggering false positives in submitMovesBalanceByTypedAmount. Restrict the carrier to Home AccountCard totals via the computeHomeTotalBalance helper. * test(spec): cover computeHomeTotalBalance carrier behaviour Tests Home sums, carrier passthrough on off-Home steps, the Ledger scale-mismatch case, and a Home > off-Home > Home sequence. * feat(runner): treat transient apply errors as transitional steps Sidecar input RPCs occasionally hang with DEADLINE_EXCEEDED or UNAVAILABLE on long fuzzing runs. The per-step loop previously propagated any applyAction error and killed the run after a single flake. Detect transient gRPC failures via status.FromError, mark the step transitional, skip the post-action idle poll, and continue to the next step. Fatal errors (outer ctx cancellation, non-transient codes, verifier crashes) still propagate. * test(runner): cover transient apply error resilience TestRunner_TransientApplyErrorMarksTransitional drives the runner through a wrapper that fails the first TapSelector with a gRPC DeadlineExceeded then succeeds. Asserts the run does not exit, the failed step is marked transitional with no violations, and the next step runs cleanly. TestIsTransientApplyError_Classification covers the helper's matching rules directly so future code changes don't quietly drop a transient case. * fix(folio): gate submit-balance property on Home route landing totalBalance is only freshly computed when AccountCards are visible on Home; off-Home landings return the carrier and would false-fire the property, latching always(next(F)) to false and masking the real double-submit bug. Skip vacuously when route is not "home". * test(spec): cover route gate in submit-balance predicate Adds route arg to existing cases (all use "home") and adds five new cases: ledger landing with stale carrier, add-transaction with double-insert delta, null route, plus home-landing positive and double-insert negative cases anchoring the gate's allow path.	2026-06-01 12:48:51 +05:30
pj	eed99e58aa	refactor: code organization cleanup (#35 ) * chore: fix gitignore + decisions doc after web->inspect-ui rename Update web/ references to inspect-ui/ in .gitignore and Makefile. Add decisions.md tracking architectural decisions from code-org discussion. * refactor: rename pkg/spec-api to pkg/spec Aligns the directory name with the npm package name @sanderling/spec. Updates Makefile, package.json directory field, and resolveSpecAPIPath. * refactor(verifier): split bindings.go into types.go + bindings.go Move shared public types (Action, ActionKind, LogEntry, Exception) to types.go. bindings.go retains internal JS runtime wiring only. * refactor(inspect): split runs.go into runs.go, runs_cache.go, runs_decode.go runs.go: types (RunSummary, StepSummary, RunDetail, Run) and Scan. runs_cache.go: Cache type, Open/Step/Detail methods, parseRun, scanSteps. runs_decode.go: readMeta, tallyTrace, decodeStepSummary, validRunID. * refactor: move android_env.go to internal/android/ Extracts Android device/AVD/adb logic into internal/android package. Exports EnsureDevice, AdbReverse, AdbReverseRemove, EnvWithAndroidPlatformTools, AdbBinary. Moves tests to internal/android/android_test.go. cmd/sanderling becomes a thin caller. * refactor: extract test pipeline to internal/testrun/ runTestPipeline logic moves to testrun.Execute. buildDriver, resolveSpecAPIPath, pickFreePort, and the progress logger move to internal/testrun/. cmd/sanderling/test_run.go becomes a thin adapter. Tests follow their code. * ci: update workflow paths after pkg/spec-api -> pkg/spec rename	2026-04-22 20:35:34 +07:00