Commit Graph
187 Commits
Author SHA1 Message Date
pj 06213e444e fix(runner): gate first observe on the app window being drawn, not just resumed 2026-05-31 14:33:09 +05:30
pj 3fe92537be feat(driver): add FocusedWindowChecker capability 2026-05-31 14:30:36 +05:30
pj 2fe01ea254 feat(android): detect focused-window package via dumpsys window 2026-05-31 14:30:10 +05:30
pj 05125b69f5 chore: stop tracking inspect-ui/dist build artifacts 2026-05-31 13:38:40 +05:30
pj d733218c40 test(hierarchy): cover package derivation from resource-id 2026-05-31 12:59:20 +05:30
pj 8401ae908f feat(hierarchy): derive package from resource-id prefix
The Android sidecar omits an explicit package attribute, so the verifier's package scope filter was a no-op and the keyboard still leaked into targets. Native nodes carry their package as the resource-id prefix; derive it there when the attribute is absent. Compose testTags are colon-less and stay empty, keeping them in scope.
2026-05-31 12:59:20 +05:30
pj 6d3561bf6c test(verifier): cover package-scoped target selection 2026-05-31 12:50:30 +05:30
pj 6d0f81ffa2 feat(testrun): pass app package into verifier scope filter 2026-05-31 12:50:30 +05:30
pj 15f3906f1c feat(verifier): scope random-action targets to app package
Random tap/doubleTap/type/swipe candidates now exclude nodes whose package differs from the app under test, so exploration never fuzzes the soft keyboard, system UI, or permission dialogs. An unset app package or an element with no package stays in scope, preserving behavior on iOS.
2026-05-31 12:50:26 +05:30
pj 16a9136b28 test(runner): cover startup foreground gate and back-press 2026-05-31 12:12:58 +05:30
pj f22e1bada8 feat(runner): gate first action on app reaching foreground 2026-05-31 12:12:58 +05:30
pj ddec95a2c5 feat(runner): re-fetch on transitional hierarchy capture
Some actions trigger async work (DB write, ViewModel coroutine) whose
navigation transition begins after the sidecar settle poll has already
exited. Without intervention, the next iteration's hierarchy fetch
lands mid cross-fade and the verifier observes a partial extractor
state which then surfaces as a false-positive violation at the step
where the transition completes.

fetchSyncedState pairs hierarchy + screenshot in one goroutine and
retries the pair (up to 4 times, 200ms apart) while the captured tree
contains more than one route-level *Screen tag. Steps that observe
no transition get no added cost; steps that catch a transition pay
up to ~600ms extra wall time but record a tree that matches the
post-transition state the property language expects to compare.
2026-05-31 12:11:29 +05:30
pj 8deef1a425 test(sidecar): cover streak reset and route-transition rejection
Verify the poll honors MIN_STABLE_STREAK_MILLIS, that a transient
mid-stream change resets the streak, that null returns block streak
progress through a NavHost cross-fade, and that stabilitySnapshot
counts only route-level attribute keys when summing Screen tags.
2026-05-31 12:11:20 +05:30
pj f32fbe540b feat(sidecar): streak-based settle with route-transition detection
Two changes layered into the stability poll:

1. stabilitySnapshot returns null while the tree carries more than one
   route-level Screen tag (resource-id / testTag / identifier ending
   in "Screen"), so the poll cannot declare a NavHost cross-fade
   stable. Apps following the Compose route convention get this
   detection for free; apps that don't fall through to the generic
   signal below.

2. pollUntilStable now requires an uninterrupted stable streak of at
   least MIN_STABLE_STREAK_MILLIS rather than just N consecutive
   matches. A late transition that fires after a brief calm window
   breaks the streak instead of slipping past. Interval widened to
   250ms so UiAutomation isn't hammered under fuzz load.
2026-05-31 12:11:14 +05:30
pj 0abbcd7d0c feat(cli): default --clear-data on so runs start fresh 2026-05-31 12:10:05 +05:30
pj a10791e00f fix(sidecar): cap stability poll independently of settle budget
The previous shape halved durationMillis between waitForAppToSettle
and the structural poll, then hammered hierarchy() at 80ms intervals
- on Maestro this stacked enough RPCs that hierarchy fetches began
timing out under load and the run stalled. Pass the full budget to
waitForAppToSettle and cap the follow-up structural poll at 600ms
with a 120ms interval, so the device sees at most a handful of
extra hierarchy reads per step.
2026-05-30 22:09:26 +05:30
pj d0cf5a98c7 feat(inspect-ui): render extractor-change breadcrumbs at violations
Show prev -> curr for each extractor whose value changed on the
selected step, anchored under the violation row in ActionList.
Long values collapse into <details> so the inline diff stays
readable while the full payload is one click away.
2026-05-30 21:58:13 +05:30
pj e44ad56749 feat(trace): emit extractor_changes per step
Add ExtractorChanges to trace.Step and a runner helper that converts
the verifier's diff map into the trace shape. The inspect UI keys
its violation breadcrumbs off this field.
2026-05-30 21:58:08 +05:30
pj c66c4e0ad4 test(verifier): cover ChangedExtractors diffs
Verify initial snapshot reports both named and fallback-named
extractors, a subsequent change surfaces prev/curr, and a no-op
snapshot leaves the diff empty.
2026-05-30 21:58:04 +05:30
pj 419a2d564e feat(verifier): track extractor value transitions
Cache each extractor's prior and current JSON-encoded value during
PushSnapshot; expose ChangedExtractors to surface per-step diffs the
runner can emit into the trace. The first observation flushes every
non-null extractor as a change so the inspect UI shows initial state
breadcrumbs alongside later transitions.
2026-05-30 21:57:59 +05:30
pj 07c292913c chore(folio): name every extract() call
Give each extractor in the Folio spec a debuggable label so the
inspect UI can render extractor-value diffs at violation steps
keyed by intent (ledgerRows, route, ledgerBalance, ...) rather
than by registration index.
2026-05-30 21:54:35 +05:30
pj 1b35e5a6a6 feat(verifier): name extractors for diff surfacing
bindExtract accepts an optional name argument; falls back to
extractor_N when omitted. The name is stored on extractorState
alongside prev/curr value caches that the next change will use to
emit per-step diffs.
2026-05-30 21:54:30 +05:30
pj cbef329cdd test(spec): cover extract name overload
Verify the runtime receives an undefined name in the legacy shape,
the supplied name in the (name, getter) shape, and that
extract("name") with no getter throws.
2026-05-30 21:54:25 +05:30
pj 6fdbf9d2ab feat(spec): accept optional name on extract()
Add an (name, getter) overload so each extractor handle carries a
debuggable label that future trace fields (per-step diffs) can key
off. The web-runtime falls back to extractor_\${index} when none is
supplied so existing call sites keep working unchanged.
2026-05-30 21:54:20 +05:30
pj 7291d5965b test(sidecar): cover pollUntilStable and structuralHash
Verify the poll returns on two equal snapshots, after transient
churn, and at the cap when never stable; assert the hash ignores
bounds-only flicker and detects content changes.
2026-05-30 21:50:52 +05:30
pj cd4bd5a194 feat(sidecar): structural-hash settle poll
Add pollUntilStable and structuralHash helpers; wire them into the
Stub, Maestro, and iOS backends' waitForIdle. The structural hash
ignores bounds-only flicker (measure passes) but trips on any change
in resource-id/class/content-desc/text, so a Compose cross-fade where
both source and destination composables are momentarily alive no
longer slips through Maestro's waitForAppToSettle and contaminates
the next hierarchy fetch.
2026-05-30 21:50:46 +05:30
pj 50e03244d6 refactor(inspect-ui): use next step's screenshot for state after
Each step now has one screenshot (the moment of observation). The
"state after" view of step N is the same moment as step (N+1)'s
observation, so reuse that file rather than expecting a separate
-after.png.
2026-05-30 21:48:26 +05:30
pj fb69c1b774 refactor(runner): one concurrent screenshot per step
Move screenshot capture into the post-action errgroup so it observes
the same UI moment as the hierarchy fetch. Drop the pre-action and
deferred -after captures. Skip WaitForIdle when the action is Wait
since the wait itself provides settling time.
2026-05-30 21:48:21 +05:30
pj 095b58afcf refactor(trace): drop WriteScreenshotAfter
Only one screenshot per step is captured now (concurrently with
hierarchy after settle), so the -after.png variant is unused.
2026-05-30 21:48:17 +05:30
pj f9512354ed refactor(folio): replace txn invariants with balanceMatchesAddedTxn
Collapse noDuplicateTxnPerStep and newTxnChangesBalance into a single
per-row property: every newly-appearing ledger row's signed amount must
match the ledger balance delta. A double-submit lands two rows whose
individual amounts cannot both equal the aggregate delta, so each row
fires the property, catching both the row-count and balance-math
classes of bug under one semantic invariant.
2026-05-30 21:44:41 +05:30
pj 352118c199 fix(verifier): canonicalize selector strings
Object/chain JS selectors used to fall through to goja's default
stringification, producing "[object Object]" tags that surfaced as
garbage in trace.action.selector. Emit canonical "k:v" / " > "-joined
strings instead so the tag round-trips back through the hierarchy
selector grammar.
2026-05-30 21:43:27 +05:30
pj 09c1b8df26 fix(folio): make login spec content-driven (idempotent across re-entries) 2026-05-30 17:04:11 +05:30
pj c69052ae89 style(verifier): use maps.Copy for verdict snapshot 2026-05-30 16:33:23 +05:30
pj 60c4ef7458 refactor(runner): emit onset-only violations to trace and summary
Switch the per-step violation list from the sticky verdict map to the
verifier's onset set. Each property now appears exactly once across a
run: at the step it first violates, not on every subsequent step where
the residual stays false. Removes the dead violationNames helper.
2026-05-30 16:32:17 +05:30
pj b9fa41553f feat(verifier): track newly-violated property set per step
Sticky `always(P)` violations re-surfaced on every step after onset,
flooding traces and summaries with duplicate records. EvaluateProperties
now diffs against the prior verdict map and records the onset set; a new
NewlyViolatedProperties accessor exposes it so callers can emit each
violation exactly once at its onset step. The verdict-map return is
preserved for residual / current-verdict consumers.
2026-05-30 16:30:37 +05:30
pj 3d67e5e3b6 fix(folio): make ledgerRowsSeen monotonic to suppress transient-render false positives 2026-05-30 16:01:16 +05:30
pj aa42b8e504 refactor(folio): drop doubleSubmitTxn; fuzzer surfaces double-submit via defaultActions 2026-05-30 16:00:14 +05:30
pj 801f09d455 feat(verifier): add doubleTaps random-target generator 2026-05-30 16:00:09 +05:30
pj 177b571b0a feat(spec): add doubleTaps random-target builtin to defaultActions 2026-05-30 16:00:06 +05:30
pj f08d9119b2 fix(folio): track ledger row count across non-ledger steps; pin reproducer seed 2026-05-30 11:44:43 +05:30
pj 9f17afecc2 feat(folio): add noDuplicateTxnPerStep invariant and doubleSubmitTxn action 2026-05-30 11:07:31 +05:30
pj 87834caf96 test(doubleTap): cover constructor, verifier round-trip, and runner dispatch 2026-05-30 11:05:58 +05:30
pj fb085eeaa0 feat(runner): dispatch DoubleTap as two taps inside one step 2026-05-30 11:05:53 +05:30
pj 6766f1ad58 feat(verifier): bind doubleTap and decode DoubleTap actions 2026-05-30 11:05:50 +05:30
pj 237ead2d33 feat(spec): wire DoubleTap through web-runtime serializer 2026-05-30 11:05:47 +05:30
pj 7006e82ca4 feat(spec): add DoubleTap action type and constructor 2026-05-30 11:05:44 +05:30
pj a9b2b4b4e1 fix(spec): drop hardware back from defaultActions to stay in-app 2026-05-28 11:24:31 +05:30
pj dff91095b8 feat(runner): relaunch app when foreground escapes during exploration 2026-05-28 11:23:50 +05:30
pj dea39d3bac feat(sidecar): implement ForegroundApp via adb for android 2026-05-28 11:21:52 +05:30
pj 09528d84e6 feat(android): detect foreground package via adb dumpsys 2026-05-28 11:21:14 +05:30