mirror of
https://github.com/priyanshujain/sanderling.git
synced 2026-10-02 19:17:10 +00:00
6e85cac8b3d9b708f473fb661458c7a0c38bc011
20
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
6e85cac8b3 |
merge origin/master into llm-recording-and-analysis
both sides independently fixed the same three bugs, so each one had to pick a winner rather than keep both implementations. extractor encoding: master's recordableValue in worker.go wins over ours in marshal.go, since master's is pinned by extractor_encoding_test.go and ours had no tests. our error semantics stay: encodeExtractorValue still returns an error instead of nil, so an extractor cannot vanish from the trace silently. apply errors: only the residual generic branch takes master's unconfirmed copy, where the device may have committed the action before the call failed. the finer branches that know nothing was dispatched keep lastAction = nil, and our actionSkipReason taxonomy stays alongside master's held/skippedVerification. selector matching: our matchAttr with matchSelectorKind wins over master's match, since ours also handles idPrefix. matchSelector now calls it, which git did not flag as a conflict and left calling a function our side had deleted. the ltl doc comment takes master's correction: an unbounded eventually that never fires IS violated at run end. |
||
|
|
de654334b0 |
feat(hierarchy): store the tree shape and tolerate an unreadable boolean flag
a Tree marshalled to json kept only the flat element array, so a stored tree decoded with a nil Root and resolved no selector. it now stores each element's pre-order depth and rebuilds Root from it, re-seating elements so Tree.Elements and &node.Element stay one pointer. a stored tree without depths keeps the old shape. a boolean field the producer sent as something other than a boolean now leaves the flag unset and increments UnreadableFlags rather than failing the whole dump. |
||
|
|
ca667b1b77 |
fix(selectors): resolve text to the innermost match and scan the root in both forms
an element's text is its whole subtree's text on web and on ios, so every ancestor of a matching element matched too, up to the root. a match a descendant also makes is now dropped, in internal/hierarchy, in the chrome xpath translation and in the page-side web runtime, so all three resolvers name the same element. a raw attribute now matches on a substring (exact for true/false) the way the docs describe, and tree-level FindBySelector considers the root, so ax.find("id:page") and ax.find({id: "page"}) agree.
|
||
|
|
1b01bdbaf1 |
feat(ios): derive scrollable from the snapshot's tree depth
the companion now emits each node's depth, so the hierarchy mapper can find the containers that clip content reaching past their own frame and mark them scrollable:true, the same fact android reads off uiautomator and the web driver derives from overflow. a dump without depth makes every element a root and roots are never marked, so the legacy bridge reports no scroll rather than a guessed one. |
||
|
|
9fb121e9d0 |
correctness fixes from the first real folio dispatch, and spec authoring skills (#82)
* docs: add the apache 2.0 license text
package.json has declared Apache-2.0 since the first release and .goreleaser.yaml
globs LICENSE* into the archives, so that glob has been matching nothing. npm
only picks up a license from the package directory, hence the copy under
pkg/spec.
* fix(sidecar): close the soft keyboard after typing on android
* test(sidecar): pin the guarded ime dismissal
* fix(sidecar): treat a failed ime probe as no keyboard open
* ci(folio): let the ios leg clear state for itself
* ci(folio): drop the stale frontboard note from the ios job
* docs(ci): record what the ios calibration assumes and where it was measured
* fix(ios): replace the session when a launch blows its bound
a launch the simulator refuses is never reported: xctest records it as a test
failure the runner cannot see, then holds the session's main thread for about
four minutes on a diagnostic chain. so the only signal is the expired bound,
and every later call queues behind the same wedge. restart the session once and
launch again, bounded so the launch path stays inside testrun's backstop.
* test(ios): cover the session replacement a wedged launch needs
* fix(ios): share one deadline across the restart and the second launch
the recovery a blown bound triggers now costs at most launchRecoveryTimeout
whatever it spends it on, so the launch path tops out at 150s and testrun's
three minute backstop stays a backstop.
* test(ios): the restart a blown launch triggers has to be bounded
* docs(ci): the ios leg does not convict on the runner, and a seed cannot fix it
seed 28 reproduced its walk on macos-15 and reached the bug at the step it
convicts at locally. it still could not be judged: the run did not return
home between step 19 and step 136, so the counting invariant saw a rise of
15 against a window of 37 submits.
* docs(sidecar): record why the stale ime flag stays out of reach
* test(folio): add commonTest source sets to core and shared
* fix(folio): reject amounts parseCents cannot represent
* fix(folio): cap a transaction at one million dollars
* test(spec): give the fake dom a real tree and a walking querySelectorAll
* style(sidecar): make ktlint clean, formatting only
ktlint -F over every kotlin file except DriverBackend.kt, then hand
fixes where the reflow read worse and for the long lines ktlint cannot
break. No behaviour changes.
DriverBackend.kt is left untouched to avoid a conflict with concurrent
work; its three over-long lines still fail fmt-kotlin.
* fix(spec): deepQueryAll returns matches in document order
* ci(folio): say what the android gate found, not why
The gate proves only that AddTransactionScreen is absent from the trace.
Claiming the run never got past login was an inference it cannot make: the
run that produced it had logged in and was stuck on the new account screen.
Name the routes the trace does record instead.
* test(runner): a relaunch must not convict the submit counting property
* test(browser): compare ax.find across both hosts on one page
* test(spec): name the shadow match in the grammar both hosts parse
* ci: move every action off the node20 runtime
checkout v4->v7, setup-go v5->v7, setup-node v4->v7, setup-java v4->v5,
upload-artifact v4->v7, cache v4->v6, upload-pages-artifact v3->v5,
deploy-pages v4->v5, setup-chrome v1->v2, setup-android v3->v4,
goreleaser-action v6->v7. setup-bun and android-emulator-runner are
already node24; buf-setup-action stays on its deliberate SHA pin.
setup-chrome v2 resolves stable from Chrome for Testing rather than the
official installer, so the ci.yml comment about the action's default no
longer held.
* feat(verifier): report a relaunch on state.lastAction
* test(verifier): pin the relaunch field on both hosts
* fix(runner): keep the action the app was relaunched after
* test: pin that a nested undefined does not survive the wire
* fix(folio): uninstall before installing in just ios
folio's signed-in session lives in the data container, which an install
over the top keeps, so a local run started right after just ios opened on
the previous run's Home screen and diverged at step 1. On CI's fresh
simulator the uninstall is a no-op, so the ios leg is unchanged.
* style(sidecar): bring the last three lines under the line limit
* fix(ios): the runner must not answer ok for a launch that failed
XCTest records a refused launch as a test issue that never throws, so the
companion returned ok for an app that never started. Check the state the
app actually reached and report the refusal instead.
* test(ios): a refusal the runner names costs no session restart
The session restart is for a launch that never answers. A launch that
reports the app's state has already said what a fresh session would.
* fix(folio): attribute a created account by its whole key, not a suffix
createdAccountHasNonZeroBalance matched the created card with endsWith, so
an older account whose name ends with the typed one ("Emergency Fund" for a
typed "Fund") was judged instead whenever the new card was clipped out of
the reading. Build both keys the card can carry, the plain name and web's
initials + name, and compare them whole.
* fix(hierarchy): object selectors resolve by the same rule as string ones
* test(verifier): both ax.find selector forms resolve the same element
* test(browser): the cross-host fixture uses the object selector form
* fix(replay-ui): wrap the tab strip so its last tabs stay clickable
* test(replay-ui): drive the fuzzer onto a violating step with a panel
* feat(folio): judge a submit against the account's own balance
The counting invariant can only close its window on Home, and the iOS run
in #78 went 117 steps between two Home readings: 37 submits against a rise
of 15 transactions is no evidence about the double tap sitting inside it.
The ledger and the add-transaction screen both show the account's own
balance, and an accepted submit pops back to the ledger, so a window
bounded by those readings holds one action.
The bound is an upper one: a balance that has not moved is a commit still
in flight, a rejected submit or a tap that never landed, and none of those
is a violation. Moving by more than the one submit in the window typed is.
* test(runner): an overlay dismissal must not convict the counting property
* docs(runner): point the guard comment at the renamed test
* docs(replay-ui): name the viewport the tab overflow was measured at
* feat(folio): close the submit window on the account's own screens
submitCommitsOneTransactionPerAction now states its rule over two windows:
the Home counts it already compared, and the account balance the ledger and
the add-transaction screen redraw on nearly every frame of the transaction
flow. Same rule, and the second window is usually one action wide.
* fix(sidecar): reach the adb server the environment names
buildDadb hardcoded localhost:5037, so a serial-addressed device always
resolved through this machine's adb server and ADB_SERVER_SOCKET was
ignored. Read the endpoint the way the adb CLI does instead.
Fixes #79
* test(sidecar): pin the adb server endpoint parsing
* fix(sidecar): close a keyboard standing in the snapshot
A tap on a text field raises the keyboard and nothing closed it, so the
tree the picker chooses from was missing every app node underneath it,
the submit control included. Close it before the read rather than after
the tap: the picker only ever sees snapshots, and the keyboard is still
on its way up when the tap returns.
Fixes #78
* test(sidecar): pin the tree-guarded keyboard dismissal
* chore(make): a target that runs folio's unit tests
* chore(folio): a just recipe for the unit tests
* ci: run folio's unit tests on every pr
* ci: switch to jdk 21 only for the folio step
* docs(sidecar): put the measured read cost in the dismissal bound
* fix(folio): decline the two demanding properties across a relaunch
The runner now keeps lastAction and marks it relaunched: true where it used
to report nothing at all, so the two properties that demand an effect judge
a step whose process may have died before the write landed.
submitChangesBalanceByTypedAmount and createdAccountHasNonZeroBalance both
decline there. The counting bound does not: a relaunch cannot manufacture a
transaction, and the submit is counted, so declining would throw away the
detection the runner fix restored.
* docs(folio): say why the merged card key cannot be made injective
Folio rejects a duplicate account name, so the twin the drop rule guards
against is two names the web key cannot tell apart, not two accounts
sharing a name. State what closing the rest would cost and what the tree
would have to carry to close it properly.
* test(folio): pin that two accounts can render the same card text
The proof behind the comment: "Travel1" holding 25 transactions and
"Travel12" holding 5 merge to the same string, so no identity key read off
a web card can tell them apart.
* fix(sidecar): bound the diagnostic adb reads
adbOutput and readLogcat read to EOF and then waited with no timeout, so
a wedged adb held the step for as long as it liked; one stall over a
remote adb server measured ~100s. The bound has to sit on the read, not
on waitFor: a wedged adb never reaches EOF, so a bounded waitFor after
the read is a line that never runs.
* test(sidecar): pin the bound on a wedged adb read
* fix(sidecar): an unreadable animation count is not idle
Defaulting the count to zero made a dumpsys that said nothing mean
nothing is animating, so a degraded link broke out of the settle early
and handed the runner a frame caught mid-animation. Unknown now waits,
inside the deadline waitForIdle already holds.
* test(sidecar): unknown animation state must not read as idle
* fix(folio): stop spending the submit budget on taps the app refused
The window is an upper bound on the transactions an interval could hold, and
a bound inflated by taps that commit nothing is a bound the app can never
exceed: #78 read a rise of 15 transactions against 37 submits. TxnSubmit is
clickable(enabled = amount.isNotBlank()) and parseCents refuses anything its
regex misses, so a tap whose landing frame shows a refused amount cannot have
committed. Over four recorded android runs that is 19, 11, 25 and 25 of 35,
26, 42 and 42 submit taps.
A relaunch is excepted: a fresh process draws an empty field whatever was
submitted.
* feat(folio): read the amount field into every submit window
Each of the three windows asks whether the tap could have committed, off the
field as the landing frame shows it.
* fix(sidecar): a foreground read that fails degrades the typing guard
An unreadable dumpsys passed a null owner to typeChunks, which switches
the mid-type focus guard off outright and lets the rest of the string
spray into whatever holds the foreground. Fall back to the launched
bundle instead: the guard stays armed, typing still happens, and the
degradation is said out loud rather than assumed away.
* test(sidecar): pin the degraded typing guard both ways
* test(runner): answer Snapshot and Hierarchy off one tree in the fakes
* feat(runner): skip a step whose tree changed between two reads
* test(runner): cover the reread's cost to the existing snapshot rules
* fix(ios): clear app state before the automation session attaches
New performs the clear-state reset, so the uninstall and reinstall no
longer land underneath a live XCTest session that is already bound to
the app. Launch refuses a clear-state request the driver was not built
for rather than reinstalling under its own session.
* fix(ios): the device path clears before its runner session too
* fix(testrun): thread clear-data into the ios drivers
* test(runner): a skipped step must not swallow the action before it
* fix(runner): hold the action back on a step nothing verified
* refactor(runner): drop the empty branch from the hold path
* docs(runner): describe both modes of the composing test driver
* fix(sidecar): erase a field by selecting it, not one delete per character
maestro's eraseText sends one delete per character through its
instrumentation, measured 29.6 ms/char on the API 34 emulator. The
4096-character string the corpus types cost ~121s to clear, a fifth of a
20 minute run spent on one step, and it recurred every time that field
was typed into again.
Select the content and delete the selection instead: two key events at
any length, measured 0.15s to 1.16s for 4096 characters across API 34,
35 and 36. The result is read back off the tree, and a field that is not
empty, or that the tree cannot report on, is finished off per character
in batches rather than assumed clear.
Fixes #80
* test(sidecar): pin the constant-cost erase and its residue check
* fix(sidecar): find the erased field by class, past the keyboard's own focus
The check that decides whether the select-all worked looked for an
"editable" attribute maestro's tree does not carry, so it answered
"cannot tell" every time and every erase paid the per-character
fallback. Worse, an open keyboard puts a second focused node in the
tree, one of the IME's own keys, carrying no text: taking the first
focused node would read a field still holding 4096 characters as empty,
which is the one answer that stops the erase early.
Match the text field by class instead. Measured against the real
backend, 4096 characters now clear in 385ms on API 34, 409ms on API 35
and 870ms on API 36, verified empty, where the fallback took ~4s.
* test(sidecar): use the tree the device really returns
* docs(ci): the android step number describes a local emulator, not ci
the leg disables animations and the number was measured with them on. the
first real dispatch carries 4 transitional steps over 200, so the cross-fade
wait does still fire in ci, just far less often.
* fix(android): say what the sdk lookup checked, not just to set ANDROID_HOME
* fix(doctor): resolve adb and emulator the way a run does
* docs(cli): the android doctor checks are not path-only
* fix(testrun): preflight resolves adb through the sdk, not just PATH
* docs(skills): add a spec review skill and the skills index
* fix(testrun): report a sidecar that dies at startup as the exit it was
* test(testrun): cover the sidecar shutdown path after an early exit
* docs(skills): add a property patterns catalogue skill
* fix(folio): bound the total-balance move instead of demanding it exactly
The write finishes before AddTransactionViewModel navigates, but nothing
establishes that Home's total has re-rendered before the frame is read, and
an equality convicts a healthy app for a total one frame behind. A delta of
zero is exactly the shape nine of the eleven measured android false
convictions had. 2x still exceeds x, so all four recorded convictions
survive, checked against the traces.
The trade is real: a balance that moves by LESS than the amount typed is no
longer judged anywhere in this spec.
* docs(folio): say what property 2 demands now that it is a bound
* docs(skills): add a spec authoring skill
covers hooks, extractors, selectors, properties, actions and the order to write them in, with a complete sample spec that typechecks against the real export surface.
* docs(skills): ground the property patterns catalogue in the merged specs
* docs(skills): name the selector keys that still substring match
* docs(skills): add a setup skill for adopting sanderling
* docs(skills): add a run triage skill
* docs(skills): point the setup skill at its siblings
* docs(manual): correct the flags the cli reference gets wrong
--launcher-activity does not exist in cmd/sanderling/main.go. --device,
--android-app-path and --arm do and were undocumented. runs.md still listed
--max-steps and --exit-on-violation as unshipped, and described --clear-data
as opt-in when the default is already true, contradicting itself ten lines on.
* test(folio): pin that a commit stays in the window until Home reads it
The interaction that keeps a stale Home card list from ever banking counts
the budget has already forgotten: a submit lands on the ledger, so the
reading that resets the window is a whole action later and the submit is
still in it. Characterization, not a regression: no code changed and it
cannot go red first.
* docs(folio): record why a banked card reading can be trusted as current
The freshness rule rests on the app popping one entry back to the ledger,
not on anything the frame carries, so the assumption and the measurements
behind it belong next to it.
* refactor(folio): name the balance property for the bound it asserts
it stopped being an equality and became |delta| <= typed, so the old name
demanded more than the property does. renamed with the ci gate's
GATED_PROPERTIES in the same commit so the gate never sees a name it does
not know.
* fix(android): a refused uninstall must not pass for clear-state
adb uninstall answers Failure [DELETE_FAILED_INTERNAL_ERROR] both when the package was never installed and when it refuses to remove one, so the failure text cannot say which happened and the old code installed over the top either way, keeping the data clear-state was asked to drop. Ask pm path instead, and fall back to pm clear when the app is still there.
* fix(ios): a failed simctl uninstall must fail the reinstall
simctl install over an installed app carries its data container across, so discarding the uninstall error reported a clear-state that never happened. Uninstalling an app that is not installed exits 0 on a booted simulator, so every failure here is a real one.
* fix(ios): a failed devicectl uninstall must fail the reinstall
same hole as the simulator path: devicectl install over an app keeps its data, and the discarded uninstall error hid it. Uninstalling a bundle id that is not installed exits 0 with 'App uninstalled.' on a paired iPhone, so a failure here is always real.
* docs(android): say why the uninstall text cannot be read
* test(android): name the uninstall failure for what it says, not why
* docs(ci): the ios leg convicts on the runner now, and why it did not before
* docs(ci): the cross-fade wait does not fire on ci, say so
* fix(runner): a bounded hold puts the swallow back one step later
the hold carries one action; letting the runner act again while the verifier
is still skipped overwrites it, so the carried action reaches no spec. hold
for as long as the verifier is skipped, and settle on a held step so the
reread pair is not tighter than the window the detector was measured over.
* test(runner): pin what the two reads are compared on
structuralShape excluding text and bounds is the decision separating this
feature from a run that verifies nothing, and only prose held it. adding
either field back now turns a case red.
* fix(ci): close shell injection into the npm publish job
A refname is attacker-controlled and git permits backtick, $, (, ; and |
in it. Three sites substituted it into a run: block, and NODE_AUTH_TOKEN
sat at job level, so a pushed tag ran arbitrary commands with the publish
credential in reach.
The tag now goes through env:, is validated against an anchored version
pattern before anything consumes it, and reaches the other jobs as a job
output. The token is scoped to the publish step. release-npm declares
contents: read instead of inheriting the repo default.
* fix(ios): recognise every shape a blown launch bound arrives in
The runner transport reports a blown budget two ways, its own comment says
so: the context's error once cancellation has landed, and the connection's
i/o timeout when the deadline armed from that context fires first. The
legacy transport reports it as a gRPC status. errors.Is against
context.DeadlineExceeded only matches the first, so the session restart
never fired for the other two and a wedged session stayed wedged.
* test(ios): drive the launch recovery with what the transports return
The wedged-session fake answered with ctx.Err() raw, which is the one
shape the guard already matched. The recovery now runs against the error
each transport really produces for the same expiry, taken from a runner
and a legacy companion that never answer.
* test(runner): pin both guard writes to what the spec reads
deleting lastAction.Relaunched or lastAction.Applied left the whole suite
green, so the only producer of the two fields every spec-side guard reads
had nothing holding it. both now assert the value out of the trace.
* fix(testrun): a run that judged nothing is not a green run
every step skipped means no property ever evaluated, so no violations is the
absence of a verdict rather than a clean one. the hold makes that reachable
now, so the run says it instead of exiting 0.
* fix(ios): stop the app before clearing its state
Launch terminated and then cleared; the clear moved to construction and
left nothing stopping the app first. The container wipe deletes files a
live app still holds open, and the CI ios leg passes no app path so the
wipe is the path it takes. simctl stops it, since the clear now runs
before any automation session exists. On a device the uninstall that is
its only clear takes the running app with it.
* test(ios): pin the stop that has to precede a clear
The ordering probe now records the stop, and a scripted xcrun holds what
reaches the tool: terminate before get_app_container, with the previous
run's files gone after. A simctl terminate that finds nothing to stop
still leaves the clear a success.
* docs(spec): an unbounded eventually is violated at run end
* docs(skills): an unreached eventually convicts at run end
* docs(skills): noUncaughtExceptions only fires on web
* fix(ios): a device clear-state that cannot happen must fail
--clear-data on a physical device with no --ios-app-path warned and then
ran anyway, so the run started on the previous run's data while the flag
said it started clean. There is no data-container wipe on a device, so
there is nothing to fall back to.
* test(ios): a device clear-state without an app path ends the run
* docs(skills): the stock properties each cover one platform
* docs(ci): the balance property demands a bound, not an equality
* fix(ios): the clear-state guard checks the bundle that was cleared
A bool only said that something was cleared, so Launch(ctx, otherBundle,
clearState=true) passed the guard and reported a reset that had reached a
different app. Record what was cleared and compare against the bundle
being launched.
* test(ios): a clear-state launch for an uncleared bundle is refused
* docs(manual): the flagship property is a bound, and say what that costs
* fix(ios): one address picker for every bring-up
bringUpRunner reads the picker from a field, and NewDevice only ever set
the device one, so a device driver that reached bringUpRunner would call
nil. The two fields held the same function; keeping one leaves no path
that can be wired without it.
* test(ios): a device driver can bring a runner up
* docs(skills): both shipped balance forms are bounds now
* docs(skills): name the balance predicate that still exists
* docs(skills): quote the doctor the binary actually prints
* docs(skills): screen= is the chrome driver's url, web only
* docs(skills): substring selector matching is native only
* docs(skills): web selectors are exact, native ones are substrings
* fix(ci): a run that wrote no trace is not evidence about folio
run_dir is empty when the run produced no output directory, and the
fallback made trace ./trace.jsonl. A stray trace in the working directory
was then read as this run's, so a run that wrote nothing reported 'found
the submit bug' and exited 0, defeating the missing-trace check below it.
* fix(ci): fail folio when a gated property is not in the spec
Nothing tied GATED_PROPERTIES to the spec it gates. Renaming a property
left the classifier matching nothing: ios and web blamed the spec for
finding a different bug, and android silently reclassified a real
conviction as 'judging health only' and stayed green.
replay-ui-summary.sh already makes this check for its own list. The spec
path becomes SPEC-overridable the same way, so the check is testable.
* test(ci): cover the folio classifier's verdicts
21 cases through a stubbed sanderling: every exit path, the drift check,
a missing trace, a zero-byte trace, an empty glob and a truncated line.
Asserts the flags that reached the binary, not just the exit code.
Invoked as bash -eo pipefail -c, which is what a run: block does. Running
folio-run.sh itself under -e would kill it at the first non-zero
sanderling test, which is the exit code it exists to read.
* docs(skills): defaultActions bundles five of the eight generators
* test(ios): name the picker test for what it covers
* docs(skills): three of the replay-ui properties are cross-panel
* docs(manual): state.exceptions is web only and reportError does not exist
* fix(folio): the bound carries no unconfirmed-submit guard
deleting confirmedApplied here broke 0 of 355 tests: under a bound a submit
that may not have landed moves the balance by 0, which the bound already
permits, so the guard could only ever drop the double commit it exists to
catch. the relaunch guard stays for a reason the bound does not cover, and
both tests now assert a verdict that changes when their guard does.
* docs(ci): three of the replay-ui properties are cross-panel
* docs(manual): the starter property only fires on web
* test(folio): judge the conjunct on the landings a real run produces
three of the 18 frames the recorded ios run drove it down, each with the
second commit the bound is there to catch. neutering the comparison reddens
it: a second commit on 357900 went unjudged.
* test(folio): the walk drives the composition the spec runs
countSubmitsInWindow never saw an amountText here, so every walk test counted
submits the app must have refused. with the field passed, a refused submit no
longer buys a later double tap an alibi: without it the window reads 3, not 1.
* docs(folio): say which double submit the conjunct can see, and which it cannot
the home landing is the counting invariant's, three of three in the recorded
ios run; this one gets the interleaving whose second pop is cancelled. it is
still the only judge on the 18 ledger landings that run produced.
* docs(folio): the narrow window is not where the detection comes from
the double taps land on home, so the counting form convicts them; what turned
0 convictions into 4 on the recorded ios run is submitCouldCommit, which drops
the windows at those three steps from 5/4/7 to 2/1/2.
* docs(skills): folio drives three platforms from one spec
* fix(folio): an amount over the app's cap spends no window budget
the corpus reaches TxnSubmit with 999999999999999999999, AMOUNT_REGEX takes it
and AddTransactionViewModel refuses it against MAX_TRANSACTION_AMOUNT_CENTS, so
counting it was budget a double submit could hide behind.
* docs(folio): say which form judged one step, not which node was read once
* docs: a bound still needs the relaunch guard, and eventually does convict
* fix(sidecar): the hierarchy rpc serves the tree the snapshot reads
the runner compares the two per step, but snapshot settles and closes a
keyboard while hierarchy was a bare contentDescriptor. measured on emulator
-5556 (api 34) with an ime open: 489 nodes against the snapshot's 134. both
now come off snapshotTree under the same lock; the reread still costs ~75ms
when no keyboard is up.
* docs(runner): say what makes the two reads comparable
the reread's comment claimed the round trip was the only interval between
them; what it left out is that the two rpcs have to read the same way, which
the repo's own android backend did not do.
* refactor(runner): name the settle predicate for what it means
* ci: add a headless-chrome composite action
The setup-chrome / apparmor sysctl / launch-check trio is copied across
three jobs. The old comment described setup-chrome v1 semantics: under v2
stable is the default and the alternative is Chrome for Testing latest,
not a dev Chromium, so it is restated for what the pin actually does.
* ci(examples): add the folio setup actions
folio-app holds the per-platform toolchain and app build, so a caller
guards one step instead of eight. folio-simulator boots the simulator,
installs folio and leaves the app stopped.
* ci(examples): add the replay-ui fixture action
Records a trace and serves it with sanderling replay. The step page URL
is a composite output rather than GITHUB_ENV, so it is scoped to the one
step that drives it.
* ci(examples): one dispatch workflow for every example
folio.yml and replay-ui.yml ran the same operation: build sanderling for
a platform, bring a target up, run a spec against it, classify the trace,
upload the run. They are now one matrix over four examples, each naming
its own runner.
The job is named for what it fuzzes. 'dogfood' named why we run it, not
what runs, the same error as a diagnostic that reports a motivation
instead of an observation.
The matrix is computed by a plan job because jobs.<id>.if cannot read the
matrix context, so a static matrix has no way to leave a leg out. Seeds,
budgets, timeouts, runners and artifact names are unchanged.
* ci: reuse the headless-chrome action in the browser job
Same three steps the examples workflow needs, and the comment explaining
the AppArmor sysctl now lives in one place.
* ci: move the folio jdk step to setup-java v5
The only setup-java left on v4; every other one moved.
* ci: pin third-party actions to commit shas
buf-setup-action was already pinned with a comment saying why; the other
five rode mutable major tags, so a tag move is an unreviewed change to
what runs. Each major currently resolves to the release named in the
comment, so this freezes today's behaviour rather than changing it.
actions/* stay on major tags: they are first-party to the runner.
* ci(replay-ui): name the run directory for what it fuzzes
runs/dogfood and the '### replay-ui dogfood' heading carried the same
naming error as the job name: dogfooding is why the run exists, not what
it fuzzes.
* docs(driver): state the log level scale on LogEntry
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* docs(sidecar): name the device the node counts came off
* fix(chrome): keep a log entry the level scale cannot rank
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* fix(chrome): record console levels on the logcat scale
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* docs(ci): say why upload-pages-artifact needs no include-hidden-files
v3 to v5 crossed v4's change to exclude dot-files. build/site has none,
so nothing was dropped, and the underscore directory is not hidden.
* test(browser): drive a console error through to the spec
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* docs(ioscompanion): name the vacuity behind the empty log slice
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* ci: run the folio classifier's test in make test-ci-scripts
* fix(chrome): keep the message of an object console argument
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* test(browser): cover console.error with an error object
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* feat(spec): let the runner install state.logs in the page
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* feat(verifier): encode state.logs for the web host
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* feat(chrome): install the step's logs in the page
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* test(ci): pin three real folio traces from run 31902501859
ios convicted on submitCommitsOneTransactionPerAction, web on both gated
properties, android ran its full 200 steps healthy. Every step is kept;
of each step only step, violations, witnesses and residuals survive.
hierarchy is replaced by the quoted "...Screen" resource ids it held, in
order. It cannot just be dropped: it is 95% of the bytes and also the
only place the android route gate's grep can match, so dropping it flips
that leg from healthy to 'never reached'. 8.4MB to 59KB.
* test(ci): drive the classifier over the real traces
Four cases on real data: each leg's real verdict, plus the android trace
cut before it reached the transaction screen, which is what proves the
route gate reads a real hierarchy dump.
Also corrects the hand-written fixtures. They set is_error to false on a
plain violation; internal/trace/writer.go tags that field omitempty, so a
real trace omits it entirely. Harmless to the classifier, but a fixture
that does not look like reality is the thing that hides drift.
* test(runner): teach the web fakes to take the step's logs
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* fix(runner): install the step's logs before the page extracts
On web every extractor reading is replaced by the one the page computed,
and the page answered logs: [], so noLogcatErrors counted an empty array
however full of errors the console was.
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* test(runner): cover the logs reaching the page and failing to
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* test(spec): cover the host pushing state.logs into the page
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* test(browser): drive console.error through to a fired property
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* fix(runner): report a log fetch the driver could not make
The comment claimed the failure was warned about; nothing warned, so a
device whose log fetch failed every step held noLogcatErrors on evidence
nobody collected.
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* test(runner): cover the silently dropped log fetch
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* fix(ci): read the route the spec reports, not the hierarchy dump
The android health gate grepped the trace for "AddTransactionScreen",
which occurs in exactly one place: the hierarchy dump, as a resource-id.
That is a debug artifact standing in for a fact the spec already reports,
and it was wrong in both directions. Against the real 8.4MB trace with
hierarchy stripped, the old gate failed a healthy 200-step run; against a
trace carrying the marker on a transition frame without the route ever
being reported, it passed and called it healthy.
It reads extractor_changes.route now, whose values come from SCREENS in
the spec, so the gate and the app agree on what being on a screen means.
routeOf answers null on a frame showing two screens, which is exactly the
frame the marker was matching.
The drift check grows to cover both new names: extract("route") and the
SCREENS key. The fixtures are re-derived keeping the route entry and no
hierarchy at all; the full artifacts and the fixtures give byte-identical
verdicts, which is what proves the coupling is gone.
Script and fixtures move together: either alone leaves the suite red.
* ci: check that the workflow references resolve
actionlint reads a local action's inputs but never checks its path
exists: uses: ./.github/actions/typo lints clean and fails only when the
job runs. Covers composite action paths, make targets including the ones
the examples matrix builds from $SANDERLING, and the scripts a run: step
invokes plus their executable bit.
Fails when it parses fewer references out of a file than that file
mentions, because a checker that matches nothing reports a safety it
never looked for.
* ci: lint the workflows on every pr
The workflows that fuzz the examples are dispatch-only, and GitHub will
not dispatch a workflow that is not on the default branch, so their first
real run is after merge. actionlint and the reference checker are the
only things that can fail before that.
actionlint is pinned by commit, and its tool version is pinned too so a
new release cannot change what CI enforces.
* ci: collapse the four workflows into one
Nine jobs written out one by one, each with its own steps and its own
calibrated seed and budget as literals. Triggers are pull requests, master
and v* tags, and a dispatch with no inputs.
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* ci: inline the two composite actions with one caller each
Both existed to give the matrix a per-target hook. folio-app and
headless-chrome stay: three and three callers.
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* ci: check that no run: block interpolates an expression
A ${{ }} lands in the script text before bash reads the line, and
actionlint only flags the contexts it already knows are attacker
controlled. Nothing enforced the rule the workflow follows. Also drops the
matrix table lookup, which has no table to read now.
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* ci(folio): name the run, not the fuzzer, in the clean-run message
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* docs: point at the workflow that holds the release secrets now
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* ci: name every job Category (variant), and gate the lot on one check
Follows the convention in antithesishq/bombadil: the display name is what
groups a run in the Actions UI, so Check (tests), Check (browser),
Check (workflows), Folio (android), Folio (ios), Folio (web), Replay UI,
Release and Docs. Every job carries a name, so none of them falls back to
its kebab-case id.
All checks passed needs all nine and runs with if: always(), so branch
protection has one check to point at and a skipped job cannot read as a
pass. Release and docs now gate on startsWith(github.ref, 'refs/tags/v')
alongside master, which is the form the trigger filter already uses.
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* ci: split the release job back in two
Collapsing them left the npm publish steps in a job holding contents:
write, because GoReleaser needs it, so npm ci ran its dependency lifecycle
scripts with a write-capable GITHUB_TOKEN in reach of the same job as a
live NPM_TOKEN. Release (npm) is back on contents: read and Release (cli)
keeps contents: write, which is what they each had before.
Each validates the tag from its own copy of the pattern rather than
waiting on a job that exists only to pass a string. Release (cli) is tags
only: there is no CLI to cut on a merge.
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* ci: run folio on pull requests
folio was skipped on pull requests, so ios, android and web only ever ran
after a merge. The three legs are 3 to 19 minutes and run in parallel, and
a superseded pull request run already cancels itself.
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
* ci: draw each group as its own box in the run graph
The run graph boxes jobs together when they share the same dependencies and
the same dependents. All ten jobs fed only all-checks-passed, so all ten drew
as one pile. A gate per group gives each group a dependent that is exactly
that group.
Release and docs now need the checks, which they should have all along: npm
publish and the pages deploy ran on a merge without waiting for the test job.
Folio stays unblocked so a 20 minute leg does not wait on a 3 minute one.
Claude-Session: https://claude.ai/code/session_01ShuAy8q8ZfPi8KHxwc8JpQ
|
||
|
|
35b0856e6c | test(hierarchy): pin both selector forms and the unknown-key report | ||
|
|
1fcb7a9f7d |
feat(hierarchy): match identifiers by role prefix
idPrefix: is id: with starts-with in place of equality, so a list whose
rows are named <role>_<record id> is reachable by the durable half. The
Android package prefix is skipped the same way id: skips it.
Routing both prefix kinds through matchAttr also makes the object form
work: {descPrefix: ...} matched nothing on the native side while the web
runtime honoured it.
|
||
|
|
7343085614 |
llm action-selection backend (#68)
* feat(spec): add llm() action-backend marker * feat(spec): make llm marker inert on the JS picker * feat(spec): expose __sanderlingSampleInput__ corpus draw * feat(openrouter): minimal chat-completions client * test(openrouter): cover request shape, parse, and errors * feat(verifier): thread screenshot + capture corpus sampler * feat(verifier): LLM accessors — candidates, config, sampler * test(verifier): cover AllCandidates, LLMConfig, SampleInput * feat(trace): record action Source and LLMReasoning * feat(runner): thread step screenshot into PushSnapshot * feat(runner): llmSource selects actions via OpenRouter * feat(runner): wire llmSource selection and trace stamping * test(runner): cover llmSource selection, mapping, downscale * docs(folio): add llm action-backend example spec * docs(folio): document the LLM action backend run * feat(llmclient): support OPENAI_API_KEY, openrouter wins * refactor(runner): rename openrouter package to llmclient * docs: both api keys, example model gpt-5.4-nano * docs: add pr style rules to claude.md * fix(runner): explain action kinds in llm prompt to stop swipe loops * feat(trace): record llm ranked list and chosen rank * feat(runner): stamp llm ranked list and chosen rank on trace * fix(runner): tap by selector to survive layout shift after observe * revert(runner): drop selector-first tap; broke path/testTag selectors * feat(spec): llm() accepts optional instructions * feat(verifier): read llm instructions off config * feat(runner): append spec instructions to llm system prompt * docs(folio): describe app in llm spec instructions * feat(bundler): map generator export to globalThis.generator * feat(verifier): read llm config off globalThis.generator * feat(runner): gate llm source on --generator flag * feat(cmd): add --generator llm|seeded flag * test: cover --generator flag parsing and pickSources gating * feat(verifier): enumerate llm candidates by walking actionsRoot collect-walk the weighted action tree: recurse weighted branches accumulating selection probability, call authored leaves once for concrete actions, enumerate builtins per element. label controls by visible text (borrowing descendant text), fold gestures into directional scrolls over scrollable containers, drop disabled, dedup descriptions. * test(verifier): cover candidate enumeration walk * feat(verifier): add SetupAction to walk setup without the seeded root * test(verifier): cover SetupAction setup-only precedence * refactor(llmclient): make JSONSchema.Schema raw json for pinned field order * feat(trace): record llm choice number and chosen_action echo * feat(runner): llm picks one number from weighted candidates drop the seeded-root call for a setup-only precedence path, render a numbered weighted candidate list, pin a reasoning-first choice schema, strict-skip when chosen_action does not echo the numbered entry, and let the model supply typed values (corpus fallback when empty). * test(runner): cover choice schema, strict-skip, and setup precedence * refactor(verifier): drop the superseded AllCandidates enumeration * feat(folio): drive spec.ts under --generator llm; drop spec-llm.ts * fix(verifier): label editable fields by hint, not the typed value an editable field's own text is its transient content; prefer the hint so the field is named by purpose and the label stays stable. * test(runner): cover weight-suffixed echo and stripWeightSuffix * fix(runner): accept chosen_action echo that carries the weight suffix real runs showed the model copies the whole numbered line including the trailing (w34) weight annotation, so strict-skip rejected ~91% of picks and the llm was paralyzed. strip the weight suffix before comparing. also nudge the prompt to stress-test repeated submissions (idempotency). * fix(verifier): skip llm enumeration on cross-fade frames a navhost mid-transition carries >1 route *Screen in a collapsed coordinate space; acting on it taps garbage (soft keyboard). real runs showed the llm acting on 44% of steps being such frames. skip them so the llm re-observes a settled frame next step. * feat(folio): show current balance on the add-transaction screen renders the account's balance (testTag TxnCurrentBalance) below the account name, above the credit/debit toggle, so before/after screenshots carry comparison data. * fix(replay): derive device space from screen extent, not first node the first positive-bounds element is often a short status-bar node (320x24 on android); using it gave a 320/24 aspect ratio that squashed the screenshot overlay into a grey horizontal band. use the max extent across elements (like the runner's screenBounds) instead. * fix(folio): show balance as a compact one-line label per review: one line, account-name-sized, e.g. "Balance: $0.00" instead of a large balance card. * fix(folio): move balance into the header, one compact line under the account name * fix(replay): attribute deferred violations to the causing step, not detection * fix(replay): show a step's own violations in both panels, no next-step bleed * refactor(hierarchy): one Tree.Transitional, drop the duplicated cross-fade check * chore: ignore .playwright-mcp scratch output * docs: document the llm generator and --generator flag * docs(spec): correct the llm() comment; config reads off globalThis.generator * docs: add pr description rules |
||
|
|
94d9511312 |
test: full test-suite refactor sweep (#61)
* chore(test): start test-suite refactor sweep * test(ltl): pin exact multi-obligation residual AST * test(ltl): table-test finalize Kleene connective combinations * test(ltl): pin reduce over pending inner for bound, Or, Not * test(ltl): marshal bounded Always steps/duration/deadline * test(verifier): cover LTL combinator verdict transitions and within unit panic * test(verifier): table-test DecodeAction kinds and lastAction field exposure * test(verifier): assert WithPlatform(ios) reaches the picker host and key pool * test(verifier): widen weighted-selection assertion to a 5x skew margin * test(verifier): un-skip ax-find round trip with a committed tree fixture * test(runner): pin isWDADrop to sidecar reconnect-failed message origin * test(runner): assert PressKey/Wait trace encoding records kind-specific fields * test(runner): cover RenderSummary unsupported-verbs surfacing branch * test(trace): set Hierarchy in round-trip and lock lossy Tree contract Also add a -race concurrent WriteStep test that asserts N well-formed JSONL lines, catching torn lines if the writer mutex is dropped. * test(trace): round-trip witnesses/changes/metrics/exceptions, pin step-0 witness * test(trace): document ViolationsAreGreppable grep contract and lock-free WriteScreenshot * test(hierarchy): cover invalid-JSON and malformed-bounds parser paths * test(trace): guard writer mutex via WriteStep/Close race on w.file * test(replay): drop unfailable assets and devproxy assertions * test(replay): cache reuses on equal mtime, reparses after append * test(replay): violation marker falls back to detection step when attributed missing * test(replay): corrupt meta/trace dirs return 500 with error body * test(replay): SSE client receives runs.changed after a broadcast * test(replay): Run coalesces creates, ignores write/chmod, closes subs on cancel * fix(sidecar): synchronize health fixture writes and exercise healthError * test(sidecar): cover swipe/longpress/doubletap/erase/presskey/metrics/logs translations * test(sidecar): cover DoubleTapSelector composition and mid-gesture cancel * test(sidecar): assert gRPC error status surfaces from action RPC * fix(chrome): route action methods through runCtx so caller cancellation aborts CDP * fix(chrome): route hierarchy/screenshot/waitidle/metrics through runCtx * refactor(ios): extract pure simctl JSON parsers * refactor(ios): add command-runner seams for EnsureSimulator * test(ios): table-test simctl parsers and EnsureSimulator seams * test(sidecarassets): cover placeholder build path * test(sidecarassets): assert reuse via sentinel bytes not mtime * test(bundler): cover properties-only spec registration * refactor(testrun): extract prepareBundleInputs from Execute * test(testrun): cover prepareBundleInputs aliases and missing-runtime error * test(testrun): table-test resolveRuntimeSibling search edges * test(testrun): exact-output tests for progressHandler line format * fix(cmd): point bundle-check aliases at pkg/spec/src * test(cmd): smoke-test bundle-check resolves spec aliases * test(cmd): table-test hier-check parse and FindAll on fixture * test(cmd): unit-test buildBrowseURL deep-link vs root * test(cmd): drop flaky TestRun_Doctor that launched real Chromium * test(cmd): pin pipeline error to bundle resolution on web platform * test(replay-ui): add bun test script * ci(replay-ui): run bun test via make web-test target * ci(replay-ui): point bun cache key at replay-ui/bun.lock * test(replay-ui): exercise real URL encoding and non-ok throw in getJson * refactor(replay-ui): extract snapshot flatten/getAtPath into lib module * test(replay-ui): pin snapshot flatten/getAtPath path round-trip * refactor(replay-ui): extract action selector/format into lib module * test(replay-ui): pin action selector parse and row formatting * refactor(replay-ui): share one statusFor between panels * refactor(replay-ui): extract run-history derivation into lib module * test(replay-ui): pin shared statusFor precedence and ordering * test(replay-ui): pin run-history derivation alignment * refactor(replay-ui): export clampIndex for testing * refactor(replay-ui): extract keyboard-nav dispatch into pure module * refactor(replay-ui): extract metrics formatters into lib module * test(replay-ui): pin clampIndex step boundaries * test(replay-ui): pin keyboard-nav ownership and key routing * test(replay-ui): pin metrics formatters and path gap handling * refactor(sidecar): expose device-output parsers as internal for testing * test(sidecar): table-test device-output parsers against malformed input * test(sidecar): cover logcat parsing year inference and line skipping * test(sidecar): pin pressKey keycode mapping and unknown-key rejection * test(sidecar): metrics bundleId falls back to launched app and honors override * test(sidecar): loosen deadline upper bound to tolerate slow CI scheduling * test(web-runtime): export selector builders for unit tests * test(web-runtime): guard sanitize cycle, function, and depth limits * test(web-runtime): table-test selector builder quoting and escaping * test(sidecar): collapse scalar-forwarding RPC tests into a table * test(replay-ui): dedup step/summary fixtures into shared module * test(ios): collapse pickSimulator point-tests into a table |
||
|
|
410602d2e1 |
Fix action-system audit findings (#60)
* fix(replay-ui): size overlay viewBox from hierarchy root bounds
Tap points are recorded in the hierarchy's coordinate space (iOS points,
Android pixels, web CSS px) while screenshots are device pixels, so the
overlay rendered at 1/3 position on iOS 3x screens. Derive the viewBox
from the root element bounds; natural image size stays the fallback.
* fix(runner): derive trace tap point from resolveCoordinates
stampSelectorTarget preferred possibly-stale action X/Y while dispatch
preferred the fresh tree-resolved center, so the trace could record a
different point than the one tapped. Both now share resolveCoordinates.
* fix(runner): settle after InputText focus tap before key events
The focus tap raises the keyboard; with no settle the keyboard
animation races the erase/type key events on iOS, landing them in the
wrong field or dropping them. Wait for idle after a successful focus
tap, bounded by the run's idle timeout.
* fix(driver): skip pre-erase for replace-on-input drivers
The web driver's InputText already replaces content via select-all, so
the runner's unconditional EraseText was a redundant round-trip on
every InputText. A new optional TextReplacer capability lets a driver
assert replace semantics; the runner skips the erase when asserted.
* fix(hierarchy): rank spatial-fallback matches by specificity
The bounds-containment fallback returned the first pre-order match, so
a screen-sized container could win over the intended small element.
Matches are now ordered smallest-area first; equal-area matches keep
pre-order, preserving the iOS-flat equal-bounds sibling pattern.
* fix(runner): treat an unchanging transitional tree as settled
A UI persistently showing two route-level Screen ids (overlay, both
route ids alive at rest) burned the full retry budget every step and
skipped the verifier forever. A tree byte-identical to the previous
attempt now breaks the retry loop as settled; genuine cross-fades
differ between attempts and keep the retry/skip behavior.
* fix(replay-ui): skip synthetic zero-bounds root in deviceSpaceOf
The iOS hierarchy prepends a zero-bounds node before the real root
window, so elements[0] returned undefined and the overlay fell back to
the screenshot's pixel size. Take the first element with positive
extent instead; pre-order puts the root window before any content.
Verified against a real iOS trace in the replay UI.
* fix(sidecar): never replay non-idempotent actions after reconnect
A dropped connection mid-action (e.g. a read timeout while the device
is still typing) re-ran the whole block after reconnecting, typing the
text twice and double-firing taps. Non-idempotent actions now reconnect
for the next RPC's benefit but surface UNAVAILABLE, which the runner
already treats as transient; idempotent reads keep the replay.
* fix(sidecar): land the second double-tap sequentially on gesture collision
The overlapped second tap can hit the XCTest runner while the first
gesture is still executing ('only one gesture can be performed at a
time'), failing the step. The second tap now waits the first out and
retries once, keeping the tight gap on the happy path.
* fix(sidecar): map non-Exception throwables to INTERNAL status
The vendored iOS client throws failures that do not extend Exception;
runRpc missed them, killing the RPC as a channel-level Unknown the
runner cannot classify. Catch Throwable instead.
* feat(sidecar): close the driver and app under test on shutdown
* test(sidecar): cover service shutdown paths
* fix(testrun): stop the sidecar with SIGTERM before killing
* fix(sidecar): reap orphaned XCTest runner sessions at iOS init
* fix(sidecar): probe channel liveness before restarting the XCTest runner
* test(sidecar): cover WdaRecovery restart and retry policy
* fix(sidecar): absorb first-leg double-tap collision sequentially
* fix(runner): scope WDA-drop detection and cap consecutive transient failures
* chore(sidecar): silence vendored loggers on expected failure paths
* fix(runner): absorb one-off apply errors; only an unbroken streak aborts
* fix(folio): install the current build before the Android fuzz run
* chore(sidecar): silence absorbed view-hierarchy poll noise in Android runs
The driver logs an ERROR for every on-device view-hierarchy fetch that the
device-side server cancels or times out while the UI animates. The stability
poll fetches the hierarchy on a sub-second cadence and swallows those throws
to keep polling, so each line is advisory with no effect on the run. Real
failures still reach the runner as gRPC status errors, so nothing is lost.
|
||
|
|
19a470121f |
fix: iOS spec driving, InputText replace semantics, native DoubleTap (#58)
* fix(hierarchy): bounds-containment fallback for scoped and path queries Compose on iOS surfaces a testTag node as an empty leaf sibling of the content it labels instead of as an ancestor, so descendant search under the tagged node finds nothing and every path or scoped query returns null. When structural search yields no match, fall back to nodes whose bounds lie inside the scope node's bounds. * feat(sidecar): derive iOS clickable and editable from element type The XCTest hierarchy mapping dropped the element type, leaving no clickable or editable flags on iOS, so the fuzzer's tap and typing verbs never found a candidate inside the app. Map the raw accessibility tree directly and derive clickable, editable, scrollable, and class from the XCUIElementType raw value. * feat(proto): add EraseText RPC for InputText replace semantics * feat(driver): add EraseText to the device driver surface * fix(runner): erase existing field text before InputText InputText appended on native platforms, so repeated draws grew fields without bound. The folio fuzz run wedged on the add-account screen: each draw concatenated another name until the 40-character validation error became permanent. Replace semantics also makes retried typing idempotent. The web driver already replaced via select-all; native now matches. * feat(sidecar): EraseText backend support on android and ios * fix(folio): saturation-gate account creation in the spec The 2-3 step add-account loop outcompeted the 5-step transaction chain at every weighted re-draw, so runs filled with account creation and rarely exercised the balance properties. Stop offering add-account once three accounts exist; the renormalized weights then favor the transaction flow at every step of its chain. * fix(folio): author spec weights to match testing intent Revert the account saturation gate: it starved newAccountBalanceIsZero once it tripped, and a magic account count is app-state tuning, not intent. Instead weight the generators by what the properties need: the transaction chain leads, account creation stays exercised, and doubleTaps gets explicit weight everywhere because double-submission idempotency is what the spec is testing for. * fix(folio): lower doubleTaps weight to 5 * fix(sidecar): surface visible text on iOS static elements Static text and button strings live in the accessibility label on iOS, so the text attribute came through empty and every balance extractor parsed to zero, silently disarming both folio properties. Non-editable elements now fall back title, value, then label; editable fields keep value-only so an empty field's caption does not read as content. * feat(driver): native DoubleTap RPC for a tight inter-tap gap Composing two Tap round trips from the Go client spread the taps by hundreds of milliseconds on iOS, wide enough for the app to navigate between them, so double-submission races could never reproduce. The sidecar now lands both taps back-to-back next to the device transport. * feat(sidecar): pipeline iOS double-tap requests Queue the second tap at the XCTest runner while the first executes. The runner serializes handlers, so this is the tightest gap the transport allows (~350ms per tap round trip); recorded here with measurements for the iOS double-tap limitation. |
||
|
|
c5bb176be8 |
UX refactor (#52)
* feat(ltl): bound fields on AlwaysFormula and named thunks Add StepBound/Duration/Deadline to AlwaysFormula as the dual of bounded Eventually, give ThunkFormula a Name for stable identity, add ThunkNamed, and surface both in describe() and MarshalJSON. * feat(ltl): negation normal form pass nnf/pushNot rewrite a formula so every Not wraps only a Thunk or Error leaf, dualizing Always<->Eventually and preserving bounds. * feat(ltl): NNF in NewEvaluator, bounded-always, Finalize, collapse Apply nnf on construction, reduce bounded Always symmetric to bounded Eventually (vacuous holds once the window closes), add Finalize to resolve undischarged liveness obligations to Violated at run end, and collapse structurally-identical pending obligations. * test(ltl): property-based NNF laws Lock double-negation identity, Always/Eventually duality with bound preservation, leaf pushdown, and not(always true) reaching Violated. * test(ltl): Finalize, bounded eventually, latch, collapse Property tests for monotonic violation latch and eventually-within violating iff n consecutive false, plus Finalize and collapse cases. * feat(inspect): within clause on always residual node A negated bounded eventually serializes as a bounded always; render its bound instead of dropping it. * feat(ltl): witness violations and (bool,error) predicate thunks * test(ltl): migrate thunk call sites to (bool,error) * feat(ltl): flag thrown-predicate witnesses with IsError * refactor(verifier): replace predicate err side-channel with violation witness * test(verifier): witness API for thrown predicates * feat(trace): witnesses map and skipped-verification marker on Step * feat(runner): thread violation witnesses, finalize, skip marker into trace * test(ltl): lock violation witness reason, IsError, and step * test(verifier): finalize surfaces unmet eventually with witness * fix(ltl): eliminate implies and bounded-always false-negatives Rewrite a -> b to (not a) or b in NNF so a pending temporal antecedent can no longer defer the whole implication and drop a consequent that was false at the current step. Carry a pending inner past a bounded-Always window close instead of dropping it to holds, so a deferred obligation is resolved by a later step or Finalize. * test(ltl): lock implies and bounded-always false-negative regressions * fix(web-runtime): seed PRNG for reproducible runs and align weighted pick * feat(testrun): inject seed into web bundle via SANDERLING_SEED define * test: cover web-runtime seeded PRNG, weighted pick, and seed define wiring * test(spec): add Go math/rand/v2 PCG oracle and golden fixture * feat(spec): bit-exact PCG port of Go math/rand/v2 * test(spec): assert pcg.ts matches the PCG golden fixture * feat(spec): shared input corpus and press-key pools * feat(spec): action-tree types and Host interface * feat(spec): verb support matrix and warn-once helper * feat(spec): deterministic shared action picker * test(spec): verb matrix and warn-once semantics * test(spec): picker draw-order and determinism * refactor(spec): actions.ts returns pure GeneratorNode data trees * refactor(spec): wire from() sampling through the picker rng * feat(spec): shared runtime-entry installs next-action over pick.ts * feat(spec): export LongPress/Scroll/longPresses/scrolls factories * test(spec): assert data-tree shapes for action factories * test(spec): runtime-entry serializeAction wire-contract round-trip * refactor(spec): bridge data-tree nodes to the legacy goja picker tags * fix(spec): web runtime walks the spec's globalThis.actions data tree * test(spec): tolerate legacy bridge fields on builtin nodes * refactor(spec): installRuntime accepts a lazy root resolver The web bundle imports the runtime before the spec, so the action root on globalThis.actions only exists after the spec evaluates. Accept a function form so the goja and web hosts resolve the root per tick. * refactor(spec): web-runtime becomes the WEB Host, delegates to shared picker Delete the duplicate picker (resolveGenerator/pickWeighted/randomTap/ randomInput/randomSwipe/randomPressKey/pickFromArray, the mulberry32 PRNG, and the snake_case serializeAction) plus the __sanderling__ action factory binds. web-runtime now implements Host (platform/seedHi/seedLo from the injected 64-bit seed via BigInt, queryCandidates over the live DOM with a per-tick cache, reportUnsupported) and calls installRuntime so both engines run pick.ts over the same Pcg. Swipe/longPress/scroll follow the verbs.ts matrix instead of silently returning null. Keeps the DOM helpers (selector translation, queryElement, elementHandle, buildState, sanitize, extractors) and the global locking. Net -214 lines (741 -> 527). * test(spec): cover the WEB Host surface and seed precision Replace the deleted-picker tests with Host coverage: platform()==web, seedHi() parsing a 64-bit seed without Number precision loss, seedLo()==0, reportUnsupported warning, the installed next-action/extractor globals, and queryCandidates verb routing + per-tick caching over a querySelectorAll stub. * refactor(spec): picker emits native selector + scroll endpoints, setup precedence * feat(spec): goja runtime entry wires the shared picker over the Go host * feat(bundler): optional RuntimeFile prepends a runtime-entry import via stdin * feat(testrun): bundle the goja runtime entry so the verifier runs the shared picker * refactor(spec): drop the legacy goja bridge fields from action factories * feat(spec): serialize selector-only string targets for the runner to re-resolve * refactor(verifier): one DecodeAction reads the unified flat wire contract * refactor(verifier): goja host + shared picker replace the duplicate Go picker * refactor(runner): decode V8 actions via the unified DecodeAction; wire goja runtime * test(verifier): author specs through the shared picker path * test(runner): bundle authored specs with the goja runtime entry * feat(verifier): collect unsupported verbs for the run report * refactor(runner): collapse WebDriver forks behind ActionSource/ExtractorSource * feat(testrun): surface unsupported verbs in run report * test(verifier): cross-runtime goja/node parity gate on the shared picker * test(verifier): unsupported verbs collected deduped in first-seen order * test(runner): summary reports no unsupported verbs on a clean run * test(spec): golden-fixture cross-runtime parity gate for the node picker Replace the env-driven parity harness with a shared scenario module and a committed golden the node picker asserts independently. The goja side asserts the same golden, so neither runtime invokes the other at test time. * test(verifier): assert goja picker against the same cross-runtime golden Drop the node-subprocess coupling: the goja side now installs a stub __sanderlingHost__ with the fixed candidate list and asserts the committed golden, matching pkg/spec/test/parity.test.ts. * refactor(spec): rename pressKey generator export to pressKeys * refactor(spec): update barrel re-exports for pressKeys * test(spec): update pressKeys generator export name * docs(spec): rename pressKey generator to pressKeys * refactor(spec): extract samplerRng into shared sampler-rng module * feat(spec): add fluent seeded value generators (strings/integers/emails/edgeCaseText) * test(spec): cover fluent value generators determinism and chaining * refactor(bundler): inject globalThis trailer from spec named exports * refactor(bundler): reuse registration trailer in web bundler * test(bundler): cover named-export globalThis registration * feat(spec): add named() to Extracted handle type * feat(web-runtime): named() and cross-extractor read guard * feat(verifier): named() and cross-extractor read guard in goja * test(verifier): cross-extractor read guard and named() * test(web-runtime): export runtime and extractors for tests * test(web-runtime): named() and cross-extractor read guard * refactor(folio): drop manual globalThis trailer (bundler injects it) * refactor(folio): seed txn amounts via integers().between(1,500) * refactor(folio-web): drop manual globalThis trailer (bundler injects it) * fix(folio-web): seed card/txn-type selection via from().generate() for reproducible runs * refactor(folio-web): weight valid generators against edgeCaseText for names/amounts * refactor(folio-web): name extractors so violation witnesses are readable * fix(web-runtime): propagate extractor getter throws and unpoison locked global Stop swallowing getter errors in evaluateExtractors so the cross-extractor read guard aborts loudly, matching goja's PushSnapshot. Make the __sanderling__ lock configurable (still non-writable) so a shared test process can reinstall a fake. * test(spec): install fake runtime via defineProperty to survive locked global * test(web-runtime): assert uncaught cross-extractor read aborts evaluateExtractors * feat(runner): add MaxSteps bound to Options * test(runner): MaxSteps stops after exactly N steps * test(driverpb): drop proto getter round-trip tautology * test(sidecar): drop stub-mode placeholder tautology tests * test(mock): drop default-field-value assertion test * test(ltl): drop Verdict.String tautology tests * refactor(runner): extract RenderSummary for snapshot testing * test(runner): golden snapshots for trace stream and violation summary * feat(web-runtime): capture uncaught errors into state.exceptions * test(integration): add throwing and counter web fixtures * test(integration): add specs for the web fixtures * test(integration): drive web fixtures through the real pipeline in headless Chrome * chore(make): add test-browser target for the Chrome-driven suite * ci: run the Chrome-driven browser suite in a separate job * refactor(test): relocate browser suite to test/browser * refactor(permissions): delete dead internal/permissions package * refactor(test): rename package to browser_test * refactor(sidecarassets): rename internal/sidecar to internal/sidecarassets * chore(make): point test-browser at test/browser * docs(decisions): record internal/permissions deletion * refactor(doctor): use sidecarassets package * refactor(testrun): use sidecarassets package * fix(test): resolve testdata relative to browser_test.go * refactor(verifier): remove dead __sanderlingIndex compat alias * refactor(bundler): use encoding/json for JS string literals * docs(action-space): use vendor-neutral native driver wording * refactor(hierarchy): scrub backend tool name from comments * refactor(driver): scrub backend tool name from comments * refactor(driver): add DoubleTap and DoubleTapSelector to DeviceDriver * refactor(sidecar): implement DoubleTap with the sub-100ms inter-tap gap * refactor(chrome): implement DoubleTap as two taps with the gap * refactor(mock): record DoubleTap and DoubleTapSelector actions * refactor(runner): delegate double-tap to driver, drop gesture timing * test(runner): assert double-tap delegates to driver DoubleTap * docs(cmd): add package docs to CLI and developer tools * docs(driver): add package docs to driver interface and chrome backend * docs(driver): add package docs to mock and sidecar backends * docs(platform): add package docs to android and ios device prep * docs: add package docs to bundler and inspect * docs(ltl): add package doc to temporal logic evaluator * docs: add package docs to runner and testrun pipeline * docs: add package docs to trace and verifier * docs(sidecarassets): add package doc for embedded JAR loader * fix(chrome): add disable-dev-shm-usage so Chrome starts in CI * test(chrome): gate real-Chrome driver tests behind the browser tag * chore(make): run chrome driver tests in the browser job * fix(web-runtime): guard global error listeners for non-browser hosts The module registered window error/unhandledrejection listeners at top level, which threw under Node (the spec-api test runner) where globalThis.addEventListener is absent. Register only when the API exists; the real browser run is unaffected. * ci(browser): re-enable unprivileged user namespaces for headless Chrome ubuntu-latest moved to 24.04, whose AppArmor restriction on unprivileged user namespaces stops headless Chrome from opening its DevTools socket even with --no-sandbox, surfacing as the driver's 'websocket url timeout'. Relax the sysctl for the job and add a direct launch check so a future breakage shows Chrome's own stderr rather than an opaque driver timeout. * ci(browser): pin stable Chrome for the driver tests setup-chrome's default latest pulled a dev Chromium (150) whose remote debugging socket never came up under chromedp, while plain --dump-dom worked. Pin the stable channel, which the driver is tested against. * feat(defaults): add scroll and rebalance action weights Use relative-integer weights (taps/typing co-primary 100, scrolls 50, swipes 25, doubleTaps 10); the picker normalizes by their total. Adds scrolls to defaultActions as a first-class reveal behavior. * feat(defaults): trim scroll action weight wiring * fix(build): point sidecar jar ignore and embed paths at sidecarassets * test(defaults): drop stale longPresses re-export assertion longPresses is opt-in vocabulary, no longer re-exported from defaults/actions.ts since e0d3b20; its builtin resolution is already covered by api.test.ts. Trim the defaults test to scrolls, which is an actual default export. * fix(chrome): raise DevTools websocket read timeout to 60s Chrome cold-start on a loaded CI runner can exceed chromedp's 20s default for reading the DevTools websocket URL, flaking the browser tests with "websocket url timeout reached". Give launch more headroom. |
||
|
|
88db9653e5 |
refactoring default action layer (#51)
* feat(hierarchy): add editable signal with native derivation
* feat(chrome): emit editable flag in hierarchy dump
* feat(verifier): expose editable on ax element objects
* feat(spec): add editable to selector and element types
* feat(verifier): register typing builtin generator
* feat(verifier): typing builtin types edge-case corpus into editable fields
* feat(spec): export typing builtin generator
* feat(spec): add defaultActions bundle
* feat(spec): export @sanderling/spec/defaults subpath
* feat(folio): layer defaultActions breadth over targeted flows
* test(verifier): typing builtin targets editable fields, declines otherwise
* test(hierarchy): editable derivation and selector matching
* test(spec): defaultActions, typing, and defaults barrel resolve
* fix(testrun): alias @sanderling/spec/defaults for the bundler
* test(chrome): editable flag for inputs, textarea, contenteditable
* feat(spec): typing builtin for the web (V8) action path
* chore(folio): auto-boot a bootable AVD in just test/install when none connected
* feat(driver): add ForegroundChecker optional capability
* feat(android): detect foreground package via adb dumpsys
* feat(sidecar): implement ForegroundApp via adb for android
* feat(runner): relaunch app when foreground escapes during exploration
* fix(spec): drop hardware back from defaultActions to stay in-app
* feat(spec): add DoubleTap action type and constructor
* feat(spec): wire DoubleTap through web-runtime serializer
* feat(verifier): bind doubleTap and decode DoubleTap actions
* feat(runner): dispatch DoubleTap as two taps inside one step
* test(doubleTap): cover constructor, verifier round-trip, and runner dispatch
* feat(folio): add noDuplicateTxnPerStep invariant and doubleSubmitTxn action
* fix(folio): track ledger row count across non-ledger steps; pin reproducer seed
* feat(spec): add doubleTaps random-target builtin to defaultActions
* feat(verifier): add doubleTaps random-target generator
* refactor(folio): drop doubleSubmitTxn; fuzzer surfaces double-submit via defaultActions
* fix(folio): make ledgerRowsSeen monotonic to suppress transient-render false positives
* feat(verifier): track newly-violated property set per step
Sticky `always(P)` violations re-surfaced on every step after onset,
flooding traces and summaries with duplicate records. EvaluateProperties
now diffs against the prior verdict map and records the onset set; a new
NewlyViolatedProperties accessor exposes it so callers can emit each
violation exactly once at its onset step. The verdict-map return is
preserved for residual / current-verdict consumers.
* refactor(runner): emit onset-only violations to trace and summary
Switch the per-step violation list from the sticky verdict map to the
verifier's onset set. Each property now appears exactly once across a
run: at the step it first violates, not on every subsequent step where
the residual stays false. Removes the dead violationNames helper.
* style(verifier): use maps.Copy for verdict snapshot
* fix(folio): make login spec content-driven (idempotent across re-entries)
* fix(verifier): canonicalize selector strings
Object/chain JS selectors used to fall through to goja's default
stringification, producing "[object Object]" tags that surfaced as
garbage in trace.action.selector. Emit canonical "k:v" / " > "-joined
strings instead so the tag round-trips back through the hierarchy
selector grammar.
* refactor(folio): replace txn invariants with balanceMatchesAddedTxn
Collapse noDuplicateTxnPerStep and newTxnChangesBalance into a single
per-row property: every newly-appearing ledger row's signed amount must
match the ledger balance delta. A double-submit lands two rows whose
individual amounts cannot both equal the aggregate delta, so each row
fires the property, catching both the row-count and balance-math
classes of bug under one semantic invariant.
* refactor(trace): drop WriteScreenshotAfter
Only one screenshot per step is captured now (concurrently with
hierarchy after settle), so the -after.png variant is unused.
* refactor(runner): one concurrent screenshot per step
Move screenshot capture into the post-action errgroup so it observes
the same UI moment as the hierarchy fetch. Drop the pre-action and
deferred -after captures. Skip WaitForIdle when the action is Wait
since the wait itself provides settling time.
* refactor(inspect-ui): use next step's screenshot for state after
Each step now has one screenshot (the moment of observation). The
"state after" view of step N is the same moment as step (N+1)'s
observation, so reuse that file rather than expecting a separate
-after.png.
* feat(sidecar): structural-hash settle poll
Add pollUntilStable and structuralHash helpers; wire them into the
Stub, Maestro, and iOS backends' waitForIdle. The structural hash
ignores bounds-only flicker (measure passes) but trips on any change
in resource-id/class/content-desc/text, so a Compose cross-fade where
both source and destination composables are momentarily alive no
longer slips through Maestro's waitForAppToSettle and contaminates
the next hierarchy fetch.
* test(sidecar): cover pollUntilStable and structuralHash
Verify the poll returns on two equal snapshots, after transient
churn, and at the cap when never stable; assert the hash ignores
bounds-only flicker and detects content changes.
* feat(spec): accept optional name on extract()
Add an (name, getter) overload so each extractor handle carries a
debuggable label that future trace fields (per-step diffs) can key
off. The web-runtime falls back to extractor_\${index} when none is
supplied so existing call sites keep working unchanged.
* test(spec): cover extract name overload
Verify the runtime receives an undefined name in the legacy shape,
the supplied name in the (name, getter) shape, and that
extract("name") with no getter throws.
* feat(verifier): name extractors for diff surfacing
bindExtract accepts an optional name argument; falls back to
extractor_N when omitted. The name is stored on extractorState
alongside prev/curr value caches that the next change will use to
emit per-step diffs.
* chore(folio): name every extract() call
Give each extractor in the Folio spec a debuggable label so the
inspect UI can render extractor-value diffs at violation steps
keyed by intent (ledgerRows, route, ledgerBalance, ...) rather
than by registration index.
* feat(verifier): track extractor value transitions
Cache each extractor's prior and current JSON-encoded value during
PushSnapshot; expose ChangedExtractors to surface per-step diffs the
runner can emit into the trace. The first observation flushes every
non-null extractor as a change so the inspect UI shows initial state
breadcrumbs alongside later transitions.
* test(verifier): cover ChangedExtractors diffs
Verify initial snapshot reports both named and fallback-named
extractors, a subsequent change surfaces prev/curr, and a no-op
snapshot leaves the diff empty.
* feat(trace): emit extractor_changes per step
Add ExtractorChanges to trace.Step and a runner helper that converts
the verifier's diff map into the trace shape. The inspect UI keys
its violation breadcrumbs off this field.
* feat(inspect-ui): render extractor-change breadcrumbs at violations
Show prev -> curr for each extractor whose value changed on the
selected step, anchored under the violation row in ActionList.
Long values collapse into <details> so the inline diff stays
readable while the full payload is one click away.
* fix(sidecar): cap stability poll independently of settle budget
The previous shape halved durationMillis between waitForAppToSettle
and the structural poll, then hammered hierarchy() at 80ms intervals
- on Maestro this stacked enough RPCs that hierarchy fetches began
timing out under load and the run stalled. Pass the full budget to
waitForAppToSettle and cap the follow-up structural poll at 600ms
with a 120ms interval, so the device sees at most a handful of
extra hierarchy reads per step.
* feat(cli): default --clear-data on so runs start fresh
* feat(sidecar): streak-based settle with route-transition detection
Two changes layered into the stability poll:
1. stabilitySnapshot returns null while the tree carries more than one
route-level Screen tag (resource-id / testTag / identifier ending
in "Screen"), so the poll cannot declare a NavHost cross-fade
stable. Apps following the Compose route convention get this
detection for free; apps that don't fall through to the generic
signal below.
2. pollUntilStable now requires an uninterrupted stable streak of at
least MIN_STABLE_STREAK_MILLIS rather than just N consecutive
matches. A late transition that fires after a brief calm window
breaks the streak instead of slipping past. Interval widened to
250ms so UiAutomation isn't hammered under fuzz load.
* test(sidecar): cover streak reset and route-transition rejection
Verify the poll honors MIN_STABLE_STREAK_MILLIS, that a transient
mid-stream change resets the streak, that null returns block streak
progress through a NavHost cross-fade, and that stabilitySnapshot
counts only route-level attribute keys when summing Screen tags.
* feat(runner): re-fetch on transitional hierarchy capture
Some actions trigger async work (DB write, ViewModel coroutine) whose
navigation transition begins after the sidecar settle poll has already
exited. Without intervention, the next iteration's hierarchy fetch
lands mid cross-fade and the verifier observes a partial extractor
state which then surfaces as a false-positive violation at the step
where the transition completes.
fetchSyncedState pairs hierarchy + screenshot in one goroutine and
retries the pair (up to 4 times, 200ms apart) while the captured tree
contains more than one route-level *Screen tag. Steps that observe
no transition get no added cost; steps that catch a transition pay
up to ~600ms extra wall time but record a tree that matches the
post-transition state the property language expects to compare.
* feat(runner): gate first action on app reaching foreground
* test(runner): cover startup foreground gate and back-press
* feat(verifier): scope random-action targets to app package
Random tap/doubleTap/type/swipe candidates now exclude nodes whose package differs from the app under test, so exploration never fuzzes the soft keyboard, system UI, or permission dialogs. An unset app package or an element with no package stays in scope, preserving behavior on iOS.
* feat(testrun): pass app package into verifier scope filter
* test(verifier): cover package-scoped target selection
* feat(hierarchy): derive package from resource-id prefix
The Android sidecar omits an explicit package attribute, so the verifier's package scope filter was a no-op and the keyboard still leaked into targets. Native nodes carry their package as the resource-id prefix; derive it there when the attribute is absent. Compose testTags are colon-less and stay empty, keeping them in scope.
* test(hierarchy): cover package derivation from resource-id
* chore: stop tracking inspect-ui/dist build artifacts
* feat(android): detect focused-window package via dumpsys window
* feat(driver): add FocusedWindowChecker capability
* fix(runner): gate first observe on the app window being drawn, not just resumed
* test(mock): add FocusedWindowApp with foreground mirroring
* test(runner): cover startup gate waiting for app window to draw
* feat(proto): add Snapshot RPC for atomic hierarchy+screenshot
Pairs hierarchy and screenshot in a single response so the runner can
capture both under a backend mutex, avoiding the cross-fade race where
the two reads describe different frames.
* feat(sidecar): add snapshot default on DriverBackend
Default impl calls hierarchy() then screenshot(). The service layer wraps
the call in a mutex so concurrent runners observe a serialized pair.
* feat(sidecar): wire Snapshot handler with serialization lock
Synchronizes backend.snapshot() so concurrent runners observe a
serialized hierarchy+screenshot pair, eliminating the cross-fade race
where two parallel reads describe different frames.
* test(sidecar): cover Snapshot wire path and serialization lock
SnapshotHandlerTest asserts both fields are populated, concurrent calls
are serialized, and the default impl runs hierarchy then screenshot.
* feat(driver): expose Snapshot on DeviceDriver and sidecar client
Snapshot wraps the new atomic-snapshot gRPC: the runner gets hierarchy
and screenshot from one round-trip whose two reads are serialized on
the sidecar side.
* feat(driver): add Snapshot to chrome and mock drivers
The chrome tab is single-threaded so its Snapshot pairs the two reads
without extra locking. The mock records ActionSnapshot so tests can
assert the runner reaches for the paired RPC.
* refactor(runner): observe each step via the atomic Snapshot RPC
fetchSyncedState now issues one Snapshot per attempt so hierarchy and
screenshot describe the same on-device frame. The transitional retry
stays: that case handles a fully-captured but mid cross-fade frame,
which atomic capture cannot fix.
* test(runner): assert step uses Snapshot, not raw hierarchy/screenshot
TestRunner_UsesAtomicSnapshot catches regressions to the two-goroutine
race, and the existing parallel-fetch test now keys off ActionSnapshot.
* test(driver): cover Snapshot in proto descriptor and sidecar client
Adds Snapshot to the descriptor allowlist and a sidecar-client test that
asserts both fields come back over the wire.
* feat(trace): add Transitional flag to Step
* fix(runner): skip verifier for transitional trees after retry budget
When fetchSyncedState exits its retry loop with a tree that still shows a NavHost cross-fade, the runner now marks the step transitional, writes the step + screenshot to the trace, and skips Verifier.PushSnapshot / EvaluateProperties / ChangedExtractors so the previous-to-current extractor advance is not poisoned by transient state. The next clean step's previous still references the prior clean state. NextAction continues to run so the loop never deadlocks on a never-stabilizing screen.
* test(runner): cover transitional step skips verifier and clean control
* refactor(trace): rename Step.Action to Step.NextAction
The trace step's action field is the action chosen FOR THE NEXT iteration
based on observing this step's hierarchy, not the action that produced
this step. Rename Step.Action to Step.NextAction and the JSON tag to
next_action to make causality explicit at the data level.
* refactor(runner): assign trace action to Step.NextAction field
Follows the rename of trace.Step.Action to Step.NextAction. The runner
already computed the next iteration's action here; only the field name
changes.
* refactor(inspect): decode trace step's next_action JSON field
Mirrors the trace schema rename of action to next_action. The summary
shape exposed to the SPA (action_kind/action_label) keeps its current
JSON tags since these are derived labels, not the raw next-action.
* test(inspect): update fixtures to use next_action trace field
Aligns inspect tests with the trace schema rename. Step constructors
now set NextAction and the JSONL fixtures use the next_action tag.
* refactor(inspect-ui): rename Step.action to Step.next_action
Aligns the SPA type and consumers with the trace schema rename. The
StepSummary.action_kind/action_label labels stay unchanged since they
are derived labels, not the raw next-action.
* fix(folio): extract balanceMatchesAddedSum predicate as testable helper
Move the ledger-balance-vs-added-rows predicate into a pure helper module
so the property's logic is unit-testable in isolation. Marks the sanderling
example as an ES module so cross-package ESM imports resolve under node.
* fix(folio): use sum-of-added-rows in balanceMatchesAddedTxn
The old predicate (every row's signed amount equals delta) silently passed
the double-submit bug because two same-amount rows each match the delta in
isolation. Switching to the sum check (addedSum === delta) catches both the
double-submit case and any future multi-row append whose total drifts from
the balance change.
* test(spec): cover balanceMatchesAddedSum single, sum-match, over, under cases
Pins the sum-based predicate: a single new row matching delta and two new
rows summing to delta both hold; two-row over-sum (double-submit) and
under-sum cases both violate.
* fix(build): rebuild sidecar JAR when Kotlin sources change
Without source-file deps on $(SIDECAR_JAR), make never re-ran shadowJar
after a Kotlin edit, so a stale embedded JAR shipped on every install
and the new sidecar code was silently absent at runtime.
* fix(chrome): launch with no-sandbox so headless Chrome starts in CI
* fix(sidecar): type text at cursor instead of clearing the field
InputText now appends at the focus caret, matching the native driver
and the standard mobile-input contract, instead of deleting existing
content first. Adds an injectable command runner so the behavior is
testable without a device.
* test(sidecar): assert InputText types at cursor without clearing
Captures the adb command stream and verifies a single input-text call
with no preceding delete keyevents, plus the adb escaping cases.
* feat(proto): add LongPress RPC
* chore(proto): regenerate Go stubs for LongPress
* feat(driver): add LongPress to DeviceDriver interface
* feat(sidecar): add LongPress client method
* feat(mock): record LongPress action
* feat(chrome): implement LongPress as press-and-hold
* feat(sidecar): implement longPress across backends
* feat(sidecar): dispatch LongPress RPC to backend
* test(sidecar): cover LongPress dispatch
* test(sidecar): implement longPress in snapshot test backend
* feat(verifier): add LongPress and Scroll action kinds
* feat(folio-spec): predicate that gates balance check on TxnSubmit tap
Replaces the row-sum predicate (which always held by construction since
balance is derived from rows in Folio) with one that compares the typed
amount to the actual balance delta after a tap on TxnSubmit. Catches the
planted double-submit bug.
* feat(folio-spec): wire submitMovesBalanceByTypedAmount property
Adds lastAction and totalBalance extractors and uses them in the new
property. Drops ledgerRows/ledgerBalance extractors since nothing else
referenced them.
* feat(verifier): wire longPresses and scrolls generators
* test(verifier): cover longPresses and scrolls generators
* test(folio-spec): unit tests for submitChangesBalanceByTypedAmount
Covers single vs double submit, the DoubleTap variant, vacuous cases
(null action, wrong kind, wrong target, zero typed), and selector-as-
object coercion.
* feat(spec): add LongPress and Scroll authoring surface
* feat(spec): no-op LongPress and Scroll in web runtime
* feat(spec): re-export longPresses and scrolls as opt-in generators
* test(spec): cover LongPress and Scroll runtime members
* test(proto): expect LongPress in service descriptor
* feat(runner): dispatch LongPress and Scroll actions
* test(runner): cover LongPress and Scroll dispatch
* docs(action-space): move LongPress, Scroll, DoubleTap to current actions
* fix(runner): mark nil/empty hierarchy as transitional
A failed or empty sidecar hierarchy fetch was pushed straight to the
verifier, letting spec extractors crash with "Cannot read property 'map'
of undefined" when findAll returned null. Treat that case like a
transitional capture: skip the verifier push, still record the step, and
keep the loop progressing.
* fix(verifier): populate Action.On when tap chooser picks an element
Coordinate-targeted Taps/DoubleTaps left On empty, so action-gated
properties reading lastAction.on couldn't tell which target was hit and
were vacuously skipped. Resolve the picked element to a stable
key:value selector (resource-id, testTag, text, desc) and validate it
resolves back to the same element so we don't accidentally redirect the
tap to a sibling that shares the identifier.
* fix(folio): add parseTypedAmount helper matching app's parseCents
Raw user input like "50" must become 5000 cents, not 50. The existing
parseDollarCents helper strips non-digits and so reads "50" as 50 cents,
which is correct for formatted balance text but off by 100x for raw
input from the amount field.
* fix(folio): parse raw amount input as cents in submit predicate
txnAmountField holds raw user keystrokes, not formatted balance text.
Route it through parseTypedAmount so "50" reads as $50, matching how
the app commits the transaction.
* fix(folio): carry forward total balance across off-screen transitions
AddTransactionScreen shows neither AccountCard nor LedgerBalance, so the
extractor used to report 0 at the step before submit. That made every
non-zero current balance look like the full delta and tripped the typed
amount property on every honest submit. Remember the last-seen sum and
return it whenever the current snapshot has no balance signal.
* test(folio): cover submit predicate with raw typed-amount inputs
Pipes realistic raw keystrokes through parseTypedAmount + the predicate
so single submits clear and double submits fire as expected.
* feat(folio): add computeHomeTotalBalance helper
Pure helper that tracks Home multi-account total only and carries the last
Home sum across off-Home steps. Ledger's single-account balance is excluded
because mixing it would corrupt cross-screen scale comparisons.
* fix(folio): totalBalance carrier tracks only Home, not Ledger
Home cardSum is a multi-account total; Ledger's LedgerBalance is a single
account on a different scale. Blending them in the carrier produced bogus
cross-screen deltas (prev from Ledger, curr from Home), triggering false
positives in submitMovesBalanceByTypedAmount. Restrict the carrier to
Home AccountCard totals via the computeHomeTotalBalance helper.
* test(spec): cover computeHomeTotalBalance carrier behaviour
Tests Home sums, carrier passthrough on off-Home steps, the Ledger
scale-mismatch case, and a Home > off-Home > Home sequence.
* feat(runner): treat transient apply errors as transitional steps
Sidecar input RPCs occasionally hang with DEADLINE_EXCEEDED or
UNAVAILABLE on long fuzzing runs. The per-step loop previously
propagated any applyAction error and killed the run after a single
flake. Detect transient gRPC failures via status.FromError, mark the
step transitional, skip the post-action idle poll, and continue to the
next step. Fatal errors (outer ctx cancellation, non-transient codes,
verifier crashes) still propagate.
* test(runner): cover transient apply error resilience
TestRunner_TransientApplyErrorMarksTransitional drives the runner
through a wrapper that fails the first TapSelector with a gRPC
DeadlineExceeded then succeeds. Asserts the run does not exit, the
failed step is marked transitional with no violations, and the next
step runs cleanly. TestIsTransientApplyError_Classification covers the
helper's matching rules directly so future code changes don't quietly
drop a transient case.
* fix(folio): gate submit-balance property on Home route landing
totalBalance is only freshly computed when AccountCards are visible on
Home; off-Home landings return the carrier and would false-fire the
property, latching always(next(F)) to false and masking the real
double-submit bug. Skip vacuously when route is not "home".
* test(spec): cover route gate in submit-balance predicate
Adds route arg to existing cases (all use "home") and adds five new
cases: ledger landing with stale carrier, add-transaction with
double-insert delta, null route, plus home-landing positive and
double-insert negative cases anchoring the gate's allow path.
|
||
|
|
c76745e5f1 |
WIP: folio refactor - KotlinConf-style production-app shape (#47)
* feat(hierarchy): testTag alias resolves to resource-id and accessibilityIdentifier
Compose's testTag surfaces as resource-id on Android and as
accessibilityIdentifier on iOS. Selectors written as
{ testTag: "Foo" } now match either, so Sanderling specs can use the
same tag on both platforms.
Also rounds out the iOS identifier aliases so resource-id /
identifier / accessibilityIdentifier all resolve to one another.
* chore(folio): add gradle/libs.versions.toml
Centralises versions for all folio modules ahead of the module split.
Adds new entries for kotlinx-serialization, navigation3, Metro, KSP,
and the JetBrains lifecycle-viewmodel-compose multiplatform artifact.
* refactor(folio): introduce nested KotlinConf-style modules
Split the monolithic :composeApp into :core, :app:shared,
:app:ui-components, and :app:androidApp. The old module is still
present and remains the source of truth until the next commits remove
it; both compile in parallel to keep iOS/Android builds green during
the cut-over.
Highlights:
- :core - SQLDelight schema + LedgerStore + Repository (now an
injectable class, not a singleton object). Methods are suspend to
match generateAsync = true.
- :app:ui-components - design system primitives. IconButton/AppButton
APIs revised: label = real contentDescription, testTag = stable
selector. Drops the data-carrier description argument.
- :app:shared - per-screen ViewModels colocated with screens; pure
composables on (state, onEvent); LocalAppComponent CompositionLocal
for hand-rolled DI; @Serializable Route. Hosts the iOS framework
(baseName Shared).
- :app:androidApp - thin Android entry that constructs the
DriverFactory and hands it to App().
- gradle/libs.versions.toml centralises versions; settings.gradle.kts
enables type-safe project accessors.
Deferred to follow-up PRs (per the design discussion):
- Metro DI: hand-rolled AppComponent for now; Metro graphs are mostly
ceremony for an app this size and add KSP/version risk.
- Navigation3: kept the existing Navigator-as-backstack class,
injected rather than singleton; nav3 isn't shipping a stable
multiplatform artifact for commonMain consumption yet.
- :app:webApp + OPFS sqlite worker: web persistence is real new
wiring (custom worker on @sqlite.org/sqlite-wasm). Master's
WebLedgerStore + Snapshot is being removed by this PR; web stays
buildable as a klib but no app-level wasm binary lands here.
* refactor(folio): delete :composeApp and retarget tooling
Removes the old monolithic module now that :core / :app:shared /
:app:ui-components / :app:androidApp own the source. Updates:
- justfile install/uninstall recipes -> :app:androidApp
- iosApp/project.yml framework path -> ../app/shared/...,
baseName Shared (was ComposeApp); pre-build script invokes
:app:shared:linkDebugFrameworkIosSimulatorArm64
- iosApp/iosApp/iOSApp.swift -> import Shared
- README -> mentions SQLDelight unification, drops the
data-carrier contentDescription notes (now stale), no Layout
section per repo convention
* refactor(folio-spec): query testTag and identify items by visible text
Replaces every accessibilityText / descPrefix data-carrier read with
testTag selectors that resolve to resource-id (Android) or
accessibilityIdentifier (iOS) via the SDK's alias table.
- Routes detected via testTag (LoginScreen, HomeScreen, etc.)
- Account identity = visible account name (no synthetic id encoded
in semantics).
- Ledger row identity = joined text content of the row.
- Active account derived from route alone (not parsed from
contentDescription).
- Focused input read from native focused="true" attribute, not from
a custom focused_input data carrier.
* fix(folio): build green on Android assemble + iOS framework link
- Drop ksp/metro/navigation3 plugin aliases - not actually applied
by any module in this PR (deferred follow-up).
- import awaitAsOne from app.cash.sqldelight.async.coroutines for
the suspend single-row reads enabled by generateAsync = true.
- Drop kotlin.js.ExperimentalWasmJsInterop opt-in from common
compilerOptions (it isn't valid for android/jvm targets).
- :app:shared androidMain pulls in androidx.activity:activity-compose
for the BackHandler actual.
* fix(folio): testTagsAsResourceId at App root + JS-bridge regression test
App.kt sets testTagsAsResourceId=true on the root Box semantics so
Compose's testTag surfaces as Android resource-id (and equivalent on
iOS via accessibilityIdentifier). Without this, testTag stays in the
Compose semantics tree but never reaches the runtime hierarchy that
UIAutomator and Sanderling read.
Also adds TestStateAxObjectSelectorTestTagAlias as a regression
test for the {testTag: ...} object selector resolving through the
SDK alias to resource-id at the JS bridge layer.
* test(verifier): expose PredicateError latching across steps
The runner logs PredicateError once per step. The current implementation
latches the first error per thunk, so the log freezes on step 1 forever
even when later steps would observe different errors. This test fails
today and locks in the contract: PredicateError must reflect the most
recent step.
* fix(verifier): refresh predicate errors per step
EvaluateProperties short-circuits once an Always-property latches to
violated, so the underlying goja predicate stops being called and
formula.err keeps whatever it threw at step 1. The runner logs
PredicateError every step a property is violated, which made every
subsequent log line repeat the step-1 throw. That looks like the spec
runtime is seeing stale state, but it is just stale error reporting.
EvaluateProperties now invokes every registered predicate once per step
purely to refresh formula.err. Verdicts are unaffected. The thunk
itself stops latching so the new value wins on whichever path runs first.
* chore(folio): add Metro DI plugin (1.0.0-RC4) to versions catalog
Adds dev.zacsweers.metro plugin alias and applies it to :core
as a smoke test. Compiler-plugin only, no KSP required.
* chore(folio): apply Metro plugin to :app:shared and :app:androidApp
* feat(folio-core): annotate Repository and SqlLedgerStore with @Inject
* feat(folio-core): scope Repository and SqlLedgerStore as @SingleIn(AppScope)
Both are app-wide singletons so the SqlDelight-backed flows remain
shared across the graph.
* feat(folio): annotate ViewModels with Metro @Inject / @AssistedInject
LedgerViewModel and AddTransactionViewModel use @AssistedInject for
their accountId param plus a nested @AssistedFactory; the rest are
plain @Inject constructor classes.
* feat(folio): introduce Metro AppGraph in commonMain
Single shared @DependencyGraph(AppScope::class) that exposes
Repository, Navigator, and ViewModels. LedgerDatabase enters the
graph via @DependencyGraph.Factory.create(database) so the suspend
DriverFactory.create() can stay outside the DI surface.
@Binds wires SqlLedgerStore to LedgerStore; Navigator is provided
explicitly so its Route.Home start state stays in DI rather than
relying on a default-parameter being honored by the graph.
* fix(folio): expect/actual testTagsAsResourceId so iOS link succeeds
Compose's androidx.compose.ui.semantics.testTagsAsResourceId is
Android-only. Calling it directly from commonMain broke
linkDebugFrameworkIosSimulatorArm64. Replace with an expect Modifier
extension that wires the semantics on Android and is a no-op on
iOS / wasmJs.
* refactor(folio): replace AppComponent with Metro AppGraph in App.kt
App now takes a suspend graph builder; the platform constructs
LedgerDatabase off the suspend DriverFactory.create() before invoking
the Metro graph factory. Routes resolve VMs through LocalAppGraph
instead of the hand-rolled LocalAppComponent.
Drops the loading-state placeholder comment (the empty Box is enough).
* refactor(folio): resolve ViewModels through LocalAppGraph in routes
Each *Route composable now reads the AppGraph from CompositionLocal
and pulls its VM via the appropriate accessor or AssistedFactory.
* refactor(folio): build AppGraph from platform entry points
MainActivity (Android) and MainViewController (iOS) now own the
suspend DriverFactory.create() and feed the resulting LedgerDatabase
into Metro's createGraphFactory<AppGraph.Factory>().
* chore(folio): add navigation-compose 2.9.2 dependency
Adds the JetBrains KMP navigation-compose library to the shared
module. Used in subsequent commits to replace the hand-rolled
Navigator with a typed-route NavHost.
* refactor(folio): replace custom Navigator with NavHost backstack
Wraps androidx.navigation.NavHostController behind the existing
push/replace/back surface so call sites in ViewModels stay unchanged.
App.kt now wires a typed NavHost with @Serializable Route entries
and observes the controller's currentBackStackEntry to drive the
session-based Login/Home redirect.
* fix(folio-core): wire kotlinx-browser so wasmJs DriverFactory compiles
org.w3c.dom.Worker on wasmJs lives in kotlinx-browser, not the stdlib.
Pin 0.5.0 alongside the @sqlite.org/sqlite-wasm 3.53.0-build1 version
that the upcoming web app will depend on, and switch the worker
constructor to the module-worker form that webpack expects.
* feat(folio): scaffold :app:webApp wasmJs module
Compose Multiplatform target that depends on :app:shared and pulls
@sqlite.org/sqlite-wasm 3.53.0-build1 as the npm runtime for the
SQLDelight web worker.
* feat(folio-webApp): add main entrypoint and index.html
main.kt mirrors the iOS entry point: builds DriverFactory + AppGraph
factory, hooks browser back-gesture into WebBackGesture, then mounts
the shared App composable into ComposeViewport.
* feat(folio-webApp): OPFS-backed sqlite worker + webpack config
sqlite.worker.js implements the SQLDelight web-worker protocol
(exec/begin_transaction/end_transaction/rollback_transaction) on top
of @sqlite.org/sqlite-wasm. Prefers the OPFS SAH pool VFS for
persistent storage and falls back to in-memory when OPFS is
unavailable.
webpack.config.d/coopcoep.js sends COOP/COEP headers on the dev
server so cross-origin isolation is available, even though the SAH
pool itself does not require it. webpack.config.d/sqlite-wasm.js
enables asyncWebAssembly so webpack can bundle sqlite3.wasm via the
'new URL("sqlite3.wasm", import.meta.url)' reference inside the
sqlite-wasm package.
* chore(folio): add web/web-build just recipes and refresh yarn lock
Yarn lock picks up @sqlite.org/sqlite-wasm 3.53.0-build1.
* fix(folio-core): probe schema before create on wasmJs
Wasm SqlDriver doesn't auto-track user_version like the Android
driver, so awaitCreate() ran on every page load and tripped over
already-created tables. Read PRAGMA user_version, run
awaitCreate/awaitMigrate based on it, and self-heal pre-existing
tables with version 0 by stamping the current schema version.
* chore(folio-webApp): pin dev-server port and trim worker logging
webpack-dev-server now binds 8088 (or WEBAPP_PORT) so it doesn't
collide with the docs server on 8080. Drop the per-message reply
log; keep only the OPFS init line and error logging.
* chore(folio): nest iosApp under app/ for KotlinConf parity
Match KotlinConf-app's filesystem layout where every entry point (android,
ios, web, shared, ui-components) lives under app/. iosApp is still an Xcode
project, not a Gradle module, so settings.gradle.kts is unchanged.
* feat(folio): testTag identity for AccountName and ledger row cells
Replaces string-heuristic identity in the spec extractors with stable
testTags. AccountCard exposes AccountName; LedgerRow exposes TxnNote
and TxnDate. Spec extractors read those directly instead of filtering
visible text by "starts with $" / "matches digit".
* fix(folio-app): branch start destination on initial session
Read repository.session.value at first composition and pick
Route.Home or Route.Login as the NavHost startDestination. Avoids
the one-frame Home flash on cold start with no persisted session.
* refactor(folio-webApp): hard-fail when OPFS unavailable
Drops the silent in-memory fallback. The README claims OPFS
persistence; falling back without surfacing the degrade made data
loss invisible across reloads. Now the worker errors out and the
Kotlin DriverFactory rejects the create() call instead.
* docs(verifier): document extractor advancement and refresh invariants
Extractor previous/current advance only on PushSnapshot, never per
thunk-call. refreshPredicateErrors depends on this for safe re-entry.
Also flags that re-invoked predicates run outside their LTL gate, so
they must be side-effect-free reads.
* fix(hierarchy): populate ResourceID from accessibilityIdentifier
iOS Compose surfaces testTag as accessibilityIdentifier. Previously
only resource-id and identifier seeded element.ResourceID, leaving
element.id empty for iOS Compose nodes and forcing specs to walk
attrs to recover stable identifiers.
* refactor(folio-spec): use element.id for focused field tag
Now that ResourceID populates uniformly across Android/iOS Compose,
the spec can read element.id directly instead of probing attrs for
each platform's underlying field name.
* fix(folio-spec): pick account card via seeded from(), not Math.random
Math.random() breaks --seed reproducibility. The verifier's seeded
RNG flows through from(), so re-running a seed now produces the
same card pick sequence.
* docs(spec): fix README example to use scoped extractors
The previous snippet referenced `state.ax.find` inside an actions()
body where state is not in scope, and shadowed the imported actions
helper with an export of the same name.
* feat(hierarchy): add FindBySelectorPath for chained object selectors
Each selector in the chain is matched within the descendants of the
previous match. Returns the deepest match (or nil) for FindBySelectorPath
and every deepest match for FindAllBySelectorPath.
* feat(verifier): dispatch JS array selectors to FindBySelectorPath
`state.ax.find([{...}, {...}])` now walks each segment scoped under
the previous match. Strings and single objects keep their existing
single-shot lookup.
* feat(spec): expose SelectorPath in find/findAll signatures
* fix(spec): satisfy AccessibilityElement interface in Tap test fixture
* refactor(folio-spec): collapse chained finds into selector paths
* feat(spec): add keyedBy(element, tags) identity helper
Joins element.find({testTag: tag})?.text per tag with U+001F as the
delimiter so user-visible text can never collide with the separator.
Returns empty string for an undefined element.
* refactor(folio-spec): use keyedBy for ledger row identity
* feat(spec): add whenRoute action gating helper
whenRoute(route, allowedRoutes, body) wraps an actions() generator
that returns [] unless route.current matches one of the allowed
values. Accepts a single route or an array.
* test(spec): cover whenRoute matching, gating, and array routes
* refactor(folio-spec): gate addAccount and addTxn with whenRoute
* refactor(hierarchy): use maps.Copy for attribute merge
Linter flagged the manual loop after recent edits surfaced the hint.
* feat(verifier): dispatch setup generator before actions root
Setup is consulted every step; when it returns ErrNoAction the call falls
through to the existing actionGenerator retry loop. This lets specs split
deterministic preconditions (login, onboarding) out of the weighted action
pool while auto-reengaging if state regresses (e.g. logout under fuzz).
* docs(spec): document setup precondition action generator
* refactor(folio-spec): export login as setup, remove from action pool
Login is deterministic and yields no actions once the app is past the
login screen; sitting at weight 50 in the action pool wasted half of step
picks on a no-op. Promote it to setup so the runner only consults it
while it has work to do, and rebalance remaining weights to round numbers
(addAccount 50, addTxn 40, back 10).
|
||
|
|
34fb73d6cb |
fix(folio): stop abusing contentDescription as data carrier (#45)
* feat(hierarchy): full-attribute selector system
- Add Attributes map to Element (raw platform attrs + serialized booleans)
- Add Selector / AttrFilter types for multi-filter AND matching
- Add matchAttr with alias expansion and substring/boolean semantics
- Add matchSelector (AND of all filters)
- id: and desc: keep exact/suffix/prefix semantics for backward compat
- text: widens to substring via matchAttr
- default: case routes unknown kinds to matchAttr (NEW)
- Add Tree.FindNode / Tree.FindAllNodes returning *Node
- Add Node.Find / Node.FindAll for scoped subtree string search
- Add Node.FindBySelector / Node.FindAllBySelector for object AND search
- Add attributeAliases for cross-platform name expansion
* test(hierarchy): full-attribute selector coverage
- raw resource-id: substring match
- label:/content-desc: alias expansion to accessibilityText on iOS
- scrollable:true/false boolean exact match
- title: iOS-only attribute, graceful nil on Android
- text: substring widening
- Selector AND: both filters must match; single miss returns nil
- Node.Find scoped search: descendants only, not siblings
* feat(verifier): object-form selectors + attrs + element-level find
- ax.find/findAll accept string or {attr:value} JS objects
- Object form builds Selector with AND semantics
- Returned element objects expose attrs sub-object (raw platform attrs)
- Returned element objects expose .find() and .findAll() scoped to subtree
- Element-level .find/.findAll accept string or object selectors
* feat(spec): extend AccessibilityElement and AccessibilityTree types
- AccessibilityElement gains attrs, find(), findAll()
- find/findAll on both Tree and Element accept string | AttrSelector
- AttrSelector = Record<string, string> for object-form AND matching
* feat(folio): migrate to chained object-form selectors
- Replace path queries (desc:X > desc:Y) with chained API
- Screen root lookups use { accessibilityText: "ScreenName" }
- Element-scoped searches use find/findAll with string or object
- Keep string selectors for descPrefix: and desc:Back (shows both forms)
* fix(folio): use account_card:id desc, expose balance via text semantics
* fix(folio): embed accountId in LedgerScreen desc, expose values via text semantics
* fix(spec): replace desc-parsing with text-based parseDollarCents extraction
|
||
|
|
6c32fb0e1d |
feat(hierarchy): flat-to-tree + path query selector (#41)
* feat(hierarchy): add Node tree + path query support Preserve parent-child relationships in a Node tree at parse time. Extend Find/FindAll with " > " path operator for scoped queries, e.g. id:LoginScreen > desc:EmailInput. Flat Elements slice and all public signatures unchanged. * test(hierarchy): add path query tests Cover single-level, multi-level, mixed-type, not-found, wrong-subtree, and FindAll across multiple roots. * feat(folio): add modifier param to Screen composable * feat(folio): testTag LoginScreen * feat(folio): testTag HomeScreen * feat(folio): testTag AddAccountScreen * feat(folio): testTag LedgerScreen * feat(folio): testTag AddTransactionScreen * feat(folio): scope spec selectors to screen testTags via path queries * fix(sidecar): align grpc-netty/grpc-okhttp with grpc-core version Maestro 1.40.0 brings grpc-netty:1.50.2 which compiled against AbstractManagedChannelImplBuilder, removed in grpc-core 1.64+. Gradle was upgrading grpc-core to 1.68.0 while leaving grpc-netty at 1.50.2, causing NoClassDefFoundError at startup. Force grpc-netty and grpc-okhttp to match grpcVersion so all gRPC artifacts are binary-compatible. * fix(sidecar): use ephemeral host port for AndroidDriver Hard-coded port 7001 caused TcpForwarder.start() to fail with TimeoutException when a previous sidecar process held the port open. Pick a free port via ServerSocket(0) instead. * chore(sidecar): bump maestro to 2.4.0, exclude graalvm from fat JAR Maestro 2.4.0 still ships grpc-netty:1.50.2 so the grpc transport resolution strategy is kept. GraalVM JS is excluded: unused by our sidecar and causes shadow JAR expansion errors (pom treated as zip). * fix(sidecar): update AndroidDriver calls for Maestro 2.4.0 API launchApp no longer accepts a UUID argument. Third constructor param changed from hostname to emulatorName so drop the "localhost" value. |
||
|
|
2a1b263b8c |
fix: WDA startup flakiness - warmup + connection drop message (#40)
* feat(ios): add simulator management package * feat(testrun): add iOS platform path (simctl launch + direct TCP) * feat(cli): add --ios-device flag and IosDevice option * feat(sdk-ios): add Kotlin Native iOS SDK (TCP agent + POSIX socket + dispatch pauser) * feat(folio-ios): wire SanderlingIos.start() in MainViewController * feat(folio-ios): add test-ios justfile recipe * fix(sdk-ios): remove unavailable C macros; manual byte swap + no-cast warnings * fix(testrun): simctl-first launch order for iOS; Maestro init after SDK connects * feat(proto): add env map to LaunchRequest * feat(driver): add env param to Launch interface + all implementations * feat(testrun): launch iOS app via XCTest with env vars instead of simctl * feat(sidecar): add IosDriverBackend using Maestro IOSDriver + env pass-through * feat(sidecar): wire env map in DriverService + IosDriverBackend in Main * fix(sidecar): use LocalIOSDevice (WDA+simctl) + stop before relaunch * fix(sidecar): include exception type in gRPC error description * fix(sidecar): pick free WDA port instead of hardcoded 9100 Use SocketUtils.nextFreePort to pick a free port in the 22000-23000 range rather than hardcoding 9100, which only worked if a previous WDA session left a listener there. * fix(sdk-ios): check semaphore wait result and throw on snapshot timeout dispatch_semaphore_wait returns nonzero on timeout; ignoring the return value caused pauseAndSnapshot to silently return an empty map, sending a garbage empty STATE frame to the host. Now throws so the agent loop reconnects instead. * fix(folio-ios): register snapshot extractors before starting agent SanderlingIos.start() was called before the snapshot objects were initialized, so a PAUSE message arriving early produced an empty snapshot. Move start() to after all extractors are registered. * chore(ios): remove dead LaunchApp function LaunchApp had no callers since |
||
|
|
d5b6f6ccaf |
feat: Maestro driver integration + DeviceDriver architecture (#32)
* proto(driver): drop launcher_activity from LaunchRequest * refactor(driver): rename Driver to DeviceDriver, drop launcherActivity from Launch * refactor(driver): rename package maestro to sidecar * feat(driver): add ChromeDriver backed by chromedp * feat(hierarchy): replace XML parser with TreeNode JSON parser * refactor(driver): update mock and runner to DeviceDriver, drop launcherActivity * feat(runner): add platform routing for web vs sidecar * test(verifier): update hierarchy fixtures from XML to TreeNode JSON * feat(sidecar): extract readLogcat/readProcMetrics; add MaestroDriverBackend - Extract readLogcat() and readProcMetrics() as internal package-level helpers parameterised by serial - Add MaestroDriverBackend wiring maestro-client AndroidDriver - Drop launcherActivity from DriverBackend interface and StubDriverBackend - Add maestro-utils and micrometer-core as explicit compile deps * refactor(sidecar): inject DriverBackend into DriverService; drop launcherActivity - Remove serial and default backend from DriverService constructor - Make backend a required parameter - Drop launcherActivity from launch RPC handler - Create MaestroDriverBackend in Main.kt when platform is android * test(sidecar): update tests for dropped launcherActivity and required backend * chore(sidecar): remove unnecessary micrometer-core direct dependency |
||
|
|
eea9a0067a | feat(hierarchy): descPrefix selector for Compose testTag + UUID suffixes | ||
|
|
e539e89438 | feat(hierarchy): XML parser and selector resolver for uiautomator dumps |