Commit Graph
126 Commits
Author SHA1 Message Date
pj 4406d3b5ab test(ios): name the picker test for what it covers 2026-08-16 01:08:51 +05:30
pj cb45508e72 test(ios): a device driver can bring a runner up 2026-08-16 01:05:59 +05:30
pj 01f9ea942c fix(ios): one address picker for every bring-up
bringUpRunner reads the picker from a field, and NewDevice only ever set
the device one, so a device driver that reached bringUpRunner would call
nil. The two fields held the same function; keeping one leaves no path
that can be wired without it.
2026-08-16 01:05:59 +05:30
pj d124354379 test(ios): a clear-state launch for an uncleared bundle is refused 2026-08-16 01:04:58 +05:30
pj 6beea91046 fix(ios): the clear-state guard checks the bundle that was cleared
A bool only said that something was cleared, so Launch(ctx, otherBundle,
clearState=true) passed the guard and reported a reset that had reached a
different app. Record what was cleared and compare against the bundle
being launched.
2026-08-16 01:04:58 +05:30
pj 99b9fe23f5 test(ios): a device clear-state without an app path ends the run 2026-08-16 01:03:48 +05:30
pj 220c7fe1d7 fix(ios): a device clear-state that cannot happen must fail
--clear-data on a physical device with no --ios-app-path warned and then
ran anyway, so the run started on the previous run's data while the flag
said it started clean. There is no data-container wipe on a device, so
there is nothing to fall back to.
2026-08-16 01:03:48 +05:30
pj 9ede9dc3e2 test(ios): pin the stop that has to precede a clear
The ordering probe now records the stop, and a scripted xcrun holds what
reaches the tool: terminate before get_app_container, with the previous
run's files gone after. A simctl terminate that finds nothing to stop
still leaves the clear a success.
2026-08-16 01:03:02 +05:30
pj 119877c78a fix(ios): stop the app before clearing its state
Launch terminated and then cleared; the clear moved to construction and
left nothing stopping the app first. The container wipe deletes files a
live app still holds open, and the CI ios leg passes no app path so the
wipe is the path it takes. simctl stops it, since the clear now runs
before any automation session exists. On a device the uninstall that is
its only clear takes the running app with it.
2026-08-16 01:02:57 +05:30
pj f911079e32 test(ios): drive the launch recovery with what the transports return
The wedged-session fake answered with ctx.Err() raw, which is the one
shape the guard already matched. The recovery now runs against the error
each transport really produces for the same expiry, taken from a runner
and a legacy companion that never answer.
2026-08-16 01:00:49 +05:30
pj f1a5db886e fix(ios): recognise every shape a blown launch bound arrives in
The runner transport reports a blown budget two ways, its own comment says
so: the context's error once cancellation has landed, and the connection's
i/o timeout when the deadline armed from that context fires first. The
legacy transport reports it as a gRPC status. errors.Is against
context.DeadlineExceeded only matches the first, so the session restart
never fired for the other two and a wedged session stayed wedged.
2026-08-16 01:00:45 +05:30
pj ea952378fe test(android): name the uninstall failure for what it says, not why 2026-08-16 00:07:58 +05:30
pj 87aa7d9284 docs(android): say why the uninstall text cannot be read 2026-08-16 00:07:12 +05:30
pj 6234bf5ba0 fix(ios): a failed devicectl uninstall must fail the reinstall
same hole as the simulator path: devicectl install over an app keeps its data, and the discarded uninstall error hid it. Uninstalling a bundle id that is not installed exits 0 with 'App uninstalled.' on a paired iPhone, so a failure here is always real.
2026-08-16 00:02:36 +05:30
pj abc5e7db3e fix(ios): a failed simctl uninstall must fail the reinstall
simctl install over an installed app carries its data container across, so discarding the uninstall error reported a clear-state that never happened. Uninstalling an app that is not installed exits 0 on a booted simulator, so every failure here is a real one.
2026-08-16 00:01:40 +05:30
pj 1ec6eb3847 fix(android): a refused uninstall must not pass for clear-state
adb uninstall answers Failure [DELETE_FAILED_INTERNAL_ERROR] both when the package was never installed and when it refuses to remove one, so the failure text cannot say which happened and the old code installed over the top either way, keeping the data clear-state was asked to drop. Ask pm path instead, and fall back to pm clear when the app is still there.
2026-08-16 00:00:36 +05:30
pj 48e901bd9b Merge branch 'android-home-diagnostic' into trial-merge 2026-08-15 22:51:48 +05:30
pj e61b39f2e3 test(testrun): cover the sidecar shutdown path after an early exit 2026-08-15 22:46:47 +05:30
pj 708a8a2929 fix(testrun): report a sidecar that dies at startup as the exit it was 2026-08-15 22:42:26 +05:30
pj f819dad7b6 fix(testrun): preflight resolves adb through the sdk, not just PATH 2026-08-15 22:39:38 +05:30
pj a5f86334af fix(android): say what the sdk lookup checked, not just to set ANDROID_HOME 2026-08-15 22:36:50 +05:30
pj b415323b08 Merge branch 'relaunch-signal-and-badge-reach' into trial-merge 2026-08-15 21:52:19 +05:30
pj cc2e0ecc35 docs(runner): describe both modes of the composing test driver 2026-08-15 21:46:30 +05:30
pj 7d4e85f407 refactor(runner): drop the empty branch from the hold path 2026-08-15 21:44:21 +05:30
pj dba9da8575 Merge branch 'ios-clear-before-session' into trial-merge 2026-08-15 21:43:31 +05:30
pj 5aee168098 fix(runner): hold the action back on a step nothing verified 2026-08-15 21:42:48 +05:30
pj 9f8afab9c0 test(runner): a skipped step must not swallow the action before it 2026-08-15 21:42:48 +05:30
pj fef80e00ff fix(testrun): thread clear-data into the ios drivers 2026-08-15 21:31:53 +05:30
pj 67da9313dd fix(ios): the device path clears before its runner session too 2026-08-15 21:31:53 +05:30
pj aed65976e3 fix(ios): clear app state before the automation session attaches
New performs the clear-state reset, so the uninstall and reinstall no
longer land underneath a live XCTest session that is already bound to
the app. Launch refuses a clear-state request the driver was not built
for rather than reinstalling under its own session.
2026-08-15 21:31:48 +05:30
pj 985c6befdd test(runner): cover the reread's cost to the existing snapshot rules 2026-08-15 21:30:21 +05:30
pj 81b737188f feat(runner): skip a step whose tree changed between two reads 2026-08-15 21:30:21 +05:30
pj 2762dc3011 test(runner): answer Snapshot and Hierarchy off one tree in the fakes 2026-08-15 21:30:15 +05:30
pj 5b218edb92 Merge branch 'companion-honest-launch' into trial-merge 2026-08-15 21:27:48 +05:30
pj 79dbc471b8 Merge branch 'spec-parity-holes' into trial-merge 2026-08-15 21:27:47 +05:30
pj e686362fb6 docs(runner): point the guard comment at the renamed test 2026-08-15 21:03:43 +05:30
pj 94a1d59d60 test(runner): an overlay dismissal must not convict the counting property 2026-08-15 21:03:23 +05:30
pj 4324573d97 test(verifier): both ax.find selector forms resolve the same element 2026-08-15 20:51:43 +05:30
pj ad63c5eb74 fix(hierarchy): object selectors resolve by the same rule as string ones 2026-08-15 20:50:55 +05:30
pj a7e94c85df test(ios): a refusal the runner names costs no session restart
The session restart is for a launch that never answers. A launch that
reports the app's state has already said what a fresh session would.
2026-08-15 20:49:35 +05:30
pj 56d6aa53f1 test: pin that a nested undefined does not survive the wire 2026-08-15 20:42:36 +05:30
pj ca969f495a fix(runner): keep the action the app was relaunched after 2026-08-15 20:42:20 +05:30
pj 99160b4c12 test(verifier): pin the relaunch field on both hosts 2026-08-15 20:42:20 +05:30
pj 365418f73e feat(verifier): report a relaunch on state.lastAction 2026-08-15 20:42:20 +05:30
pj 46f2969350 test(runner): a relaunch must not convict the submit counting property 2026-08-15 20:38:17 +05:30
pj 0929ae54bf test(ios): the restart a blown launch triggers has to be bounded 2026-08-15 19:31:24 +05:30
pj f073f2cc74 fix(ios): share one deadline across the restart and the second launch
the recovery a blown bound triggers now costs at most launchRecoveryTimeout
whatever it spends it on, so the launch path tops out at 150s and testrun's
three minute backstop stays a backstop.
2026-08-15 19:31:24 +05:30
pj 8baea14245 test(ios): cover the session replacement a wedged launch needs 2026-08-15 19:24:27 +05:30
pj d64781ce39 fix(ios): replace the session when a launch blows its bound
a launch the simulator refuses is never reported: xctest records it as a test
failure the runner cannot see, then holds the session's main thread for about
four minutes on a diagnostic chain. so the only signal is the expired bound,
and every later call queues behind the same wedge. restart the session once and
launch again, bounded so the launch path stays inside testrun's backstop.
2026-08-15 19:24:23 +05:30
pj 11f72a722a follow-ups from the pr #73 review (#77)
* ci(folio): run gradle on jdk 21 for the metro plugin

the metro gradle plugin folio builds with publishes org.gradle.jvm.version 21
and java 21 class files, so every leg failed at the folio build on a 17
runtime. local builds pass on jdk 25, which is why only ci saw it.

* fix(build): clean pkg/spec/dist, not the dead spec-api path

* chore: point stale spec-api comments at pkg/spec

* fix(spec): publish src so an installed package carries the runtime entries

* fix(testrun): alias the installed spec package so one module graph loads

* fix(spec): export Direction, ScrollAction and LongPressAction from the entry

* docs(spec): cut the package readme to a description and doc links

* docs: say how the cli and spec package versions relate

* fix(verifier): report whether the last action was confirmed applied

Both hosts get applied: true when the runner saw the dispatch succeed and
applied: null when it could not, so an unconfirmed action stops arriving at
the spec as no action at all.

* fix(runner): an apply error leaves the action's fate unknown, not undone

A deadline that fires after the tap was dispatched leaves the effect
committed. Reporting nil made the spec see an effect with no action to cause
it, which is how the counting property convicts a healthy app.

* fix(release): stage the sidecar jar at the renamed embed path

* test(replay-ui): trace fixtures for the vacuity counts

one real green run, one run that rendered nothing, one that judges every property at least once.

* ci(replay-ui): count the steps each property judged

the exit code says no property returned false; it does not say any property was ever evaluated. this reads the trace and reports judged vs declined per property, and fails when the step page never rendered.

* test(replay-ui): cover the summary script from make test

* ci(replay-ui): summarise through the vacuity script

* docs(ci): explain the replay-ui judged/declined counts

* fix(verifier): encode element-valued extractors into the trace

An ax element exports with its find/findAll host functions attached, and
json.Marshal refuses the whole value over them: json: unsupported type:
func(goja.FunctionCall) goja.Value. The encoding failed, curr stayed nil,
and the goja hosts (ios, android) recorded null for every element-valued
extractor in both the per-step diff and the violation witness.

Apply the web host's sanitize rule before marshaling, so one rule encodes
an element on both hosts.

* test(verifier): pin element encoding to one rule on both hosts

* test(runner): assert an element reaches trace.jsonl and its witness

* feat(spec): give state.lastAction an applied field

Three states, not two: no action is a null lastAction, applied: true is an
action the runner confirmed, applied: null is one it dispatched and never
learned the fate of.

* fix(folio): do not attribute an effect to an unconfirmed action

submitChangesBalanceByTypedAmount and createdAccountHasNonZeroBalance both
convict by pinning an effect on the last action, so both decline unless the
runner saw it applied. The fixtures now say which fate they mean.

* test(folio): an unconfirmed submit belongs in the window

The count is an upper bound on the submits a window holds, so the tap that may
have landed counts and committedTransactionsExceedSubmits has nothing to
convict on.

* test(runner): a tap that lands under a failed apply is not a double submit

Drives the real folio counting predicates through the runner against a device
that commits the tap and then times out. The double-submit case is the control:
without it a green proves only that the property never fired.

* test(verifier): pin the three lastAction states on both hosts

The web page is handed the same applied field the goja object exposes, so a
property cannot read one thing on native and another on web.

* docs(spec-language): document the three lastAction states
2026-08-15 15:51:33 +05:30