Commit Graph
38 Commits
Author SHA1 Message Date
pj 69002b07b1 fix(runner,sample-app): surface silent errors and demo a failing property (#15)
* fix(runner): surface non-deadline WaitForIdle errors

Previously the WaitForIdle return value was discarded entirely, hiding
real driver failures (gRPC transport errors, sidecar crashes) behind
the expected deadline-exceeded case. Log non-deadline errors so they
are visible without changing control flow.

* chore(sample-app): drop unused uptime_millis extractor

Registered in SampleApplication but never consumed by spec.ts.

* fix(sample-app): drop trivial appIsRunning property

app_state was hardcoded to 'running' so the property was a tautology
that could never fail. Removing both the extractor and the property
is the simplest fix; demo-grade properties that can fail land next.

* feat(sample-app): add Reset button that zeroes clickCount

Pairs with the next commit's tap-reset action so the fuzzer can
violate clickCountNeverDecreases and demonstrate uatu actually
finding a property violation.

* feat(sample-app): add tap-reset action to exercise Reset button

Weighted at 10/122, fuzzer reaches it within a short run. Pairs with
the Reset button to demonstrate uatu detecting the
clickCountNeverDecreases violation.

* fix(runner): filter WaitForIdle errors via context state, not errors.Is

errors.Is(err, context.DeadlineExceeded) misses gRPC's wrapped
status.DeadlineExceeded, so every step under the maestro driver
logged a spurious warning. Check idleCtx.Err() instead — captures
both deadline-fired and parent-canceled cases regardless of how the
driver wraps them.

* chore(sample-app): tune action weights so demo violates in ~30s

Prior weights left tap-reset rare enough that short demo runs missed
the violation by chance. Bumped to 30/107, with typeUsername reduced
since username noise doesn't help exercise clickCount.

* refactor(runner): route warnings through slog

Adds Options.Logger (defaults to slog.Default()) and converts the
three warning sites that were using fmt.Printf. Progress line stays
on Printf since it's user-facing UI, not a log. Makes the warnings
testable via a capturing handler.

* test(runner): assert WaitForIdle driver errors are logged

Captures slog output via TextHandler into a buffer and asserts the
warning message + injected error text appear when the mock driver
returns a non-context error from WaitForIdle. Guards against a
regression of the silent-error swallow.
2026-04-18 18:48:25 +07:00
pj 00f66ba48d fix: pre-v0.1 review feedback (#14)
* fix(sdk-android): add @JvmOverloads to Uatu.start

Java callers can now invoke start(application) without supplying a
Configuration, matching the Kotlin default-arg ergonomics.

* feat(agent): add protocol_version to HELLO handshake

ProtocolVersion=1 lives on Message and is set by Hello(). Server.Accept
rejects mismatches with a clear error. SDK upgrades that don't change
the wire format keep the same protocol_version; bump on breaking changes.

* test(agent): assert protocol_version in Hello round-trip

* feat(sdk-android): send protocol_version=1 in HELLO

Mirrors agent.ProtocolVersion on the Go side. Bump in lockstep with
the Go constant when the wire format breaks.

* chore(sample-app): pull @uatu/spec from npm next tag

Replaces the file: dep. Copy-paste users can now npm install against
the registry. The release workflow publishes pre-release tags to
npm dist-tag 'next', so the sample tracks the latest rc without
manual version bumps. Lockfile currently resolves to 0.0.1-rc3.
2026-04-18 18:00:12 +07:00
pj d0578dbaaa fix(runner): warn on malformed screen snapshot (#13)
* fix(runner): warn on malformed screen snapshot

screenFromSnapshot swallowed json.Unmarshal errors, so a non-string
screen value silently became "" in the step log and trace while the
verifier still saw the raw JSON. Return the error and warn at the
call site, matching the hierarchy warning pattern.

* docs: clarify --avd is optional for uatu test

The CLI accepts --avd as an empty-string default (cmd/uatu/main.go:49)
and only requires it when no device is connected and multiple AVDs
exist (cmd/uatu/android_env.go:63). Docs and examples that showed it
as required or always-passed were misleading.
2026-04-18 17:14:31 +07:00
pj 16e55086d8 fix(runner): surface focus-tap errors in InputText (#12)
* fix(runner): surface focus-tap errors in InputText action

A failed Tap/TapSelector before InputText was swallowed, so text typed
into the wrong field (or no field) still reported success. Return the
error so the step fails explicitly.

* feat(sample-app): add username EditText and snapshot

Gives the spec a real EditText target (content-desc: username_field)
so the InputText action path can be exercised end-to-end. The typed
value is mirrored into MainActivity.username and surfaced as the
"username" snapshot for spec assertions.

* feat(sample-app): exercise InputText action against username field

Adds typeUsername action and usernameNeverShrinks property to the
sample spec, and extends the integration test to assert the bundled
spec emits an InputText(desc:username_field, "alice") action and that
the property correctly violates when a snapshot reports a shorter
string.
2026-04-18 16:51:35 +07:00
pj b457e22569 refactor(runner): drop hardcoded per-app selectors from step log (#10)
interestingTags hardcoded selectors from a specific app (etMobileNumber,
customer_row_, supplier_row_, etc.) inside the generic runner. None of
these selectors exist in the checked-in sample spec. Debug log now just
reports screen + hierarchy size; specs that want richer visibility can
log from state.ax.find themselves.
2026-04-18 16:41:12 +07:00
pj ae595526da fix(agent): race in readWithDeadline clobbers conn deadline (#7)
* refactor(docs): inline pandoc build into Makefile, drop scripts dir

* fix(agent): wait for deadline watcher before returning

readWithDeadline's watcher goroutine could clobber the conn's read
deadline with time.Unix(1, 0) after the main function reset it to zero.
When the Accept ctx was canceled shortly after Accept returned, the
watcher raced with close(done) in select and sometimes picked ctx.Done()
even though we were already done reading, leaving the conn unusable for
the next read (instant i/o timeout on step 1 snapshot).

Synchronize on the watcher's exit before resetting the deadline so it
can never override the reset.

* test(agent): cover readWithDeadline race on Accept ctx cancel

Drives Accept with a short-timeout ctx, cancels it right after Accept
returns, then does a Snapshot. Reliably fails without the readWithDeadline
synchronization fix (watcher goroutine overwrites the deadline to past).
2026-04-18 14:16:15 +07:00
pj 40c92d4569 test(verifier): rewrite integration tests for sample-app spec
The old tests loaded merchant-android uiautomator dumps from /tmp and
skipped when absent. Replace with two hermetic tests that bundle
examples/sample-app/spec.ts against a synthetic hierarchy: one checks
tapClickMe fires, the other drives the three properties through a
holds/holds/violated snapshot sequence.
2026-04-18 10:33:43 +07:00
pj 0c775c199d feat(examples): add sample-app spec
Introduce examples/sample-app/spec.ts — a minimal property-based spec that
taps the sample app's "Click me" button and asserts click_count is
monotonic. Bundle-check and the trace writer test now reference the new
path.
2026-04-18 10:33:35 +07:00
pj 6e4c832678 chore: stop tracking sidecar JAR (build artifact)
make uatu copies the real fat JAR into assets/ before
go build -tags withsidecar. Keeping that path tracked was
the root cause of the 130 MB push rejection.
2026-04-18 06:46:32 +07:00
pj ae10354ff0 test(sidecar): split tests across stub and withsidecar builds
Default !withsidecar build: assert IsPlaceholder, empty JAR,
Extract errors. withsidecar build: existing extract/checksum
coverage.
2026-04-18 06:46:24 +07:00
pj 4744736dc5 refactor(sidecar): gate JAR embed behind withsidecar build tag
Splits embed.go so the go:embed directive only fires under
-tags withsidecar. Default builds get a stub with a nil JAR
and IsPlaceholder()=true. This removes the landmine where
make uatu overwrote a tracked placeholder file, making any
git add silently stage 130 MB.
2026-04-18 06:46:20 +07:00
pj f2fe54debc test(verifier): verify dismissMultiDevice fires against real confirm dialog 2026-04-18 06:39:00 +07:00
pj 4b43b9a22a chore(runner): log screen + tag hits per step for debuggability 2026-04-18 06:39:00 +07:00
pj cefb398437 feat(verifier): retry NextAction up to 16x so gated generators eventually fire 2026-04-18 06:39:00 +07:00
pj 6c7ea35a7d feat(hierarchy): expose checked/focused/selected on element objects 2026-04-18 06:39:00 +07:00
pj b248ad2e5a test(verifier): integration tests against live uiautomator dumps 2026-04-18 01:59:54 +07:00
pj eea9a0067a feat(hierarchy): descPrefix selector for Compose testTag + UUID suffixes 2026-04-18 01:59:46 +07:00
pj fb6c3ca517 fix(runner): fetch hierarchy before SDK pause to avoid stale uiautomator dumps 2026-04-18 01:59:41 +07:00
pj c4cf0ff530 feat(driver): optional launcher_activity on Launch RPC for multi-alias apps 2026-04-18 01:59:36 +07:00
pj 447fd39363 feat(runner): fetch hierarchy per step and resolve Tap coords 2026-04-18 01:24:00 +07:00
pj 5f2a2d52ab feat(verifier): wire hierarchy into state.ax and carry element coords in Action 2026-04-18 01:23:55 +07:00
pj e539e89438 feat(hierarchy): XML parser and selector resolver for uiautomator dumps 2026-04-18 01:23:51 +07:00
pj e7b3e2ba9c refactor(runner): caller manages app launch/terminate
Removes Launch + Terminate from runner.Run so the CLI can launch
the app first, wait for the SDK to connect, then start the loop.
The previous shape forced runner to launch internally which fought
with the SDK-must-be-connected-first ordering.

BundleID/ClearState fields go away too since runner no longer
launches; the CLI keeps them on its testOptions struct.
2026-04-18 00:51:39 +07:00
pj 7aa75f3d9f feat(bundler): add Aliases option for spec import resolution
Specs import from "@uatu/spec"; the bundler maps that to the
vendored pkg/spec-api/src/index.ts via esbuild's Alias map.
2026-04-18 00:08:31 +07:00
pj 6a9fac7ec0 feat(sidecar): embed sidecar JAR via go:embed
Ships with a 24-byte placeholder so fresh clones build without
requiring a sidecar build first. `make uatu` copies the real
fat JAR into internal/sidecar/assets before `go build`, so
shipping binaries carry the full sidecar (~130 MB).

Extract writes the JAR to a temp dir alongside a SHA-256 file
and skips rewrite when the checksum already matches.
2026-04-18 00:06:27 +07:00
pj 69d0800ea3 feat(driver/maestro): gRPC client implementing driver.Driver
Wraps each v0.1 RPC, plus a WaitForHealth helper that polls until
the sidecar reports Ready=true (used at startup before any other
calls happen). Tests stub the gRPC server in-process so they don't
need a real sidecar JAR.
2026-04-18 00:04:24 +07:00
pj d856c40d7c feat(permissions): grant dangerous permissions via aapt + adb
Inspector parses uses-permission entries from `aapt dump permissions`
and the granter shells out to `adb shell pm grant`. Both are pluggable
so tests can drive logic without aapt or a device. Granter failures
become warnings rather than errors — Android refuses non-runtime
permissions and we'd rather keep going than abort the run.
2026-04-17 23:57:17 +07:00
pj d4a6e33aa6 feat(runner): pause-snapshot-evaluate-resume loop
Wires agent.Conn + driver.Driver + verifier.Verifier + trace.Writer
into the v0.1 step cycle: snapshot the SDK, push to verifier,
evaluate properties, write the trace step (with violations), release
the SDK pause, apply the next action via the driver, wait for idle.

Driver.Launch happens once before the loop and Terminate runs in
defer so even an early error tears down the app cleanly. Summary
returns step count and per-step violation records for the caller
to print or persist.
2026-04-17 23:54:05 +07:00
pj d2fae427f9 feat(driver): add TapSelector and make mock WaitForIdle honor duration
Real maestro driver will resolve selectors itself rather than
forcing the runner to look up coordinates from the hierarchy. Mock
now waits the requested duration so tests don't busy-spin and
starve the SDK fixture goroutine.
2026-04-17 23:54:00 +07:00
pj abeded0b19 test(verifier): cover spec lifecycle and weighted action selection 2026-04-17 23:48:59 +07:00
pj 72175bb13c feat(verifier): goja runtime hosting the spec API
Installs globalThis.__uatu__ with extract, always, actions,
weighted, tap, inputText, and stub taps/swipes. Load runs the
bundled spec, then pulls properties + actions out of globalThis.
PushSnapshot rebuilds state.snapshots and refreshes every
extractor handle's current/previous in registration order so
chained extractors observe up-to-date values.

Properties are wired through internal/ltl as Always(Thunk(...)),
so verdicts latch to violated as soon as a predicate returns
false. NextAction resolves actions/weighted recursively with a
seedable rand source for reproducible runs.
2026-04-17 23:48:58 +07:00
pj 72c4e4c4d3 feat(trace): JSONL writer for steps + meta + screenshots
Each WriteStep appends one JSON object per line so the trace can
be filtered with jq directly. Screenshots land under screenshots/
with zero-padded indexes. Writer is concurrency-safe; Close is
idempotent and a write after Close errors loudly rather than
silently dropping.
2026-04-17 23:43:45 +07:00
pj 0c277b2833 feat(ltl): formula AST and step evaluator for v0.1
Supports Always over Pure/Thunk leaves; eventually/next/bounds
deferred. Once a thunk returns false under an Always, the verdict
latches to violated so the runner can surface the offending step
without later observations masking it.
2026-04-17 23:42:44 +07:00
pj cee90694da feat(bundler): esbuild wrapper for spec compilation
Bundles a TypeScript entry into an IIFE ES2020 blob with optional
inline sourcemaps. process.env defines are JSON-quoted before being
fed to esbuild so values with quotes/backslashes survive correctly.
Returns the bundle's SHA-256 for trace meta.json.
2026-04-17 23:41:43 +07:00
pj 6353afdf83 feat(driver/mock): in-memory Driver for unit tests
Records every call as an Action and lets tests program hierarchy,
screenshot, health, and per-method failures. Actions() returns a
copy so test code can't mutate the driver's history.
2026-04-17 23:40:28 +07:00
pj d533aef04b feat(driver): Driver interface for v0.1 RPC surface
Mirrors proto/driverpb/driver.proto: Launch, Terminate, Tap,
InputText, Hierarchy, Screenshot, WaitForIdle, Health. Keeps
Image/Health as dedicated structs so callers don't depend on
generated proto types.
2026-04-17 23:40:28 +07:00
pj b9511c0718 feat(agent): socket server with PAUSE/STATE/RESUME flow
Accept waits for an SDK HELLO then hands back a Conn. Conn.Snapshot
sends a PAUSE, blocks on the matching STATE (id-correlated), and
leaves the SDK paused until Release sends RESUME. Conn.Close sends
GOODBYE best-effort.

v0.1 supports one client at a time; transport is left to the caller
so tests can use TCP loopback while production wires via adb reverse
to localabstract:uatu-agent.
2026-04-17 22:50:54 +07:00
pj ca92bb1492 feat(agent): Go wire protocol with length-prefixed JSON framing
Six message types: HELLO, PAUSE, RESUME, STATE, EXTRACT_RESULT,
GOODBYE. 4-byte big-endian length + JSON payload. 16MB frame cap.
Typed constructors per message; snapshots carry json.RawMessage so
downstream goja evaluation sees exact types.
2026-04-17 22:48:07 +07:00