Commit Graph
14 Commits
Author SHA1 Message Date
pj 323878c34a fix(verifier): don't crash on throwing JS predicates (#21)
* fix(verifier): don't crash on throwing JS predicates

formulaThunk used to panic whenever goja returned an error from a
predicate callable, and nothing on the LTL -> runner path recovered, so
a malformed spec (e.g. a property whose body throws or touches an
undefined field) would kill the verifier process.

Latch the first error on formulaState, return false so LTL marks the
property violated, and expose PredicateError(name) that walks the
property's formula-spec tree and surfaces the latched cause.

* fix(runner): log predicate errors alongside violations

For each violated property, surface the verifier's latched predicate
error via logger.Warn so operators can distinguish a genuine false
verdict from a malformed spec. Add a runner-level test asserting that a
throwing predicate no longer crashes the run and that the error message
appears in the log.
2026-04-20 16:55:10 +07:00
pj a2e96af1af WIP: rename sample app to Folio (#20)
* refactor: rename examples/sample-app to examples/folio

Directory-level rename and path references in Go tests, bundle-check,
top-level README, and getting-started docs. Package declarations,
Gradle config, iOS bundle IDs, and class names follow in later commits.

* refactor(folio): rename Kotlin package dev.uatu.sample to app.folio

Moves source dirs and sqldelight schema from dev/uatu/sample to
app/folio, updates package declarations and imports, and switches
Android namespace/applicationId, iOS binaryOption bundleId, and
sqldelight database packageName to the new identifier.

* refactor(folio): rename SampleApplication to FolioApplication

Android manifest now points at .FolioApplication with label 'Folio'
instead of 'Uatu Sample'.

* refactor(folio): set iOS bundle id and display name to Folio

bundleIdPrefix + PRODUCT_BUNDLE_IDENTIFIER -> app.folio.
CFBundleName + CFBundleDisplayName -> 'Folio'.

* refactor(folio): update demo email to [email protected]

* refactor(folio): point justfile at app.folio bundle id

Updates xcrun simctl launch target, uatu test --bundle-id, and the
build/uninstall comments to reference folio instead of sample.

* test: update fixture package ids to app.folio

Sidecar activity-resolver test and verifier spec-integration XML
fixtures referenced the old dev.uatu.sample Android package. Updates
them to match the folio app's real package id so the tests stay
representative of what the CLI sees on-device.

* test(verifier): rename SampleApp identifiers to Folio

Renames TestSampleAppSpec* functions, bundleSampleAppSpec helper, and
sampleAppHierarchyXML const (now loginHierarchyXML for consistency with
the other per-screen fixtures). Updates trailing sample-app mentions in
comments and assertion messages.

* refactor(folio): rename Gradle/npm/wasm project identifiers to folio

settings.gradle.kts rootProject.name, package.json + package-lock.json
name, and the WasmJS index.html <title> all still read 'uatu-sample' /
'Uatu Sample'. Realigns them with the Folio brand.

* docs(folio): rewrite README title + getting-started bundle id

examples/folio/README.md is now titled 'Folio' with the Kotlin source
paths corrected to app/folio. Getting-started example uses --bundle-id
app.folio. Harness launch message is now generic ('app under test')
since uatu-sample-harness is not specific to folio.

* chore(folio): drop trailing 'sample' reference in gradle.properties

* refactor(folio): rename LoginPage composable to LoginScreen

Align with KMP/Android industry convention (NowInAndroid, Cash App,
JetBrains samples use Screen, not Page).

* refactor(folio): rename HomePage composable to HomeScreen

* refactor(folio): rename AddAccountPage composable to AddAccountScreen

* refactor(folio): rename LedgerPage composable to LedgerScreen

* refactor(folio): rename AddTransactionPage composable to AddTransactionScreen

* refactor(folio): split Models.kt into app.folio.data package

Account, Transaction (with TxnType), and Session move into their own
files under app.folio.data, matching NowInAndroid-style per-type
organization.

* refactor(folio): move data layer into app.folio.data package

Repository, LedgerStore (expect + interface), SqlLedgerStore,
WebLedgerStore, DriverFactory (expect + actuals), AndroidLedgerContext,
and Snapshot move into app.folio.data. Update all consumer imports.

* refactor(folio): move Navigation into app.folio.navigation package

Split the former Navigation.kt into Route.kt (sealed interface) and
Navigator.kt (singleton). Update consumer imports across screens,
App.kt, and FolioApplication.

* refactor(folio): move Platform and Format into app.folio.platform

Both files carry expect declarations (Platform object, formatDate);
grouping them into a dedicated platform package makes the KMP seam
obvious and mirrors the structure used by JetBrains samples.

* refactor(folio): move login into feature/auth package

Create app.folio.feature.auth with LoginScreen + LoginUiState. Inline
the former Auth.kt (DEMO_EMAIL, DEMO_PASSWORD, checkCredentials) into
LoginScreen since it is the sole caller.

* refactor(folio): move HomeScreen into feature/home package

* refactor(folio): move account creation into feature/account package

AddAccountScreen gets its own AddAccountUiState colocated with the
screen, replacing the shared UiState.addAccountError.

* refactor(folio): move ledger screens into feature/ledger package

LedgerScreen and AddTransactionScreen move into app.folio.feature.ledger
with AddTransactionUiState (txnError, txnFormType) colocated. The
former catch-all UiState.kt is removed now that each screen owns its
state alongside its UI.

* refactor(folio): split Theme.kt; move theme and icons to subpackages

Theme split into Theme.kt (tokens, layout dims, LedgerTheme) and
Type.kt (typography) under app.folio.ui.theme. Icons moves to
app.folio.ui.icon. Update every consumer's imports to match.

* refactor(folio): split ui components into per-file under ui/component

Former Widgets.kt and Components.kt become 10 focused files: AppButton,
Card, EmptyState, ErrorText, FieldLabel, Header, IconButton (w/
BackButton), Screen, Segmented, TextInput. Matches NowInAndroid style
of one composable per file in a designsystem/component package.

* chore(folio): consolidate uatu testing files under uatu/ folder

Move spec.ts, package.json, package-lock.json into examples/folio/uatu
so all uatu-specific testing artifacts live in one place. runs/ and
node_modules/ follow the same convention (both remain gitignored).
Update justfile, README, and the two Go consumers (bundle-check tool +
verifier/trace tests) that referenced the old path.

* refactor(trace): drop folio path in writer test

Round-trip only needs a non-empty string; neutralize to keep the
library free of folio references.

* refactor(sidecar): neutralize ResolveActivity test fixtures

Swap app.folio for com.example.app in the fixture strings so the
sidecar tests don't reference the example app by name.

* refactor(bundle-check): take spec path as argument

Previously the tool hardcoded examples/folio/uatu/spec.ts. Accept a
positional spec path instead so the tool works for any example and
leaves no folio reference in the library surface.

* test(verifier): add neutral integration spec and hierarchy fixtures

Adds testdata/integration_spec.ts with two routes ("list", "form"),
an InputText on text_field, a Tap on primary/secondary_action, a
safety property (itemCountNonNegative), and a liveness property
(submitEventually). Adds hierarchies_test.go with matching XML
fixtures. Constants intentionally go in a _test.go at package root
rather than testdata/hierarchies.go because go skips .go files
under testdata/.

* refactor(verifier): replace folio integration tests with neutral ones

Renames bundleFolioSpec -> bundleIntegrationSpec and the three Test*
entry points to TestIntegrationSpec*. Uses the synthetic spec and
hierarchies added in the previous commit so the library's test suite
no longer references examples/folio at all.

Folio-specific coverage remains covered by examples/folio/justfile's
'just test' which exercises the real spec on device/emulator.

* chore: remove cmd/uatu-sample-harness

Not referenced by Makefile, docs, CI, or any script. Duplicates the
adb reverse helpers already in cmd/uatu/test_run.go, and its name
implies ownership by the sample app which violates the library/
example decoupling. If a bare-protocol debugging tool is later
needed it belongs inside cmd/uatu/.

* docs(folio): drop Layout section and KMP layout paragraph; fix AVD override syntax

The directory-tree Layout section rots faster than the code and
duplicates what ls shows for free. The expect/actual paragraph in
Stack was the same kind of filler. The README also claimed 'just
AVD=Pixel_7 test' but the justfile reads AVD as an env var via
env_var_or_default, so the correct invocation is 'AVD=Pixel_7
just test'.
2026-04-20 16:04:20 +07:00
pj c1de4bf57a fix(sample-app): address PR #18 review findings (#19)
- WebLedgerStore.accountExistsByName case-insensitive (matches SQL COLLATE NOCASE)
- drop SampleApplication.maybeInjectDebugError; sample spec now passes clean
- drop noLogcatErrors from sample spec (default matches system-wide E logs)
- openRandomAccount picks uniformly from findAll instead of first match
- clear txnError on any add-transaction interaction, not only valid amount input
- escapeForAdbInputText quotes shell metacharacters (quotes, backslash, etc.)
- extract buildClearKeyevents helper with empty/normal/cap tests
- broaden spec_integration_test.go to cover home, add-account, ledger,
  add-transaction action generators
- document FocusTracker single-focus invariant
2026-04-20 13:31:50 +07:00
pj 8381a98aaf feat(sample-app): ledger spec parity (#18)
* feat(sample-app): add FocusTracker

* feat(sample-app): support description on TextInput

* feat(sample-app): support description on AppButton and Segmented

* feat(sample-app): stable ids on login screen

* feat(sample-app): stable ids on add-account screen

* feat(sample-app): stable ids on add-transaction screen

* feat(sample-app): stable ids on home and ledger screens

* feat(sample-app): hoist error + form state into UiState

* feat(sample-app): register auth_status + accounts snapshots

* feat(sample-app): register ledger_rows + ledger_balance snapshots

* feat(sample-app): register error + focused_input snapshots

* refactor(sample-app): scaffold spec.ts extractors + safety

* feat(sample-app): spec accounting invariants

* feat(sample-app): spec state-machine monotonicity

* feat(sample-app): spec liveness properties

* feat(sample-app): spec auth + account action generators

* feat(sample-app): spec transaction action generators

* feat(sample-app): spec weighted workflow

* feat(sample-app): publish login form input snapshots

* feat(sample-app): publish account + txn input snapshots

* feat(sample-app): register form input snapshots

* fix(sample-app): sequence login + idempotent text inputs in spec

* fix(sample-app): clear UiState on page dispose

* fix(sample-app): tighten adversarial login, allow retype on error

* test(verifier): update sample-app spec integration for new selectors

* feat(sidecar): clear focused field before InputText types

* refactor(sample-app): simplify loginHelper to focus-driven sequencing

* refactor(sample-app): drop input-value UiState mirrors

* refactor(sample-app): drop input-value snapshot registrations

* test(verifier): adjust sample-app integration for replace-on-type
2026-04-20 13:28:41 +07:00
pj 7493945251 feat: LTL operators, sampling, and default generators (#17)
* feat(ltl): add Now/Next/Eventually/Implies/Or/And/Not formulas

Replace the fold-with-latch evaluator with a residual-formula reducer.
Each Observe() instantiates a fresh obligation from the root (stripping
an outer Always), reduces each pending obligation against current state,
latches Violated on first failure, and surfaces Pending verdicts for
deferred obligations. Existing Always/Pure/Thunk tests continue to pass.

* feat(ltl): support relative duration for eventually().within()

* feat(proto): add Swipe, PressKey, RecentLogs RPCs

* feat(verifier,runner): formula handles, new action kinds, rich state

- verifier: add formula-spec registry; bindNow/bindNext/bindEventually with
  chainable .implies/.or/.and/.not and .within(n,unit) on eventually; bindFrom
  for uniform sampling. bindAlways keeps accepting plain predicates.
- verifier: store lastTree, lastAction, step time, logs, exceptions on the
  Verifier; SnapshotInput replaces the (snapshots, tree) pair. stateObject now
  produces state.lastAction/time/logs/exceptions matching the TS State type.
- verifier: make taps/swipes/waitOnce/pressKey built-in generators actually
  fire; taps picks a clickable, enabled element from the last hierarchy.
- agent: add exceptions field to Message wire format.
- driver: add Swipe/PressKey/RecentLogs to Driver interface; wire maestro
  client and mock driver. LogEntry exposed for runner consumption.
- runner: apply Swipe/PressKey/Wait actions; collect logcat and exceptions;
  pass lastAction and step time into PushSnapshot.

* feat(spec-api): LTL operators, new actions, richer State

- ltl.ts exports now/next/eventually; always overload accepts a Formula
- types.ts: Formula gains implies/or/and/not; EventuallyFormula adds .within;
  State gains lastAction/time/logs/exceptions; Swipe/PressKey/Wait action types
- actions.ts: Swipe/PressKey/Wait/from constructors; waitOnce + pressKey
  default generators
- tests exercise the chaining, sampling, and new actions through a recorded
  fake runtime

* feat(sidecar): add swipe, pressKey, recentLogs RPC handlers

* feat(sdk-android): capture uncaught exceptions

Install a default uncaught handler on Uatu.start, chained with any
existing handler so Android's crash reporter still runs. Expose
Uatu.reportError for callers to forward caught throwables. A bounded
circular buffer (default 50) drains into each STATE message's new
exceptions field. Protocol.kt serializes/deserializes the field,
matching the Go wire format added to internal/agent/protocol.go.

* feat(spec-api): add @uatu/spec/defaults/properties bundle

* feat(sample-app): exercise new LTL operators + defaults

spec.ts now imports eventually/next/now/from from @uatu/spec and
noUncaughtExceptions from @uatu/spec/defaults/properties. It declares
three properties that exercise the new surface:

- accountCountNonNegative: plain always() safety
- addAccountAdvances: always(now(x).implies(next(y)))
- eventuallyLoggedIn: eventually(p).within(30, "seconds")
- noUncaughtExceptions: imported default

The weighted actions root uses from() for random phone/name sampling
and entries for taps/swipes/waitOnce/pressKey built-ins.

SampleApplication gains a debug hook gated on the system property
uatu.inject_error so the e2e run can synthesize an Uatu.reportError and
verify noUncaughtExceptions violates.

cmd/uatu/test_run.go adds a subpath alias so specs importing
"@uatu/spec/defaults/properties" resolve against the in-tree source
when running from the uatu checkout. The spec-integration tests swap
the old click-counter fixtures for the new login hierarchy.

* feat(trace): record swipe/key/wait details + exceptions

trace.Step gains an Exceptions array so the trace captures the
class/message/stackTrace for each SDK-reported throwable in a step.
trace.Action gains FromX/FromY/ToX/ToY/Key/DurationMillis so the full
payload of Swipe/PressKey/Wait actions is visible in trace.jsonl.

sample-app's debug error hook now gates on ApplicationInfo.DEBUGGABLE
instead of a system property (adb setprop fails on non-rooted
emulators).
2026-04-20 02:19:39 +07:00
pj 16e55086d8 fix(runner): surface focus-tap errors in InputText (#12)
* fix(runner): surface focus-tap errors in InputText action

A failed Tap/TapSelector before InputText was swallowed, so text typed
into the wrong field (or no field) still reported success. Return the
error so the step fails explicitly.

* feat(sample-app): add username EditText and snapshot

Gives the spec a real EditText target (content-desc: username_field)
so the InputText action path can be exercised end-to-end. The typed
value is mirrored into MainActivity.username and surfaced as the
"username" snapshot for spec assertions.

* feat(sample-app): exercise InputText action against username field

Adds typeUsername action and usernameNeverShrinks property to the
sample spec, and extends the integration test to assert the bundled
spec emits an InputText(desc:username_field, "alice") action and that
the property correctly violates when a snapshot reports a shorter
string.
2026-04-18 16:51:35 +07:00
pj 40c92d4569 test(verifier): rewrite integration tests for sample-app spec
The old tests loaded merchant-android uiautomator dumps from /tmp and
skipped when absent. Replace with two hermetic tests that bundle
examples/sample-app/spec.ts against a synthetic hierarchy: one checks
tapClickMe fires, the other drives the three properties through a
holds/holds/violated snapshot sequence.
2026-04-18 10:33:43 +07:00
pj f2fe54debc test(verifier): verify dismissMultiDevice fires against real confirm dialog 2026-04-18 06:39:00 +07:00
pj cefb398437 feat(verifier): retry NextAction up to 16x so gated generators eventually fire 2026-04-18 06:39:00 +07:00
pj 6c7ea35a7d feat(hierarchy): expose checked/focused/selected on element objects 2026-04-18 06:39:00 +07:00
pj b248ad2e5a test(verifier): integration tests against live uiautomator dumps 2026-04-18 01:59:54 +07:00
pj 5f2a2d52ab feat(verifier): wire hierarchy into state.ax and carry element coords in Action 2026-04-18 01:23:55 +07:00
pj abeded0b19 test(verifier): cover spec lifecycle and weighted action selection 2026-04-17 23:48:59 +07:00
pj 72175bb13c feat(verifier): goja runtime hosting the spec API
Installs globalThis.__uatu__ with extract, always, actions,
weighted, tap, inputText, and stub taps/swipes. Load runs the
bundled spec, then pulls properties + actions out of globalThis.
PushSnapshot rebuilds state.snapshots and refreshes every
extractor handle's current/previous in registration order so
chained extractors observe up-to-date values.

Properties are wired through internal/ltl as Always(Thunk(...)),
so verdicts latch to violated as soon as a predicate returns
false. NextAction resolves actions/weighted recursively with a
seedable rand source for reproducible runs.
2026-04-17 23:48:58 +07:00