pj
e62319e916
* chore(prose): remove em-dashes from config files * chore(prose): remove em-dashes from android sdk config * docs(spec-api): remove em-dash from README * fix(doctor): reword sidecar-jar error without em-dash * test(sidecar): reword assertion message without em-dash * docs: add CLAUDE.md with project conventions * build: add docs target for pandoc site * docs(site): add pandoc template and stylesheet * docs(site): add pandoc build script * docs(site): add landing pages * docs(manual): add getting-started * docs(manual): add writing-specs * docs(manual): add runs * docs(manual): add cli reference * docs(dev): add design principles * docs(dev): add architecture * ci: deploy docs site to github pages * docs: rewrite README as entry point to docs site
2.7 KiB
title
| title |
|---|
| Runs |
What is a run?
One uatu test invocation. Fresh install, spec-driven exploration, then the trace lands in runs/<timestamp>/. Typically minutes to hours, not seconds.
A run is not analogous to a unit test. A closer framing is: boot a fuzzer for an hour and see what breaks. Violations are recorded in the trace and exploration continues, so one run can surface many bugs.
Lifecycle
uatu test --spec spec.ts --bundle-id com.example.app --avd Pixel_7 --duration 30m
│
├── uninstall and reinstall the app (clean slate, every run)
├── boot the sidecar, connect the agent socket
├── bundle the spec, load it into goja
│
├── step 0..N: pause, capture state, evaluate properties, pick action, resume, dispatch
│
└── terminate when --duration elapses (or SIGINT)
└── trace written to ./runs/<timestamp>/
├── trace.jsonl
├── screenshots/
└── meta.json
Why runs are long and linear
uatu does not restart the app every N steps. Each restart throws away two things.
Novelty and coverage signal. The exploration strategy weights actions by whether they reach previously unseen state. Restarting resets that history.
Deep app states. Many screens take many actions to reach: nested settings, a loaded cart, post-checkout flows. A 50-step prefix to reach "cart with 3 items" does not happen if every run starts cold.
Long-linear trajectories find bugs that restart-based testing structurally cannot.
Setup cost amortizes
Preconditions (login, onboarding, consent dialogs) are written as weighted action generators gated on extractors. See writing specs. They fire only when applicable, so login happens once per run, not per step.
| Run length | Login cost | % of run |
|---|---|---|
| 5 min | ~15s | 5% |
| 30 min | ~15s | 0.8% |
| 1 hour | ~15s | 0.4% |
| CI: 3 seeds × 10 min | ~45s total | 2.5% |
At any non-trivial run length, preconditions are a rounding error.
Session state
Session tokens, keychain, shared prefs, cookies, and other app-managed persistence survive the full run. If the app logs the user out mid-run, the doLogin generator re-fires automatically because its gating extractor (onLoginScreen) becomes true again. No retry logic. No special-casing.
Termination
A run ends when either of these happens.
--durationelapses.- The process is interrupted (SIGINT).
The trace is written incrementally, so an interrupted run is still fully inspectable.
Additional termination conditions (--max-steps, --exit-on-violation, hard crash handling) land in v0.1.0.