mirror of
https://github.com/priyanshujain/sanderling.git
synced 2026-10-02 11:07:10 +00:00
docs: pandoc-based site and v0.1.0 groundwork (#5)
* chore(prose): remove em-dashes from config files * chore(prose): remove em-dashes from android sdk config * docs(spec-api): remove em-dash from README * fix(doctor): reword sidecar-jar error without em-dash * test(sidecar): reword assertion message without em-dash * docs: add CLAUDE.md with project conventions * build: add docs target for pandoc site * docs(site): add pandoc template and stylesheet * docs(site): add pandoc build script * docs(site): add landing pages * docs(manual): add getting-started * docs(manual): add writing-specs * docs(manual): add runs * docs(manual): add cli reference * docs(dev): add design principles * docs(dev): add architecture * ci: deploy docs site to github pages * docs: rewrite README as entry point to docs site
This commit is contained in:
25 files changed
+920
-56
No files matched your search
@@ -0,0 +1,174 @@
|
||||
:root {
|
||||
--bg: #ffffff;
|
||||
--fg: #2a2a2a;
|
||||
--muted: #6a6a6a;
|
||||
--border: #e5e5e5;
|
||||
--accent: #0a66c2;
|
||||
--code-bg: #f6f6f6;
|
||||
--sidebar-width: 240px;
|
||||
}
|
||||
|
||||
@media (prefers-color-scheme: dark) {
|
||||
:root {
|
||||
--bg: #0f0f10;
|
||||
--fg: #e5e5e5;
|
||||
--muted: #a0a0a0;
|
||||
--border: #2a2a2a;
|
||||
--accent: #4aa3ff;
|
||||
--code-bg: #1a1a1b;
|
||||
}
|
||||
}
|
||||
|
||||
* { box-sizing: border-box; }
|
||||
|
||||
html { background: var(--bg); color: var(--fg); }
|
||||
|
||||
body {
|
||||
margin: 0;
|
||||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", system-ui, sans-serif;
|
||||
font-size: 16px;
|
||||
line-height: 1.55;
|
||||
}
|
||||
|
||||
.layout {
|
||||
display: grid;
|
||||
grid-template-columns: var(--sidebar-width) 1fr;
|
||||
min-height: 100vh;
|
||||
max-width: 1280px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
|
||||
.sidebar {
|
||||
border-right: 1px solid var(--border);
|
||||
padding: 1.5rem 1rem;
|
||||
position: sticky;
|
||||
top: 0;
|
||||
height: 100vh;
|
||||
overflow-y: auto;
|
||||
}
|
||||
|
||||
.sidebar .brand {
|
||||
display: block;
|
||||
font-weight: 700;
|
||||
font-size: 1.25rem;
|
||||
text-decoration: none;
|
||||
color: var(--fg);
|
||||
margin-bottom: 1.5rem;
|
||||
}
|
||||
|
||||
.sidebar h3 {
|
||||
font-size: 0.75rem;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 0.05em;
|
||||
color: var(--muted);
|
||||
margin: 1.25rem 0 0.25rem;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
.sidebar nav ul {
|
||||
list-style: none;
|
||||
padding: 0;
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.sidebar nav li a {
|
||||
display: block;
|
||||
padding: 0.2rem 0;
|
||||
color: var(--fg);
|
||||
text-decoration: none;
|
||||
font-size: 0.95rem;
|
||||
}
|
||||
|
||||
.sidebar nav li a:hover {
|
||||
color: var(--accent);
|
||||
}
|
||||
|
||||
main {
|
||||
padding: 2rem 2.5rem 4rem;
|
||||
max-width: 860px;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
main article h1 { font-size: 2rem; margin: 0 0 1.5rem; line-height: 1.2; }
|
||||
main article h2 { margin-top: 2.5rem; font-size: 1.4rem; }
|
||||
main article h3 { margin-top: 2rem; font-size: 1.1rem; }
|
||||
|
||||
a { color: var(--accent); }
|
||||
|
||||
code {
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
|
||||
font-size: 0.9em;
|
||||
background: var(--code-bg);
|
||||
padding: 0.1rem 0.3rem;
|
||||
border-radius: 3px;
|
||||
}
|
||||
|
||||
pre {
|
||||
background: var(--code-bg);
|
||||
padding: 1rem;
|
||||
border-radius: 4px;
|
||||
overflow-x: auto;
|
||||
font-size: 0.875rem;
|
||||
line-height: 1.5;
|
||||
}
|
||||
|
||||
pre code {
|
||||
background: transparent;
|
||||
padding: 0;
|
||||
font-size: inherit;
|
||||
}
|
||||
|
||||
table {
|
||||
border-collapse: collapse;
|
||||
margin: 1rem 0;
|
||||
width: 100%;
|
||||
font-size: 0.95rem;
|
||||
}
|
||||
|
||||
th, td {
|
||||
border: 1px solid var(--border);
|
||||
padding: 0.5rem 0.75rem;
|
||||
text-align: left;
|
||||
vertical-align: top;
|
||||
}
|
||||
|
||||
th { background: var(--code-bg); font-weight: 600; }
|
||||
|
||||
blockquote {
|
||||
border-left: 3px solid var(--border);
|
||||
padding-left: 1rem;
|
||||
margin: 1rem 0;
|
||||
color: var(--muted);
|
||||
}
|
||||
|
||||
hr {
|
||||
border: none;
|
||||
border-top: 1px solid var(--border);
|
||||
margin: 2rem 0;
|
||||
}
|
||||
|
||||
footer {
|
||||
margin-top: 4rem;
|
||||
padding-top: 1rem;
|
||||
border-top: 1px solid var(--border);
|
||||
font-size: 0.85rem;
|
||||
color: var(--muted);
|
||||
}
|
||||
|
||||
footer a { color: var(--muted); }
|
||||
|
||||
.mermaid {
|
||||
text-align: center;
|
||||
margin: 1.5rem 0;
|
||||
}
|
||||
|
||||
@media (max-width: 768px) {
|
||||
.layout { grid-template-columns: 1fr; }
|
||||
.sidebar {
|
||||
position: static;
|
||||
height: auto;
|
||||
border-right: none;
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
main { padding: 1.5rem; }
|
||||
}
|
||||
Vendored
+52
@@ -0,0 +1,52 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||||
<title>$if(title)$$title$ · $endif$uatu</title>
|
||||
<link rel="stylesheet" href="__ROOT___assets/style.css">
|
||||
<style>$highlighting-css$</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="layout">
|
||||
<aside class="sidebar">
|
||||
<a class="brand" href="__ROOT__index.html">uatu</a>
|
||||
<nav>
|
||||
<h3>Manual</h3>
|
||||
<ul>
|
||||
<li><a href="__ROOT__manual/getting-started.html">Getting started</a></li>
|
||||
<li><a href="__ROOT__manual/writing-specs.html">Writing specs</a></li>
|
||||
<li><a href="__ROOT__manual/runs.html">Runs</a></li>
|
||||
<li><a href="__ROOT__manual/cli.html">CLI reference</a></li>
|
||||
</ul>
|
||||
<h3>Development</h3>
|
||||
<ul>
|
||||
<li><a href="__ROOT__development/design-principles.html">Design principles</a></li>
|
||||
<li><a href="__ROOT__development/architecture.html">Architecture</a></li>
|
||||
</ul>
|
||||
</nav>
|
||||
</aside>
|
||||
<main>
|
||||
<article>
|
||||
$body$
|
||||
</article>
|
||||
<footer>
|
||||
<a href="https://github.com/priyanshujain/uatu">github.com/priyanshujain/uatu</a>
|
||||
· <a href="https://github.com/priyanshujain/uatu/issues/4">v0.1.0 roadmap</a>
|
||||
</footer>
|
||||
</main>
|
||||
</div>
|
||||
<script type="module">
|
||||
import mermaid from 'https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs';
|
||||
mermaid.initialize({ startOnLoad: false, theme: 'neutral', fontFamily: 'inherit' });
|
||||
for (const pre of document.querySelectorAll('pre.mermaid, pre > code.language-mermaid')) {
|
||||
const node = pre.tagName === 'PRE' ? pre : pre.parentElement;
|
||||
const div = document.createElement('div');
|
||||
div.className = 'mermaid';
|
||||
div.textContent = node.textContent;
|
||||
node.replaceWith(div);
|
||||
}
|
||||
await mermaid.run();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,69 @@
|
||||
---
|
||||
title: Architecture
|
||||
---
|
||||
|
||||
# Architecture
|
||||
|
||||
Three processes, two transports.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Go["uatu (Go)"]
|
||||
Bundler[Bundler<br/>esbuild] --> Verifier[Verifier<br/>goja + LTL]
|
||||
Verifier <--> Runner[Runner]
|
||||
Runner --> Trace[Trace writer<br/>JSONL + PNG]
|
||||
Runner <--> Driver[Driver iface]
|
||||
end
|
||||
|
||||
subgraph Sidecar["Maestro Sidecar (JVM)"]
|
||||
Maestro[maestro-client]
|
||||
end
|
||||
|
||||
subgraph Device["Emulator"]
|
||||
subgraph App["Android app (debug)"]
|
||||
SDK[uatu-sdk<br/>pause / hierarchy<br/>logs / coverage]
|
||||
end
|
||||
end
|
||||
|
||||
Driver -- gRPC --> Maestro
|
||||
Maestro -- UIAutomator --> App
|
||||
Runner -- Unix socket --> SDK
|
||||
Trace --> Runs[(runs/)]
|
||||
```
|
||||
|
||||
## Processes
|
||||
|
||||
**uatu (Go).** The top-level binary. Bundles the spec with esbuild, evaluates it in goja, runs the main loop, dispatches actions through the driver, writes the trace.
|
||||
|
||||
**Maestro sidecar (JVM).** A Kotlin process that wraps `maestro-client` and exposes a gRPC surface matching the `driver.Driver` interface. Handles UI input, screenshots, the system accessibility tree, and OS-level alerts.
|
||||
|
||||
**In-app SDK.** A Kotlin (or Swift for iOS) library linked into the app under test. Exposes a Unix socket to the runner. Provides pause and resume, view-hierarchy dumps, coverage reads, log capture, and user-registered state extractors.
|
||||
|
||||
## Transports
|
||||
|
||||
| Channel | Transport | Purpose |
|
||||
|---|---|---|
|
||||
| Go to Maestro sidecar | gRPC (localhost TCP) | UI input, screenshots, system alerts |
|
||||
| Go to in-app SDK | Unix domain socket | Pause / resume, hierarchy, coverage, logs, extractors |
|
||||
|
||||
The split exists for one reason: only real UI events need the cost of crossing process and OS-API boundaries. Introspection is cheap, frequent, and lives on a fast local socket directly to the app.
|
||||
|
||||
## Per-step cycle
|
||||
|
||||
The heart of the system is:
|
||||
|
||||
```
|
||||
pause ─► capture state ─► evaluate properties ─► pick action ─► resume ─► dispatch
|
||||
```
|
||||
|
||||
1. The runner asks the driver to wait until the UI is idle.
|
||||
2. The runner sends `PAUSE` to the SDK over the agent socket. The SDK freezes the main runloop at a safe point.
|
||||
3. The SDK sends back a `STATE` message: view hierarchy, coverage delta, logs since last step, exception list, snapshot values.
|
||||
4. The runner feeds state into goja. Extractors re-read; properties re-evaluate; the action generator returns a weighted tree.
|
||||
5. The runner writes the trace entry for this step.
|
||||
6. The runner picks an action by weight.
|
||||
7. The runner sends `RESUME` to the SDK, then dispatches the action through the driver (gRPC to sidecar, which talks to Maestro, which talks to UIAutomator or XCTest).
|
||||
8. Loop.
|
||||
|
||||
The cycle runs hundreds of times per minute. Every step produces one row in `trace.jsonl` and one screenshot.
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
---
|
||||
title: Design principles
|
||||
---
|
||||
|
||||
# Design principles
|
||||
|
||||
## 1. The app owns introspection; the driver owns input
|
||||
|
||||
The in-app SDK knows the state: view hierarchy, coverage, logs, exceptions, custom extractors. Maestro causes the state to change through taps, swipes, typed text, and deep links. The Go runner decides what to do.
|
||||
|
||||
Splitting these responsibilities is what makes the system work across iOS and Android with one spec surface. Neither Maestro nor the SDK alone is sufficient.
|
||||
|
||||
- Maestro can read a coarse accessibility tree, but not the real `UIView` or `View` hierarchy, not coverage, not in-process logs.
|
||||
- The SDK can see everything inside the app, but cannot dispatch UI events the way the OS would. Touch injection through Maestro goes through XCTest or UIAutomator, which the OS treats as real input.
|
||||
|
||||
## 2. One TypeScript surface across platforms
|
||||
|
||||
Spec authors write against `state.ax`, `state.logs`, `state.snapshots`, and so on, regardless of iOS or Android. Platform differences (back button semantics, system alerts, coverage format) are absorbed in the Go runner and the SDKs.
|
||||
|
||||
Corollary: if a concept only exists on one platform, it does not belong in the spec API. It belongs behind a feature flag or an extractor.
|
||||
|
||||
## 3. The driver is an interface
|
||||
|
||||
Today `driver.Driver` has one production implementation (`maestro`) and a `mock` for tests. Tomorrow it might be Appium, direct XCTest, or UIAutomator. The runner never knows. This keeps the Maestro dependency contained. If we ever outgrow it, the blast radius is one package.
|
||||
|
||||
## 4. Hot loops bypass Maestro
|
||||
|
||||
Per-step introspection (hierarchy dump, coverage read, pause and resume) goes over a local Unix socket directly to the SDK. Only physical UI events go through Maestro's gRPC.
|
||||
|
||||
A 30-minute run is about 10,000 steps. Every step has at least one hierarchy dump and one coverage read. If those went through Maestro, the JVM sidecar would be the bottleneck. Instead the hot path is a 2 ms round-trip to an in-process Swift or Kotlin SDK.
|
||||
|
||||
## 5. Deterministic where it can be
|
||||
|
||||
A seeded PRNG drives action selection. Spec evaluation is pure given state and snapshots. The bundle hash and seed are recorded in `meta.json`.
|
||||
|
||||
uatu does not attempt byte-exact replay. Animation timing, keyboard popup timing, and system daemons are non-deterministic on mobile, and the cost of trying to suppress that is not worth the payoff. Same seed produces a similar trajectory, which is usually enough to reproduce the bug.
|
||||
|
||||
## 6. Fail honest
|
||||
|
||||
If coverage is not available (release build, instrumentation off), tell the user. Do not pretend exploration is guided when it is random. If the SDK is not linked, say so. If a property is unparseable, fail the run at startup, not step 1000.
|
||||
|
||||
The alternative, graceful degradation that silently weakens guarantees, is how testing tools lose trust.
|
||||
|
||||
## 7. Specs are authoritative; no hidden setup
|
||||
|
||||
There is exactly one authoring surface: the TypeScript spec. There is no separate YAML for login, no fixtures directory, no `setup.sh`. Login, onboarding, permission prompts, and teardown are all expressed as action generators or extractors, evaluated in the same loop as the rest of the spec.
|
||||
|
||||
This is intentional. A test harness with two authoring languages (YAML plus code, JSON plus code) splits concerns in a way that always drifts. Something works in one surface and not the other, and debugging requires holding both in your head.
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
---
|
||||
title: Development
|
||||
---
|
||||
|
||||
# Development
|
||||
|
||||
- [Design principles](./design-principles.html)
|
||||
- [Architecture](./architecture.html)
|
||||
- v0.1.0 scope: [issue #4](https://github.com/priyanshujain/uatu/issues/4)
|
||||
|
||||
## Building the docs site locally
|
||||
|
||||
```
|
||||
make docs
|
||||
```
|
||||
|
||||
Outputs to `build/site/`. Requires [pandoc](https://pandoc.org/) on your PATH. Preview with:
|
||||
|
||||
```
|
||||
cd build/site && python3 -m http.server 8000
|
||||
```
|
||||
@@ -0,0 +1,21 @@
|
||||
---
|
||||
title: uatu
|
||||
---
|
||||
|
||||
# uatu
|
||||
|
||||
Autonomous property-based testing for mobile apps. Specs in TypeScript. Core in Go. Drives the app under test through Maestro and an in-app SDK.
|
||||
|
||||
Alpha: Android emulator only. Scope of v0.1.0 is tracked in [issue #4](https://github.com/priyanshujain/uatu/issues/4).
|
||||
|
||||
## Manual
|
||||
|
||||
- [Getting started](./manual/getting-started.html)
|
||||
- [Writing specs](./manual/writing-specs.html)
|
||||
- [Runs](./manual/runs.html)
|
||||
- [CLI reference](./manual/cli.html)
|
||||
|
||||
## Development
|
||||
|
||||
- [Design principles](./development/design-principles.html)
|
||||
- [Architecture](./development/architecture.html)
|
||||
@@ -0,0 +1,41 @@
|
||||
---
|
||||
title: CLI reference
|
||||
---
|
||||
|
||||
# CLI reference
|
||||
|
||||
```
|
||||
uatu <command> [flags]
|
||||
```
|
||||
|
||||
## `uatu test`
|
||||
|
||||
Run a spec against an app for a fixed duration.
|
||||
|
||||
| Flag | Default | Description |
|
||||
|---|---|---|
|
||||
| `--spec` | required | Path to the TypeScript spec. |
|
||||
| `--bundle-id` | required | Target app bundle ID (Android: applicationId). |
|
||||
| `--launcher-activity` | resolved | Optional `<pkg>/<activity>` to launch. Overrides default resolution. |
|
||||
| `--platform` | `android` | Target platform. Only `android` in the current alpha. |
|
||||
| `--avd` | required (android) | Android AVD name. |
|
||||
| `--duration` | `5m` | Total test duration (`30s`, `5m`, `2h`, `1d`). |
|
||||
| `--seed` | `0` | PRNG seed. `0` uses a random seed and records it in `meta.json`. |
|
||||
| `--output` | `./runs` | Output directory for traces. |
|
||||
|
||||
## `uatu doctor`
|
||||
|
||||
Check the host environment for a working uatu setup: Go toolchain, JDK, Maestro availability, emulator reachability, SDK linkage hints.
|
||||
|
||||
## `uatu version`
|
||||
|
||||
Print the CLI version.
|
||||
|
||||
## Flags coming in v0.1.0
|
||||
|
||||
- `--permissions` to pre-set OS-level permissions (for example `--permissions location=allow,notifications=deny`).
|
||||
- `--max-steps` hard cap on step count.
|
||||
- `--exit-on-violation` stop the run on the first property violation.
|
||||
- `uatu inspect` command for browsing traces in the built-in UI.
|
||||
|
||||
Tracked in [issue #4](https://github.com/priyanshujain/uatu/issues/4).
|
||||
@@ -0,0 +1,76 @@
|
||||
---
|
||||
title: Getting started
|
||||
---
|
||||
|
||||
# Getting started
|
||||
|
||||
Install the CLI, link the SDK into your debug build, run a spec.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- An Android emulator with API level 30 or newer.
|
||||
- The app under test built as a debug variant with the uatu Android SDK linked in.
|
||||
- `adb` on your PATH.
|
||||
|
||||
Run `uatu doctor` to check the host environment.
|
||||
|
||||
## Install
|
||||
|
||||
### CLI
|
||||
|
||||
macOS arm64:
|
||||
|
||||
```sh
|
||||
curl -L https://github.com/priyanshujain/uatu/releases/latest/download/uatu_<version>_darwin_arm64.tar.gz | tar xz
|
||||
./uatu version
|
||||
```
|
||||
|
||||
Linux amd64:
|
||||
|
||||
```sh
|
||||
curl -L https://github.com/priyanshujain/uatu/releases/latest/download/uatu_<version>_linux_amd64.tar.gz | tar xz
|
||||
./uatu version
|
||||
```
|
||||
|
||||
Pre-built for `darwin/arm64`, `darwin/amd64`, `linux/amd64`, `linux/arm64`.
|
||||
|
||||
### Spec package (npm)
|
||||
|
||||
```sh
|
||||
npm install --save-dev @uatu/spec
|
||||
```
|
||||
|
||||
### Android SDK (Maven Central)
|
||||
|
||||
```kotlin
|
||||
dependencies {
|
||||
implementation("io.github.priyanshujain:sdk-android:<version>")
|
||||
}
|
||||
```
|
||||
|
||||
## Your first run
|
||||
|
||||
The repo ships a working sample at `examples/sample-app`. From that directory:
|
||||
|
||||
```sh
|
||||
./gradlew :sample-app:installDebug
|
||||
uatu test \
|
||||
--spec spec.ts \
|
||||
--bundle-id dev.uatu.sample \
|
||||
--platform android \
|
||||
--avd Pixel_7 \
|
||||
--duration 2m
|
||||
```
|
||||
|
||||
When the run ends, the trace lands in `runs/<timestamp>/`:
|
||||
|
||||
```
|
||||
runs/2026-04-18T12-34-56/
|
||||
├── trace.jsonl
|
||||
├── screenshots/
|
||||
└── meta.json
|
||||
```
|
||||
|
||||
Open the screenshots directory to scrub visually, or read `trace.jsonl` step by step.
|
||||
|
||||
Next: [writing specs](./writing-specs.html).
|
||||
@@ -0,0 +1,10 @@
|
||||
---
|
||||
title: Manual
|
||||
---
|
||||
|
||||
# Manual
|
||||
|
||||
- [Getting started](./getting-started.html)
|
||||
- [Writing specs](./writing-specs.html)
|
||||
- [Runs](./runs.html)
|
||||
- [CLI reference](./cli.html)
|
||||
@@ -0,0 +1,65 @@
|
||||
---
|
||||
title: Runs
|
||||
---
|
||||
|
||||
# What is a run?
|
||||
|
||||
One `uatu test` invocation. Fresh install, spec-driven exploration, then the trace lands in `runs/<timestamp>/`. Typically minutes to hours, not seconds.
|
||||
|
||||
A run is not analogous to a unit test. A closer framing is: boot a fuzzer for an hour and see what breaks. Violations are recorded in the trace and exploration continues, so one run can surface many bugs.
|
||||
|
||||
## Lifecycle
|
||||
|
||||
```
|
||||
uatu test --spec spec.ts --bundle-id com.example.app --avd Pixel_7 --duration 30m
|
||||
│
|
||||
├── uninstall and reinstall the app (clean slate, every run)
|
||||
├── boot the sidecar, connect the agent socket
|
||||
├── bundle the spec, load it into goja
|
||||
│
|
||||
├── step 0..N: pause, capture state, evaluate properties, pick action, resume, dispatch
|
||||
│
|
||||
└── terminate when --duration elapses (or SIGINT)
|
||||
└── trace written to ./runs/<timestamp>/
|
||||
├── trace.jsonl
|
||||
├── screenshots/
|
||||
└── meta.json
|
||||
```
|
||||
|
||||
## Why runs are long and linear
|
||||
|
||||
uatu does not restart the app every N steps. Each restart throws away two things.
|
||||
|
||||
**Novelty and coverage signal.** The exploration strategy weights actions by whether they reach previously unseen state. Restarting resets that history.
|
||||
|
||||
**Deep app states.** Many screens take many actions to reach: nested settings, a loaded cart, post-checkout flows. A 50-step prefix to reach "cart with 3 items" does not happen if every run starts cold.
|
||||
|
||||
Long-linear trajectories find bugs that restart-based testing structurally cannot.
|
||||
|
||||
## Setup cost amortizes
|
||||
|
||||
Preconditions (login, onboarding, consent dialogs) are written as weighted action generators gated on extractors. See [writing specs](./writing-specs.html#pattern-preconditions-login-onboarding). They fire only when applicable, so login happens once per run, not per step.
|
||||
|
||||
| Run length | Login cost | % of run |
|
||||
|---|---|---|
|
||||
| 5 min | ~15s | 5% |
|
||||
| 30 min | ~15s | 0.8% |
|
||||
| 1 hour | ~15s | 0.4% |
|
||||
| CI: 3 seeds × 10 min | ~45s total | 2.5% |
|
||||
|
||||
At any non-trivial run length, preconditions are a rounding error.
|
||||
|
||||
## Session state
|
||||
|
||||
Session tokens, keychain, shared prefs, cookies, and other app-managed persistence survive the full run. If the app logs the user out mid-run, the `doLogin` generator re-fires automatically because its gating extractor (`onLoginScreen`) becomes true again. No retry logic. No special-casing.
|
||||
|
||||
## Termination
|
||||
|
||||
A run ends when either of these happens.
|
||||
|
||||
- `--duration` elapses.
|
||||
- The process is interrupted (SIGINT).
|
||||
|
||||
The trace is written incrementally, so an interrupted run is still fully inspectable.
|
||||
|
||||
Additional termination conditions (`--max-steps`, `--exit-on-violation`, hard crash handling) land in [v0.1.0](https://github.com/priyanshujain/uatu/issues/4).
|
||||
@@ -0,0 +1,193 @@
|
||||
---
|
||||
title: Writing specs
|
||||
---
|
||||
|
||||
# Writing specs
|
||||
|
||||
A spec has three parts: extractors, properties, and actions.
|
||||
|
||||
```ts
|
||||
import { extract, always, actions, weighted, Tap, taps, swipes } from "@uatu/spec";
|
||||
|
||||
// 1. Extractors pull values from each observed state.
|
||||
const loggedIn = extract((s) => !!s.ax.find("id:home-tab-bar"));
|
||||
const cartCount = extract<number>((s) => (s.snapshots.cart_count as number) ?? 0);
|
||||
|
||||
// 2. Properties are LTL formulas evaluated every step.
|
||||
export const properties = {
|
||||
cartNeverNegative: always(() => cartCount.current >= 0),
|
||||
};
|
||||
|
||||
// 3. Actions are a weighted tree of what uatu is allowed to do.
|
||||
export const actions = weighted(
|
||||
[10, taps],
|
||||
[2, swipes],
|
||||
);
|
||||
```
|
||||
|
||||
The Go runner calls into the JS runtime each step. Extractors re-read the current state. Properties re-evaluate with their residual formulas. The action generator returns a tree, and one leaf is sampled by weight and dispatched.
|
||||
|
||||
## The `State` object
|
||||
|
||||
What extractors see:
|
||||
|
||||
```ts
|
||||
interface State {
|
||||
ax: AccessibilityTree; // view hierarchy
|
||||
snapshots: Record<string, unknown>; // values registered by the in-app SDK
|
||||
screen: { id: string; hash: string };
|
||||
lastAction: Action | null;
|
||||
logs: LogEntry[]; // since previous state
|
||||
exceptions: Exception[];
|
||||
time: number; // ms since run start
|
||||
}
|
||||
```
|
||||
|
||||
`ax.find("text:Click me")`, `ax.find("id:login-form")`, `ax.findAll("role:todo-row")` are the common accessors. Prefer stable testID-style identifiers over positional selectors, for the same reason you would in Espresso or XCUITest.
|
||||
|
||||
`snapshots` is populated by the in-app SDK via `Uatu.extract("name") { value }`. Use it when the UI does not expose a value you need, such as business-logic state or hidden fields.
|
||||
|
||||
## Pattern: preconditions (login, onboarding)
|
||||
|
||||
uatu has no setup phase and no fixtures. Preconditions are action generators with two properties:
|
||||
|
||||
1. High weight, so they fire whenever applicable.
|
||||
2. Gated on a state extractor, so they return an empty tree when not applicable and self-disable once the precondition is met.
|
||||
|
||||
```ts
|
||||
const onLoginScreen = extract((s) => !!s.ax.find("id:login-form"));
|
||||
|
||||
const doLogin = actions(() => {
|
||||
if (!onLoginScreen.current) return [];
|
||||
const emailField = state.ax.find("id:email-field");
|
||||
const signInButton = state.ax.find("id:sign-in-button");
|
||||
if (!emailField || !signInButton) return [];
|
||||
return [
|
||||
InputText({ into: emailField, text: "[email protected]" }),
|
||||
Tap({ on: signInButton }),
|
||||
];
|
||||
});
|
||||
```
|
||||
|
||||
Stack these for onboarding, consent dialogs, cold-start flows:
|
||||
|
||||
```ts
|
||||
const dismissOnboarding = actions(() => {
|
||||
const skip = state.ax.find("text:Skip");
|
||||
return skip ? [Tap({ on: skip })] : [];
|
||||
});
|
||||
|
||||
export const actions = weighted(
|
||||
[100, dismissOnboarding], // clear the path first
|
||||
[50, doLogin], // log in when the login screen appears
|
||||
[10, taps], // exploration
|
||||
[2, swipes],
|
||||
);
|
||||
```
|
||||
|
||||
Lifecycle of a run:
|
||||
|
||||
```
|
||||
Step 1: fresh install, onboarding visible
|
||||
eligible: dismissOnboarding (weight 100)
|
||||
picks: Tap "Skip"
|
||||
|
||||
Step 2-3: login screen visible
|
||||
eligible: doLogin (weight 50)
|
||||
picks: InputText / Tap to sign in
|
||||
|
||||
Step 4+: home screen, onboarding and login generators return []
|
||||
eligible: taps, swipes
|
||||
picks: autonomous exploration
|
||||
```
|
||||
|
||||
Session state (tokens, keychain, prefs) persists through the rest of the run. If the app logs the user out mid-run, `doLogin` re-fires automatically. No retry logic, no special-casing.
|
||||
|
||||
## Pattern: conditional properties
|
||||
|
||||
Use gating extractors the same way inside properties. Express "only check X when Y holds":
|
||||
|
||||
```ts
|
||||
const loggedIn = extract((s) => !!s.ax.find("id:home-tab-bar"));
|
||||
|
||||
export const properties = {
|
||||
cartPersistsWhenLoggedIn: always(() => {
|
||||
if (!loggedIn.current) return true;
|
||||
return state.snapshots.cart_count !== undefined;
|
||||
}),
|
||||
};
|
||||
```
|
||||
|
||||
When `implies` ships in v0.1.0, this becomes:
|
||||
|
||||
```ts
|
||||
cartPersistsWhenLoggedIn: always(() =>
|
||||
implies(loggedIn.current, () => cartCount.current !== undefined)
|
||||
),
|
||||
```
|
||||
|
||||
## Pattern: eventually (once the operator lands)
|
||||
|
||||
`always` asserts something holds at every step. `eventually` asserts it holds at some step, usually with a time bound:
|
||||
|
||||
```ts
|
||||
// v0.1.0+
|
||||
loginSucceedsWithin30s: eventually(
|
||||
() => loggedIn.current
|
||||
).within(30, "seconds"),
|
||||
```
|
||||
|
||||
Useful for liveness checks: the loading spinner eventually goes away, the deep link eventually lands on `/home`.
|
||||
|
||||
## Pattern: snapshot-backed properties
|
||||
|
||||
When the UI does not expose a value but the app knows it, use the SDK's extractor registry:
|
||||
|
||||
```kotlin
|
||||
// in the app (Android)
|
||||
Uatu.extract("cart_count") { store.cart.size }
|
||||
```
|
||||
|
||||
```ts
|
||||
// in the spec
|
||||
const cartCount = extract<number>((s) => (s.snapshots.cart_count as number) ?? 0);
|
||||
|
||||
export const properties = {
|
||||
cartMonotonicAfterAdd: always(() => {
|
||||
const previous = cartCount.previous;
|
||||
return previous === undefined || cartCount.current >= previous;
|
||||
}),
|
||||
};
|
||||
```
|
||||
|
||||
This pattern lets you write properties against business logic that no UI element exposes.
|
||||
|
||||
## Pattern: weighted exploration sub-trees
|
||||
|
||||
Nest `weighted` to group related actions and tune their collective rate:
|
||||
|
||||
```ts
|
||||
export const actions = weighted(
|
||||
[100, dismissOnboarding],
|
||||
[50, doLogin],
|
||||
[10, taps],
|
||||
[2, swipes],
|
||||
[1, weighted(
|
||||
[3, openLink("todos://home")],
|
||||
[1, openLink("todos://settings")],
|
||||
[1, openLink("todos://item/42/edit")],
|
||||
)],
|
||||
);
|
||||
```
|
||||
|
||||
Weights are relative within a tree, so nested trees get their own local budget. This is how you keep low-frequency but high-value actions (deep links, background/foreground, rotate) from drowning out normal tapping.
|
||||
|
||||
## Anti-patterns
|
||||
|
||||
**Positional taps.** `Tap({ on: { x: 100, y: 200 } })` works for a demo but breaks on any layout change. Always prefer an `ax.find("id:...")` reference.
|
||||
|
||||
**Sleep or wait-for-time.** `Wait(3000)` inside an action generator is a smell. If you need to wait for a condition, use an extractor and gate the next action on it.
|
||||
|
||||
**Retry logic inside generators.** Generators should be pure: given the same state they produce the same actions. Retry is the runner's responsibility.
|
||||
|
||||
**Unbounded `eventually`.** Without a `.within(...)`, `eventually` never fails within a finite run. It just stays residual. Almost always you want a bound.
|
||||
Reference in new issue
Block a user