pj
88db0cbea8
* chore(docs): replace d2 diagram pipeline with mermaid Remove docs/_diagrams/ and d2 build step from Makefile. The HTML template already initialises mermaid.js; diagrams are now inline code fences rendered client-side. * docs(architecture): add mermaid diagram + web/CDP platform docs Replace SVG img tag with inline mermaid flowchart showing both native (Maestro sidecar + in-app SDK) and web (Chrome CDP) paths. Update Processes, Transports table, and per-step cycle sections. * docs(design-principles): update principles 1-4 for web platform Principles 1, 2, 3, and 4 referenced Maestro and native-only concepts. Add web/CDP context and update driver-is-an-interface to name both sidecar and chrome implementations. * docs(manual): add web prerequisites and folio-web example Update --platform flag to list android, ios, web. Add web prerequisites section (Chrome, no SDK needed) and folio-web quick-start to getting-started. * chore(gitignore): untrack inspect dist/index.html build artifact index.html is regenerated by vite on every build with a new content hash, making it permanently dirty. Only .gitkeep is needed for //go:embed to compile on a fresh checkout. Also remove duplicate dist/* line and stale d2 diagram ignore entries. * feat(docs): click-to-zoom for mermaid diagrams * docs(architecture): change diagram layout from LR to TB * docs(getting-started): link npm and Maven Central package headers * update docs root * docs(spec): rewrite npm package README Update usage example to current API, drop stale version-compatibility and license sections. * build(docs): output pages as pagename/index.html for clean URLs Split DOCS_OUT into INDEX_OUT (index.md files stay as index.html) and PAGE_OUT (all other pages become pagename/index.html). The __ROOT__ depth computation already handles the extra directory level correctly. * chore(docs): update sidebar links to directory-style URLs * docs: update cross-links from .html to directory-style paths * ci(docs): remove d2 install step * feat(docs): dark/light mode toggle Add theme toggle button (top-right, fixed). Persists preference in localStorage; falls back to prefers-color-scheme. Flash-free via inline script in <head> that sets data-theme before first paint. * fix(docs): fix inspect image path broken by directory URL restructure * feat(docs): click-to-fullscreen for all article images * fix(inspect): allow AssetsFS override in ServerOptions; drop unused request param from serveIndex * fix(inspect): use in-memory FS in tests so TestAssets_FallbackToIndexHTML passes without web build
2.7 KiB
title
| title |
|---|
| Runs |
What is a run?
One sanderling test invocation. Fresh install, spec-driven exploration, then the trace lands in runs/<timestamp>/. Typically minutes to hours, not seconds.
A run is not analogous to a unit test. A closer framing is: boot a fuzzer for an hour and see what breaks. Violations are recorded in the trace and exploration continues, so one run can surface many bugs.
Lifecycle
sanderling test --spec spec.ts --bundle-id com.example.app --duration 30m
│
├── uninstall and reinstall the app (clean slate, every run)
├── boot the sidecar, connect the agent socket
├── bundle the spec, load it into goja
│
├── step 0..N: pause, capture state, evaluate properties, pick action, resume, dispatch
│
└── terminate when --duration elapses (or SIGINT)
└── trace written to ./runs/<timestamp>/
├── trace.jsonl
├── screenshots/
└── meta.json
Why runs are long and linear
sanderling does not restart the app every N steps. Each restart throws away two things.
Novelty and coverage signal. The exploration strategy weights actions by whether they reach previously unseen state. Restarting resets that history.
Deep app states. Many screens take many actions to reach: nested settings, a loaded cart, post-checkout flows. A 50-step prefix to reach "cart with 3 items" does not happen if every run starts cold.
Long-linear trajectories find bugs that restart-based testing structurally cannot.
Setup cost amortizes
Preconditions (login, onboarding, consent dialogs) are written as weighted action generators gated on extractors. See writing specs. They fire only when applicable, so login happens once per run, not per step.
| Run length | Login cost | % of run |
|---|---|---|
| 5 min | ~15s | 5% |
| 30 min | ~15s | 0.8% |
| 1 hour | ~15s | 0.4% |
| CI: 3 seeds × 10 min | ~45s total | 2.5% |
At any non-trivial run length, preconditions are a rounding error.
Session state
Session tokens, keychain, shared prefs, cookies, and other app-managed persistence survive the full run. If the app logs the user out mid-run, the doLogin generator re-fires automatically because its gating extractor (onLoginScreen) becomes true again. No retry logic. No special-casing.
Termination
A run ends when either of these happens.
--durationelapses.- The process is interrupted (SIGINT).
The trace is written incrementally, so an interrupted run is still fully inspectable.
Additional termination conditions (--max-steps, --exit-on-violation, hard crash handling) land in v0.1.0.