mirror of
https://github.com/priyanshujain/sanderling.git
synced 2026-10-02 19:17:10 +00:00
pj
991c583eb9
* docs(manual): add introduction page * docs(manual): rewrite getting started as guided first run * docs(manual): rewrite writing specs as a folio tutorial * docs(manual): document missing spec API in reference * docs(manual): plain-language rewrite of runs page * docs: real introductions on index pages and README * fix(docs): sibling links from directory-style pages need ../ * fix(docs): correct sampling and restart-cost claims to match implementation * docs: nav lists Introduction and Case study; roadmap points to milestone * docs(manual): make getting started target the reader's own app, not Folio * docs(manual): add Folio case study page * docs: point manual navigation at the case study * docs(readme): lead with the case study, fix roadmap link * docs: roadmap links to milestone, sync clear-data default and cross-links
67 lines
2.8 KiB
Markdown
67 lines
2.8 KiB
Markdown
---
|
|
title: Runs
|
|
---
|
|
|
|
# Runs
|
|
|
|
A run is one `sanderling test` invocation: launch the app, explore it under the spec for a fixed duration, write a trace. Runs typically last minutes to hours, not seconds.
|
|
|
|
A run is not a unit test. The closer picture is: boot a fuzzer for an hour and see what breaks. A violated property is recorded in the trace and exploration continues, so one run can surface many bugs.
|
|
|
|
## Lifecycle
|
|
|
|
```
|
|
sanderling test --spec spec.ts --bundle-id com.example.app --duration 30m
|
|
│
|
|
├── launch the app under test (pass --clear-data to wipe app data first)
|
|
├── boot the sidecar (or connect to Chrome on web)
|
|
├── bundle the spec, load it into the JS runtime
|
|
│
|
|
├── step 0..N: read state, check properties, pick and perform an action
|
|
│
|
|
└── stop when --duration elapses (or on Ctrl+C)
|
|
└── trace written to ./runs/<timestamp>/
|
|
├── trace.jsonl
|
|
├── screenshots/
|
|
└── meta.json
|
|
```
|
|
|
|
The trace is written incrementally. An interrupted run is complete up to the step where it stopped.
|
|
|
|
## App state across runs
|
|
|
|
By default each run wipes app data before launch and starts cold. Pass `--clear-data=false` to resume whatever the previous run left behind (an account, cached responses, completed onboarding). See the [CLI reference](../cli/#sanderling-test).
|
|
|
|
## Why runs are long
|
|
|
|
sanderling does not restart the app every few steps. Restarting throws away two things.
|
|
|
|
**Accumulated data.** Accounts created, items added, caches warmed, settings changed. Interesting bugs live in apps with history, and a restart wipes it.
|
|
|
|
**Deep app states.** Many bugs live in states that take many actions to reach: nested settings, a loaded cart, the screen after the third transaction. A 50-step path to "cart with 3 items" never happens if every run starts cold.
|
|
|
|
Long runs reach states that restart-per-test approaches structurally cannot.
|
|
|
|
## Setup cost is paid once
|
|
|
|
Preconditions like login run through the spec's `setup` export (see the [case study](../case-study/#reaching-the-screens-that-matter)). They fire when their condition is unmet and go quiet after, so login costs a few seconds once per run, not once per test case.
|
|
|
|
| Run length | Login cost | Share of run |
|
|
|---|---|---|
|
|
| 5 min | ~15s | 5% |
|
|
| 30 min | ~15s | 0.8% |
|
|
| 1 hour | ~15s | 0.4% |
|
|
|
|
## Session state
|
|
|
|
Session tokens, keychain entries, shared preferences, and cookies survive the whole run. If the app logs the user out mid-run, the gating extractor flips, `setup` re-engages, and the run logs back in. No retry logic needed in the spec.
|
|
|
|
## Termination
|
|
|
|
A run ends when:
|
|
|
|
- `--duration` elapses, or
|
|
- the process is interrupted (Ctrl+C).
|
|
|
|
Additional conditions (`--max-steps`, `--exit-on-violation`, hard crash handling) land in the [v0.1.0 milestone](https://github.com/priyanshujain/sanderling/milestone/1).
|