mirror of
https://github.com/priyanshujain/sanderling.git
synced 2026-10-02 19:17:10 +00:00
WIP: docs: refresh after iOS + web support (#50)
* docs: README covers iOS + web, surface both example apps * docs(cli): document --ios-device and per-platform doctor * docs: tighten README, fold examples into Docs list * docs(runs): correct --clear-data lifecycle wording Default behavior no longer wipes app data between runs; --clear-data is now opt-in. * docs(getting-started): add iOS path, separate folio and folio-web Document just test-ios under examples/folio, and distinguish the KMP sample from the React + Vite folio-web sample. * docs(inspect): document the eight panels Lists Screenshot, ActionList, Timeline, ViolationsPanel, HierarchyPanel, SnapshotTable, MetricsChart, ExceptionsPanel. Cross-links HierarchyPanel to the spec language reference. * docs(writing-specs): document setup export, flag noLogcatErrors as android-only Mirrors pkg/spec/README.md so the manual covers the runner's setup-first fall-through. Marks noLogcatErrors as Android-only so iOS/web spec authors know it silently no-ops. * docs(folio): document web target and iOS sanderling test recipe After the KMP refactor folio also runs on wasmJs and the justfile exposes just web, just web-build, and just test-ios. Surface all three. * docs(folio-web): add README Covers prerequisites, demo credentials, just test recipe, and how the React + Vite host exposes state to the sanderling spec via stable ids and data-* attributes. * docs: scrub driver-implementation name from user docs Drop the implementation tool name from README, cli.md doctor table, and spec-language.md. These docs should describe behaviour, not the specific underlying tool the native sidecar wraps. * docs(development): scrub driver-implementation name from dev docs architecture, design-principles, decisions now describe the native sidecar by role (gRPC surface over OS UI-test pipeline) rather than by the specific tool it wraps.
This commit is contained in:
12 files changed
+178
-28
No files matched your search
+14
-1
@@ -19,6 +19,7 @@ Run a spec against an app for a fixed duration.
|
||||
| `--launcher-activity` | resolved | Optional `<pkg>/<activity>` to launch. Overrides default resolution. |
|
||||
| `--platform` | `android` | Target platform: `android`, `ios`, or `web`. |
|
||||
| `--avd` | optional (android) | Android AVD name to boot if no device is connected. Required only when no device is connected and multiple AVDs exist. |
|
||||
| `--ios-device` | optional (ios) | iOS simulator name or UDID to boot if none is running. |
|
||||
| `--duration` | `5m` | Total test duration (`30s`, `5m`, `2h`, `1d`). |
|
||||
| `--seed` | `0` | PRNG seed. `0` uses a random seed and records it in `meta.json`. |
|
||||
| `--output` | `./runs` | Output directory for traces. |
|
||||
@@ -38,7 +39,19 @@ See [the inspect UI page](./inspect/) for the panel reference and keyboard short
|
||||
|
||||
## `sanderling doctor`
|
||||
|
||||
Check the host environment for a working sanderling setup: Go toolchain, JDK, Maestro availability, emulator reachability, SDK linkage hints.
|
||||
Check the host environment for a working sanderling setup.
|
||||
|
||||
```
|
||||
sanderling doctor [--platform web|android|ios|all]
|
||||
```
|
||||
|
||||
`--platform` defaults to `all`, which runs every platform's checks (deduped). Pass a specific platform to scope the output.
|
||||
|
||||
| Platform | Checks |
|
||||
|---|---|
|
||||
| `web` | headless Chromium can launch (the bundled CDP surface boots a real browser). |
|
||||
| `android` | `adb` on PATH; `emulator` on PATH or under `ANDROID_HOME`; Java 17+; embedded native sidecar JAR is real. |
|
||||
| `ios` | `xcrun` on PATH; `simctl` on PATH; Java 17+; embedded native sidecar JAR is real. |
|
||||
|
||||
## `sanderling version`
|
||||
|
||||
|
||||
@@ -35,9 +35,11 @@ npm install --save-dev @sanderling/spec
|
||||
|
||||
## Your first run
|
||||
|
||||
The repo ships two sample apps. `examples/folio` is a Kotlin Multiplatform personal-ledger app that covers Android, iOS, and web (wasmJs) from one shared codebase. `examples/folio-web` is a smaller React + Vite app that covers only the web path. Both carry a TypeScript spec under `sanderling/spec.ts`. Install `just`, then pick a target below.
|
||||
|
||||
### Android
|
||||
|
||||
The repo ships a working sample at `examples/folio`, a Kotlin Multiplatform app with a TypeScript spec under `sanderling/spec.ts`. Install `just`, then from `examples/folio`:
|
||||
From `examples/folio`:
|
||||
|
||||
```sh
|
||||
just install # build and install the folio APK on a booted emulator or device
|
||||
@@ -52,15 +54,32 @@ AVD=Pixel_7 just test
|
||||
|
||||
Persistent settings can live in a `.env` alongside the justfile (`AVD=Pixel_7`, `DURATION=5m`, and so on).
|
||||
|
||||
### Web
|
||||
### iOS
|
||||
|
||||
The repo also ships a web sample at `examples/folio-web`, a React/Vite app with the same domain logic. From `examples/folio-web`:
|
||||
From `examples/folio` (requires Xcode 16+ and `xcodegen`):
|
||||
|
||||
```sh
|
||||
just test # starts Chrome, runs the spec
|
||||
just test-ios # default simulator: iPhone 17 Pro
|
||||
IOS_DEVICE="iPhone 15" just test-ios # pick a different simulator
|
||||
```
|
||||
|
||||
No emulator or SDK setup needed.
|
||||
`just test-ios` boots the simulator if needed, builds and installs the app, then runs `sanderling test --platform ios`.
|
||||
|
||||
### Web
|
||||
|
||||
From either example. For the KMP wasmJs build, use `examples/folio`:
|
||||
|
||||
```sh
|
||||
just web # serve the wasmJs app on a webpack dev server
|
||||
```
|
||||
|
||||
For the React + Vite build, use `examples/folio-web`:
|
||||
|
||||
```sh
|
||||
just test # starts the Vite dev server, then sanderling drives Chrome via CDP
|
||||
```
|
||||
|
||||
No emulator or SDK setup needed for either web path.
|
||||
|
||||
### Trace output
|
||||
|
||||
|
||||
@@ -14,6 +14,19 @@ The positional argument can be a runs directory or a single run directory (auto-
|
||||
|
||||

|
||||
|
||||
## Panels
|
||||
|
||||
| Panel | What it shows |
|
||||
|---|---|
|
||||
| Screenshot | Device screenshot for the focused step, with the dispatched action's target spotlit. |
|
||||
| ActionList | Ordered list of steps with the verb and target for each action. |
|
||||
| Timeline | Per-property lane chart over the run; cells coloured by violated, pending, or holds. |
|
||||
| ViolationsPanel | Property statuses at the focused step with residual formulas for any that failed. |
|
||||
| HierarchyPanel | Filterable table of every UI element captured at the focused step; mirrors the selectors documented in [Spec language reference](./spec-language/). |
|
||||
| SnapshotTable | Flattened `snapshots` map for the focused step, with diffs against the previous step. |
|
||||
| MetricsChart | Heap and other host-side metrics sampled per step. |
|
||||
| ExceptionsPanel | Uncaught exceptions captured during the run, with a jump-to-first control. |
|
||||
|
||||
## Keyboard shortcuts
|
||||
|
||||
| Key | Action |
|
||||
|
||||
+5
-1
@@ -13,7 +13,7 @@ A run is not analogous to a unit test. A closer framing is: boot a fuzzer for an
|
||||
```
|
||||
sanderling test --spec spec.ts --bundle-id com.example.app --duration 30m
|
||||
│
|
||||
├── uninstall and reinstall the app (clean slate, every run)
|
||||
├── launch the app under test (pass --clear-data to wipe app data first)
|
||||
├── boot the sidecar, connect the agent socket
|
||||
├── bundle the spec, load it into goja
|
||||
│
|
||||
@@ -26,6 +26,10 @@ sanderling test --spec spec.ts --bundle-id com.example.app --duration 30m
|
||||
└── meta.json
|
||||
```
|
||||
|
||||
## App state across runs
|
||||
|
||||
By default the installed app is left in place between runs. Whatever state the previous run left behind (account, cached responses, onboarding completion) carries over. Pass `--clear-data` to wipe app data before launch and start cold every run. See [CLI reference](./cli/#sanderling-test) for the flag.
|
||||
|
||||
## Why runs are long and linear
|
||||
|
||||
sanderling does not restart the app every N steps. Each restart throws away two things.
|
||||
|
||||
@@ -125,7 +125,7 @@ Fields available on every element returned by `find` / `findAll`:
|
||||
### iOS
|
||||
|
||||
- `id` maps to the `accessibilityIdentifier` set via `.accessibilityIdentifier` in SwiftUI/UIKit.
|
||||
- `desc` maps to `accessibilityText`, which Maestro builds by merging `accessibilityLabel` and the element's value (e.g., `"Close, icon description"`). The `desc:<value>` selector handles this by also matching when the description starts with `<value>, `.
|
||||
- `desc` maps to `accessibilityText`, which the iOS sidecar builds by merging `accessibilityLabel` and the element's value (e.g., `"Close, icon description"`). The `desc:<value>` selector handles this by also matching when the description starts with `<value>, `.
|
||||
- `class` is the XCUITest element type (e.g., `XCUIElementTypeButton`).
|
||||
- `attrs` contains raw XCUITest attributes: `title`, `placeholderValue`, `hasFocus`, etc.
|
||||
|
||||
|
||||
@@ -183,10 +183,12 @@ import { noUncaughtExceptions, noLogcatErrors } from "@sanderling/spec/defaults/
|
||||
|
||||
export const properties = {
|
||||
noUncaughtExceptions, // fails if the app throws an uncaught exception
|
||||
noLogcatErrors, // fails if logcat emits any error-level lines
|
||||
noLogcatErrors, // android-only; reads logcat, no-ops on ios/web
|
||||
};
|
||||
```
|
||||
|
||||
`noLogcatErrors` reads from logcat and only applies on Android. Including it in a spec that targets iOS or web is harmless; it silently holds.
|
||||
|
||||
## Pattern: preconditions
|
||||
|
||||
sanderling has no setup phase. Preconditions are action generators with high weight that self-disable once the condition is satisfied.
|
||||
@@ -221,6 +223,25 @@ export const actionsRoot = weighted(
|
||||
|
||||
Once `onLoginScreen.current` is false, `doLogin` returns `[]` and drops out of the eligible set automatically.
|
||||
|
||||
## Pattern: setup export
|
||||
|
||||
Preconditions that drive the app from a fresh state into the surface you actually want to fuzz (login, onboarding, permission grants, seed data) can be exported as `setup` instead of mixing into `actionsRoot`. The runner tries `setup` first; if it yields no action, it falls through to `actionsRoot`. State regressing back across the precondition (logout under fuzz) automatically re-engages setup.
|
||||
|
||||
```ts
|
||||
const login = actions(() => {
|
||||
if (loggedIn.current) return [];
|
||||
const email = loginEmailField.current;
|
||||
const submit = loginSubmit.current;
|
||||
if (!email || !submit) return [];
|
||||
return [InputText({ into: email, text: "[email protected]" }), Tap({ on: submit })];
|
||||
});
|
||||
|
||||
export const setup = login;
|
||||
export const actionsRoot = weighted([60, browse], [40, edit]);
|
||||
```
|
||||
|
||||
`setup` is just an `ActionGenerator`; compose with `actions`, `weighted`, or `whenRoute` exactly like the main pool. Works identically across Android, iOS, and web.
|
||||
|
||||
## Pattern: conditional properties
|
||||
|
||||
Gate a property so it only applies when a precondition holds:
|
||||
|
||||
Reference in new issue
Block a user