mirror of
https://github.com/priyanshujain/sanderling.git
synced 2026-10-02 11:07:10 +00:00
ci: pin calibrated seeds, skip the flaky ios reinstall, bound every job
This commit is contained in:
1 parent
83285f0c30
commit
a632dbc406
4 files changed
+25
-6
No files matched your search
@@ -27,8 +27,13 @@ case "$platform" in
|
||||
examples/folio/app/androidApp/build/outputs/apk/debug/androidApp-debug.apk)
|
||||
;;
|
||||
ios)
|
||||
# --clear-data=false because the caller has just installed a fresh build (a
|
||||
# freshly installed app IS clear state). The in-run reinstall path is worth
|
||||
# avoiding here: `simctl uninstall` + `install` immediately followed by the
|
||||
# XCTest runner's own launch hits "app.folio is unknown to FrontBoard"
|
||||
# perhaps half the time, and the run then hangs rather than failing.
|
||||
folio_args+=(--platform ios
|
||||
--ios-app-path examples/folio/app/iosApp/build/Build/Products/Debug-iphonesimulator/iosApp.app
|
||||
--clear-data=false
|
||||
--ios-device "${IOS_DEVICE:-iPhone 16 Pro}")
|
||||
;;
|
||||
web)
|
||||
|
||||
@@ -37,6 +37,7 @@ on:
|
||||
|
||||
jobs:
|
||||
android:
|
||||
timeout-minutes: 90
|
||||
if: ${{ inputs.platforms == 'all' || inputs.platforms == 'android' }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
@@ -111,6 +112,7 @@ jobs:
|
||||
retention-days: 14
|
||||
|
||||
ios:
|
||||
timeout-minutes: 90
|
||||
if: ${{ inputs.platforms == 'all' || inputs.platforms == 'ios' }}
|
||||
runs-on: macos-15
|
||||
steps:
|
||||
@@ -192,6 +194,7 @@ jobs:
|
||||
retention-days: 14
|
||||
|
||||
web:
|
||||
timeout-minutes: 60
|
||||
if: ${{ inputs.platforms == 'all' || inputs.platforms == 'web' }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
|
||||
@@ -20,6 +20,7 @@ on:
|
||||
|
||||
jobs:
|
||||
dogfood:
|
||||
timeout-minutes: 45
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
+15
-5
@@ -42,18 +42,28 @@ knew "non-zero" could not tell a working fuzzer from a broken emulator.
|
||||
|
||||
**web is a health gate**, not an expect-the-bug leg. The same spec logs into the
|
||||
wasmJs build and drives it to the transaction screen (the job asserts the trace
|
||||
reached `AddTransactionScreen`), but the submit property cannot fire there. It
|
||||
reached `AddTransactionScreen`), but the submit property cannot fire there: it
|
||||
keys off `state.lastAction`, which the web runtime reports as `null`, and off the
|
||||
action's selector, which the web picker does not carry - it emits coordinates,
|
||||
and element identity never crosses the V8 boundary. Both are fixable; neither is
|
||||
a small fix, and until they are, asserting exit 2 on web would be asserting
|
||||
something the spec cannot observe.
|
||||
action's selector, which the web picker does not carry, since it emits
|
||||
coordinates and element identity never crosses the V8 boundary.
|
||||
|
||||
The wasmJs app is served with `Cross-Origin-Opener-Policy` and
|
||||
`Cross-Origin-Embedder-Policy` headers, because its sqlite worker needs
|
||||
cross-origin isolation. Served without them the app loads a blank canvas and
|
||||
every step observes an empty accessibility tree.
|
||||
|
||||
The seeds in the workflow are calibrated, not guessed. On an M-series mac,
|
||||
android seed 3 finds the bug at step 110-116 (seeds 1, 2 and 4 run 120 steps
|
||||
clean) and ios seed 1 finds it at step 129-134, both against a 240-step budget.
|
||||
The web leg ran five seeds x 200 steps clean.
|
||||
|
||||
The ios leg passes `--clear-data=false`, because the job installs a fresh build
|
||||
immediately before the run and a freshly installed app is already clear state.
|
||||
The in-run reinstall is worth avoiding: `simctl uninstall` + `install` followed
|
||||
straight away by the XCTest runner's own launch fails with `app.folio is unknown
|
||||
to FrontBoard` maybe half the time, and the run then hangs rather than failing.
|
||||
The job timeouts are the backstop if it happens anyway.
|
||||
|
||||
## replay-ui
|
||||
|
||||
Actions -> replay-ui -> Run workflow. This one is dogfooding: it records a trace
|
||||
|
||||
Reference in new issue
Block a user