mirror of
https://github.com/priyanshujain/sanderling.git
synced 2026-10-02 19:17:10 +00:00
ci: pin calibrated seeds, skip the flaky ios reinstall, bound every job
This commit is contained in:
1 parent
83285f0c30
commit
a632dbc406
4 files changed
+25
-6
No files matched your search
@@ -27,8 +27,13 @@ case "$platform" in
|
|||||||
examples/folio/app/androidApp/build/outputs/apk/debug/androidApp-debug.apk)
|
examples/folio/app/androidApp/build/outputs/apk/debug/androidApp-debug.apk)
|
||||||
;;
|
;;
|
||||||
ios)
|
ios)
|
||||||
|
# --clear-data=false because the caller has just installed a fresh build (a
|
||||||
|
# freshly installed app IS clear state). The in-run reinstall path is worth
|
||||||
|
# avoiding here: `simctl uninstall` + `install` immediately followed by the
|
||||||
|
# XCTest runner's own launch hits "app.folio is unknown to FrontBoard"
|
||||||
|
# perhaps half the time, and the run then hangs rather than failing.
|
||||||
folio_args+=(--platform ios
|
folio_args+=(--platform ios
|
||||||
--ios-app-path examples/folio/app/iosApp/build/Build/Products/Debug-iphonesimulator/iosApp.app
|
--clear-data=false
|
||||||
--ios-device "${IOS_DEVICE:-iPhone 16 Pro}")
|
--ios-device "${IOS_DEVICE:-iPhone 16 Pro}")
|
||||||
;;
|
;;
|
||||||
web)
|
web)
|
||||||
|
|||||||
@@ -37,6 +37,7 @@ on:
|
|||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
android:
|
android:
|
||||||
|
timeout-minutes: 90
|
||||||
if: ${{ inputs.platforms == 'all' || inputs.platforms == 'android' }}
|
if: ${{ inputs.platforms == 'all' || inputs.platforms == 'android' }}
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
@@ -111,6 +112,7 @@ jobs:
|
|||||||
retention-days: 14
|
retention-days: 14
|
||||||
|
|
||||||
ios:
|
ios:
|
||||||
|
timeout-minutes: 90
|
||||||
if: ${{ inputs.platforms == 'all' || inputs.platforms == 'ios' }}
|
if: ${{ inputs.platforms == 'all' || inputs.platforms == 'ios' }}
|
||||||
runs-on: macos-15
|
runs-on: macos-15
|
||||||
steps:
|
steps:
|
||||||
@@ -192,6 +194,7 @@ jobs:
|
|||||||
retention-days: 14
|
retention-days: 14
|
||||||
|
|
||||||
web:
|
web:
|
||||||
|
timeout-minutes: 60
|
||||||
if: ${{ inputs.platforms == 'all' || inputs.platforms == 'web' }}
|
if: ${{ inputs.platforms == 'all' || inputs.platforms == 'web' }}
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
|
|||||||
@@ -20,6 +20,7 @@ on:
|
|||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
dogfood:
|
dogfood:
|
||||||
|
timeout-minutes: 45
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@v4
|
||||||
|
|||||||
+15
-5
@@ -42,18 +42,28 @@ knew "non-zero" could not tell a working fuzzer from a broken emulator.
|
|||||||
|
|
||||||
**web is a health gate**, not an expect-the-bug leg. The same spec logs into the
|
**web is a health gate**, not an expect-the-bug leg. The same spec logs into the
|
||||||
wasmJs build and drives it to the transaction screen (the job asserts the trace
|
wasmJs build and drives it to the transaction screen (the job asserts the trace
|
||||||
reached `AddTransactionScreen`), but the submit property cannot fire there. It
|
reached `AddTransactionScreen`), but the submit property cannot fire there: it
|
||||||
keys off `state.lastAction`, which the web runtime reports as `null`, and off the
|
keys off `state.lastAction`, which the web runtime reports as `null`, and off the
|
||||||
action's selector, which the web picker does not carry - it emits coordinates,
|
action's selector, which the web picker does not carry, since it emits
|
||||||
and element identity never crosses the V8 boundary. Both are fixable; neither is
|
coordinates and element identity never crosses the V8 boundary.
|
||||||
a small fix, and until they are, asserting exit 2 on web would be asserting
|
|
||||||
something the spec cannot observe.
|
|
||||||
|
|
||||||
The wasmJs app is served with `Cross-Origin-Opener-Policy` and
|
The wasmJs app is served with `Cross-Origin-Opener-Policy` and
|
||||||
`Cross-Origin-Embedder-Policy` headers, because its sqlite worker needs
|
`Cross-Origin-Embedder-Policy` headers, because its sqlite worker needs
|
||||||
cross-origin isolation. Served without them the app loads a blank canvas and
|
cross-origin isolation. Served without them the app loads a blank canvas and
|
||||||
every step observes an empty accessibility tree.
|
every step observes an empty accessibility tree.
|
||||||
|
|
||||||
|
The seeds in the workflow are calibrated, not guessed. On an M-series mac,
|
||||||
|
android seed 3 finds the bug at step 110-116 (seeds 1, 2 and 4 run 120 steps
|
||||||
|
clean) and ios seed 1 finds it at step 129-134, both against a 240-step budget.
|
||||||
|
The web leg ran five seeds x 200 steps clean.
|
||||||
|
|
||||||
|
The ios leg passes `--clear-data=false`, because the job installs a fresh build
|
||||||
|
immediately before the run and a freshly installed app is already clear state.
|
||||||
|
The in-run reinstall is worth avoiding: `simctl uninstall` + `install` followed
|
||||||
|
straight away by the XCTest runner's own launch fails with `app.folio is unknown
|
||||||
|
to FrontBoard` maybe half the time, and the run then hangs rather than failing.
|
||||||
|
The job timeouts are the backstop if it happens anyway.
|
||||||
|
|
||||||
## replay-ui
|
## replay-ui
|
||||||
|
|
||||||
Actions -> replay-ui -> Run workflow. This one is dogfooding: it records a trace
|
Actions -> replay-ui -> Run workflow. This one is dogfooding: it records a trace
|
||||||
|
|||||||
Reference in new issue
Block a user