mirror of
https://github.com/priyanshujain/sanderling.git
synced 2026-10-02 19:17:10 +00:00
* feat(runner): stop the step loop at the first violation on request
* feat(testrun): report violations as a typed error under exit-on-violation
* feat(cli): add --exit-on-violation and exit 2 when it fires
* docs(cli): document --exit-on-violation, --max-steps, and exit codes
* fix(web): enumerate and query across shadow roots in both producers
* test(chrome): compare both producers on a shadow-dom parity page
* test(browser): drive a canvas-under-shadow-root fixture end to end
* fix(web): select the focused field inside a shadow root before typing
* fix(web): report the pathname as the screen when there is no hash route
* fix(web): settle on dom quiescence instead of returning at body ready
* feat(replay-ui): add data-testid hooks the dogfood spec drives
* feat(replay-ui): add the dogfood spec sanderling runs against the replay ui
* fix(replay-ui): scope the screenshot property to the named state panel
* chore(make): add per-platform sanderling build targets
* ci: add dispatch workflows for folio and the replay ui
* docs: describe the dispatch workflows and how to read a failure
* ci(folio): give the ios leg its jdk, android sdk, just, and a clean app start
* refactor(web): use max for the settle budget
* ci: pin calibrated seeds, skip the flaky ios reinstall, bound every job
* ci: authenticate and pin the buf setup step
the anonymous release download hit the shared runner ip rate limit and
failed the job with 'socket hang up' after three retries.
* docs: record that canvas apps need a dom proxy to be text-fuzzable
* fix(ios): bound lifecycle rpcs and claim the target device
a launch the simulator rejects sent the xctest session into a recovery
chain that answered minutes late or never, and the rpc had no deadline,
so the run hung with no trace and no error. also take a per-udid flock:
a second run's reinstall lands under the first's live automation session
and wedges it.
* docs(ci): correct the ios hang wording and note the device lock
* refactor(verifier): derive the lastAction shape from one field list
both hosts must show a spec the same lastAction. one ordered list now
feeds the goja object and the json the web host installs, so they
cannot drift.
* fix(web): install lastAction in the page before extractors read it
state.lastAction was hardcoded null on web, so every property reading it
was silently vacuous: a correct property passed without ever firing.
* fix(web): carry element identity on actions and fix findAll on paths
an action's target was coordinates only, so a property matching on which
element was acted upon could never fire. ax.findAll([a,b]) also returned
nothing on web.
* fix(chrome): wait out a route transition before sampling facts
the tree stays byte-identical and quiet across a cross-fade, so both the
quiet timer and the unchanged-tree escape called it settled mid-flight
and extractors read two screens at once.
* fix: bound the pre-run app launch
launch happens before the runner starts, so --duration never covered it
and a wedged driver hung with no trace and no error.
* fix(folio): read balances from merged cards and treat unreadable as unknown
compose for web merges the whole accountcard subtree, so the balance
child never exists there and every card parsed as 0. the property then
compared 0 to 0 and fired on any submit, which is a false positive
generator. unknown is now null and null is vacuously true.
* test(folio): cover merged-card parsing and unknown balances
* ci(folio): make web an expect-the-bug leg
the web runtime can observe the double submit now, so the health gate
understates it. seed 1 finds it at step 109, 3 runs out of 3.
* docs(ci): explain why a submit tap landing on home is the bug
* fix(ios): read a StaticText's label as its text
AXValue was the only source for text, but a StaticText carries its
string in AXLabel, so nothing on screen had .text on ios: a spec reading
it saw everything on android and nothing here.
* docs(ci): correct the calibrated step ranges
* fix(folio): stop convicting on arithmetic float64 cannot hold
past 2^53 cents the gap between representable values is 128, so a real
1600-cent move reads back as something else and the equality is false
for a healthy submit as readily as a double one. also match parseCents:
a sign or an oversized amount is rejected, not read as an amount.
* test(folio): pin the safe-integer guard and its boundary
* docs: stop teaching the zero-default that caused a false alarm
* docs: write down the silent-vacuity failure modes
* feat(folio): tag the home total and the card transaction count
the total was the only untagged node on the screen, so the spec had to
sum cards and a clipped card broke the sum.
* fix(folio): read the app's own total and refuse contaminated windows
summing cards went null when one was clipped, and the null poisoned the
carrier for the rest of the run. the balance window also spanned every
transaction since the last home visit, so the property convicted on
deltas it could not attribute: the old web witness was 3.16x the typed
amount, not 2x.
* test(folio): pin the window rules and the count invariant
* fix(folio): never read a frame that shows two screens
android dumps a cross-fade with both screens in the tree. the route said
add-transaction while an unscoped find said home, so the oracle took a
half-rendered total as fresh and convicted on a tap that committed
nothing. one function now decides the route and returns null when the
frame is ambiguous.
* test(folio): cover transition frames, card readings and creation
* fix(folio): only disambiguate counts that came from merged text
the equal-length digit rule exists because web merges the card and an
account named -1 makes '12' ambiguous. a dedicated count node has
nothing to disambiguate, so applying it there threw away real evidence.
* ci(folio): pin the recalibrated seeds and drop android to a health gate
web 3 and ios 7 convict 3 runs out of 3 with an exactly 2x witness.
android convicts 2 in 5 because the same seed does not walk the same
trajectory there, so it proves the app runs instead.
* docs(ci): describe the two properties and why android cannot convict
* fix(android): wait out a route cross-fade before snapshotting
the dump could hold two screens at once, and the runner refuses to act
on such a tree, so a quarter of android steps applied no action and the
count varied per run: the same seed never walked the same trajectory.
the ios companion and the chrome driver already do this.
* ci(folio): let the android leg run far enough to see its conviction
* docs: only the repo owner merges
* ci(folio): a thrown predicate is not a conviction
exit 2 means the run recorded a violation, and a predicate that throws
is recorded as one too. so was newAccountBalanceIsZero, an unrelated
property in the same spec. the gate read the exit code and went green
with detection dead.
* ci: install idb-companion from its tap and stop interpolating inputs
idb-companion is not in homebrew-core, so the ios leg died before it
built anything. replay-ui expanded dispatch inputs into the shell.
* docs: correct the snippets and numbers that drifted from the code
* test(sidecar): pin that a slow read counts toward the stability streak
* fix(web): read the page's extractors only on steps that count
the page advances the spec's carriers when it evaluates, but the runner
applied the result only on non-transitional steps. a discarded step
moved the window forward anyway, so the next accepted pair bracketed two
transactions while counting one submit, and convicted a healthy app.
extractor errors now fail the run instead of leaving goja's values in
current against v8's in previous.
* fix(chrome): anchor the transition deadline when the dom goes quiet
it was anchored at script start, so a page that churned past the window
reached the check already expired and returned mid cross-fade. the
driver now publishes the idle timeout it needs, since the caller's 1s
could never spend the 800ms window.
* fix(web): fail on a partial extractor override
same mixed-producer hazard as the install error: some extractors hold
the page's value and the rest hold goja's, and a property comparing
across that split fires on a healthy app.
* docs: six of seven, the seventh is the stock property
* fix(folio): drop a name two cards answer to
homeTxnCountsOf keyed on the account name and let the last card win, so
two accounts the fuzzer named the same collapsed into one entry. a
reading that saw one Travel card and a later one that saw both then
subtracted two different accounts' counts, and
submitCommitsOneTransactionPerAction convicted a healthy app of
double-submitting. it is a gated property in folio-run.sh, so that reads
as "found the submit bug" over a card scrolling into view.
same rule createdAccountHasNonZeroBalance already applies: a name
nothing can attribute is no evidence. counted over every card, since an
unreadable twin spoils the identity too.
* perf(folio): read each frame once
every extractor asked routeOf, and routeOf does five ax.find calls. on
web each find walks the document and every shadow root beneath it, so
the spec cost 110 tree walks a step; homeCards was parsed four times
over. now 5 and once.
keyed on the identity of the state object because both hosts build a new
one per step and hand that one object to every getter, so it cannot
outlive its frame. holding the reference is what keeps that true rather
than likely.
* fix(web): keep an undefined reading's index through JSON
json has no undefined, so an extractor whose getter returned one had its
whole index dropped by JSON.stringify. that index then kept goja's
dump-derived value while its neighbours held the page's, and a property
comparing previous to current across the split fires on a healthy app.
folio has nine on(route, tag) extractors, so this was most extractors on
most steps.
each reading is wrapped in a {value} envelope: the drop now happens
inside the entry, and an absent value means the getter returned
undefined, which is what the goja host records for the same getter. a
json null would instead claim it returned null and x.current ===
undefined would answer differently on the two hosts.
* feat(verifier): report the registered extractor count
the web path needs it to check the page sent one reading per extractor.
* fix(runner): fail when the page reports fewer readings than extractors
the comment here already claimed a partial override was fatal. it was
not: the skipped check only catches indices outside the extractor list,
so a page reporting values for some extractors and not others left the
rest holding goja's reading of the dump with nothing said.
* test(browser): drive an undefined reading through the whole web path
four layers carry it: the page's envelope, the driver's unwrap, the
runner's count check and the verifier's decode. each has a unit test and
only a run proves they compose. goes red both ways, decoding an absent
value as null and dropping the envelope.
* fix(web): offer the aria roles a user activates
only role=button was in the tappable set, so link, checkbox, radio,
switch, tab, option, the menuitems and treeitem were invisible to the
enumeration however plain the control looked. the replay ui builds its
step rows as <li role="option">, and the spec dogfooding it had to
hand-write an action to reach them because no default verb could see a
single row.
both producers build the set from the same role list, since the parity
test compares them element by element.
* test(browser): tap a role-based control end to end
every control on the page is an <li role="option">, the shape the
replay ui gives its step rows, and the spec carries no action of its
own: the property firing is the evidence the default enumeration offered
a tap on one.
* fix(web): read aria-disabled as disabled
the enabled fact came off the disabled property, which only real form
controls have. it reads undefined on the role-based controls the
tappable set now covers, so every one of them looked enabled however
plainly it was marked otherwise, and the fuzzer would spend actions on
inert ones.
both producers answer the same two ways, and the parity fixture carries
a disabled row so the comparison covers it: reverting one side alone
names the element and the fact.
* docs(replay-ui): the enumeration reaches step rows now
the comment said role="option" is not in the tappable selector set,
which stopped being true a few commits ago. selectAStep stays, for the
reason the tab weight below it stays: one row among the page's clickable
elements is a thin chance, and both step-facing properties go vacuous on
a run that never selects one.
* test(runner): bound the last-action test by steps, not wall clock
100ms of wall clock against an assertion that two steps ran fatals under
load with "the web path never installed it", which reads as a
regression. every sibling test in the package uses a long duration and
MaxSteps.
* ci: run the kotlin tests in make test
RouteTransitionTest and the stability poll cover the android settle and
nothing in ci ran them. :sidecar:test needs no android sdk, checked by
running it with ANDROID_HOME pointed at nothing.
* fix(sidecar): measure the stability streak as observed quiet
parameterising pollUntilStable also moved the clock to the start of the
read that opened a run of identical snapshots, so a read's own duration
counted as quiet. the pre-existing caller polls a real uiautomator dump:
at 400ms a read, 750ms of required quiet became 250ms of observed quiet
and the poll settled in two reads instead of four.
the parameters stay, the semantics go back.
* test(sidecar): pin the transition cap by driving it
it asserted 1500 >= 700 + 300, two constants, which can only fail if
someone edits a constant. it now drives awaitSettledTree against a fade
that lands after 700ms and asserts it hands back the settled tree before
the cap. cut the cap to 1000 and it goes red.
* ci: pin buf-setup-action to a commit
it takes a token now, so a floating tag is a token handed to whatever
that tag moves to. note v1 there is a branch, not a tag, so the ref
lookup that resolves it is matching-refs/heads/v1.
* ci: declare least-privilege permissions
none of the three declared any, so each got the repository default.
release.yml and docs.yml already do this. all three only check out,
build, test and upload artifacts.
* ci: fail fast when a server never comes up
the readiness loops fell through silently after 30 tries, so a server
that never started surfaced as an opaque driver failure minutes later.
each now says what did not answer and on which port.
* ci(folio): a missing trace is not a verdict
with no trace the android gate ran its grep against ./trace.jsonl and
reported "never reached AddTransactionScreen, so it never got past
login", which is not what happened. the web and ios branches had the
same misdiagnosis on exit 0.
same class, one line up: the classifier's own failure was swallowed, so
with the evidence reader dead the gate printed a healthy run and exited
0.
* ci(replay-ui): skip a run directory with no trace
the summarise step is if: always(), and under github's bash -eo pipefail
an unmatched glob stays literal, the redirect fails, pipefail carries it
into the assignment and -e kills the step. so a failed fuzz run went red
twice, once for the real reason.
968 lines
36 KiB
Go
968 lines
36 KiB
Go
//go:build browser
|
|
|
|
package chrome
|
|
|
|
import (
|
|
"context"
|
|
"encoding/json"
|
|
"errors"
|
|
"net"
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"strings"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/chromedp/chromedp"
|
|
)
|
|
|
|
// TestLaunch_ClearStateWipesStorageForTheTargetOrigin covers the CLI's default
|
|
// path (--clear-data). The tab sits on about:blank when Launch runs, an opaque
|
|
// origin that denies storage access, so clearing by script there throws
|
|
// SecurityError and kills every web run before the app loads.
|
|
func TestLaunch_ClearStateWipesStorageForTheTargetOrigin(t *testing.T) {
|
|
const page = `<body><script>
|
|
const visits = Number(localStorage.getItem("visits") ?? "0") + 1;
|
|
localStorage.setItem("visits", String(visits));
|
|
sessionStorage.setItem("tab", "dirty");
|
|
</script></body>`
|
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
|
w.Header().Set("Content-Type", "text/html")
|
|
_, _ = w.Write([]byte(page))
|
|
}))
|
|
defer server.Close()
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
|
|
defer cancel()
|
|
|
|
if err := d.Launch(ctx, server.URL, true, nil); err != nil {
|
|
t.Fatalf("Launch with clearState on a fresh tab: %v", err)
|
|
}
|
|
if err := d.Launch(ctx, server.URL, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
var visits string
|
|
if err := chromedp.Run(d.tabCtx,
|
|
chromedp.Evaluate(`localStorage.getItem("visits")`, &visits)); err != nil {
|
|
t.Fatalf("read localStorage: %v", err)
|
|
}
|
|
if visits != "2" {
|
|
t.Fatalf("visits = %q, want 2 (two loads, storage kept)", visits)
|
|
}
|
|
|
|
if err := d.Launch(ctx, server.URL, true, nil); err != nil {
|
|
t.Fatalf("Launch with clearState on the target origin: %v", err)
|
|
}
|
|
if err := chromedp.Run(d.tabCtx,
|
|
chromedp.Evaluate(`localStorage.getItem("visits")`, &visits)); err != nil {
|
|
t.Fatalf("read localStorage: %v", err)
|
|
}
|
|
if visits != "1" {
|
|
t.Errorf("visits = %q, want 1 (storage cleared before the app loaded)", visits)
|
|
}
|
|
}
|
|
|
|
// TestLaunch_WebGLContextIsAvailable pins the SwiftShader fallback. Headless
|
|
// Chrome runs with --disable-gpu, and without --enable-unsafe-swiftshader it
|
|
// refuses the software WebGL backend: getContext returns null, so a
|
|
// canvas-rendered app paints nothing and every screenshot is identical black.
|
|
func TestLaunch_WebGLContextIsAvailable(t *testing.T) {
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, "data:text/html,<body></body>", false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
var hasContext bool
|
|
if err := chromedp.Run(d.tabCtx, chromedp.Evaluate(
|
|
`!!document.createElement("canvas").getContext("webgl2")`, &hasContext)); err != nil {
|
|
t.Fatalf("evaluate: %v", err)
|
|
}
|
|
if !hasContext {
|
|
t.Error("webgl2 context is null; a canvas-rendered app would render nothing")
|
|
}
|
|
}
|
|
|
|
// TestActionMethods_HonorCallerCancellation confirms the DeviceDriver action
|
|
// methods route through runCtx so a cancelled caller context aborts the CDP
|
|
// round-trip instead of blocking on d.tabCtx. Without this a hung browser would
|
|
// ignore step deadlines and Ctrl-C.
|
|
func TestActionMethods_HonorCallerCancellation(t *testing.T) {
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
launchCtx, launchCancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer launchCancel()
|
|
if err := d.Launch(launchCtx, "data:text/html,<body><button id=go>go</button></body>", false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
|
|
cancelled, cancel := context.WithCancel(context.Background())
|
|
cancel()
|
|
|
|
actions := map[string]func() error{
|
|
"Tap": func() error { return d.Tap(cancelled, 1, 1) },
|
|
"Swipe": func() error { return d.Swipe(cancelled, 1, 1, 2, 2, 50*time.Millisecond) },
|
|
"LongPress": func() error { return d.LongPress(cancelled, 1, 1) },
|
|
"PressKey": func() error { return d.PressKey(cancelled, "enter") },
|
|
"InputText": func() error { return d.InputText(cancelled, "x") },
|
|
"EraseText": func() error { return d.EraseText(cancelled, 1) },
|
|
"TapSelector": func() error { return d.TapSelector(cancelled, "#go") },
|
|
}
|
|
for name, action := range actions {
|
|
t.Run(name, func(t *testing.T) {
|
|
done := make(chan error, 1)
|
|
go func() { done <- action() }()
|
|
select {
|
|
case err := <-done:
|
|
if !errors.Is(err, context.Canceled) {
|
|
t.Fatalf("want context.Canceled, got %v", err)
|
|
}
|
|
case <-time.After(5 * time.Second):
|
|
t.Fatal("action ignored cancelled caller ctx and blocked")
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestHierarchy_EditableFlag confirms the injected hierarchy script marks text
|
|
// inputs, textareas, and contenteditable elements editable while leaving
|
|
// buttons and non-text inputs alone.
|
|
func TestHierarchy_EditableFlag(t *testing.T) {
|
|
const html = `<body>` +
|
|
`<input id="name">` +
|
|
`<textarea id="bio"></textarea>` +
|
|
`<button id="go">go</button>` +
|
|
`<div id="rich" contenteditable="true">x</div>` +
|
|
`<input id="chk" type="checkbox">` +
|
|
`</body>`
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, "data:text/html,"+html, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
dump, err := d.Hierarchy(ctx)
|
|
if err != nil {
|
|
t.Fatalf("Hierarchy: %v", err)
|
|
}
|
|
|
|
type node struct {
|
|
Attributes map[string]string `json:"attributes"`
|
|
Children []node `json:"children"`
|
|
Editable *bool `json:"editable"`
|
|
}
|
|
var root node
|
|
if err := json.Unmarshal([]byte(dump), &root); err != nil {
|
|
t.Fatalf("unmarshal hierarchy: %v", err)
|
|
}
|
|
editableByID := map[string]*bool{}
|
|
var walk func(n node)
|
|
walk = func(n node) {
|
|
if id := n.Attributes["resource-id"]; id != "" {
|
|
editableByID[id] = n.Editable
|
|
}
|
|
for _, c := range n.Children {
|
|
walk(c)
|
|
}
|
|
}
|
|
walk(root)
|
|
|
|
isEditable := func(id string) bool {
|
|
return editableByID[id] != nil && *editableByID[id]
|
|
}
|
|
for _, id := range []string{"name", "bio", "rich"} {
|
|
if !isEditable(id) {
|
|
t.Errorf("%q: editable = %v, want true", id, editableByID[id])
|
|
}
|
|
}
|
|
for _, id := range []string{"go", "chk"} {
|
|
if isEditable(id) {
|
|
t.Errorf("%q: editable = true, want false/absent", id)
|
|
}
|
|
}
|
|
}
|
|
|
|
// TestHierarchy_ClickableMatchesTheWebRuntimeSelector pins clickable to the
|
|
// same membership test pkg/spec/src/web-runtime.ts applies. The dump used to
|
|
// test el.onclick, which React sets on its root container for event delegation,
|
|
// so the whole viewport became a tap target in this dump and in no other
|
|
// enumeration of the same page.
|
|
func TestHierarchy_ClickableMatchesTheWebRuntimeSelector(t *testing.T) {
|
|
const html = `<body>` +
|
|
`<div id="root"><button id="go">go</button></div>` +
|
|
`<textarea id="bio"></textarea>` +
|
|
`<div id="plain">text</div>` +
|
|
`<div id="rolebutton" role="button">act</div>` +
|
|
`<script>document.getElementById("root").onclick = function () {};</script>` +
|
|
`</body>`
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, "data:text/html,"+html, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
dump, err := d.Hierarchy(ctx)
|
|
if err != nil {
|
|
t.Fatalf("Hierarchy: %v", err)
|
|
}
|
|
|
|
type node struct {
|
|
Attributes map[string]string `json:"attributes"`
|
|
Children []node `json:"children"`
|
|
Clickable *bool `json:"clickable"`
|
|
}
|
|
var root node
|
|
if err := json.Unmarshal([]byte(dump), &root); err != nil {
|
|
t.Fatalf("unmarshal hierarchy: %v", err)
|
|
}
|
|
clickableByID := map[string]*bool{}
|
|
var walk func(n node)
|
|
walk = func(n node) {
|
|
if id := n.Attributes["resource-id"]; id != "" {
|
|
clickableByID[id] = n.Clickable
|
|
}
|
|
for _, c := range n.Children {
|
|
walk(c)
|
|
}
|
|
}
|
|
walk(root)
|
|
|
|
isClickable := func(id string) bool {
|
|
return clickableByID[id] != nil && *clickableByID[id]
|
|
}
|
|
for _, id := range []string{"go", "bio", "rolebutton"} {
|
|
if !isClickable(id) {
|
|
t.Errorf("%q: clickable = %v, want true", id, clickableByID[id])
|
|
}
|
|
}
|
|
for _, id := range []string{"root", "plain"} {
|
|
if isClickable(id) {
|
|
t.Errorf("%q: clickable = true, want false/absent (an onclick property is not a target)", id)
|
|
}
|
|
}
|
|
}
|
|
|
|
// TestHierarchy_ScrollableAttribute covers the fact the goja host reads off the
|
|
// attributes map. The V8 picker computes the same overflow test in
|
|
// web-runtime.ts, so leaving it out of the dump made the two enumerations
|
|
// disagree on web: the model policy could never be offered a scroll the seeded
|
|
// policy could draw.
|
|
func TestHierarchy_ScrollableAttribute(t *testing.T) {
|
|
const html = `<body style="margin:0">` +
|
|
`<div id="overflowing" style="width:100px;height:100px;overflow:auto">` +
|
|
`<div style="width:100px;height:900px"></div></div>` +
|
|
`<div id="sideways" style="width:100px;height:50px;overflow:auto">` +
|
|
`<div style="width:900px;height:20px"></div></div>` +
|
|
`<div id="fits" style="width:100px;height:100px;overflow:auto">` +
|
|
`<div style="width:50px;height:50px"></div></div>` +
|
|
`</body>`
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, "data:text/html,"+html, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
dump, err := d.Hierarchy(ctx)
|
|
if err != nil {
|
|
t.Fatalf("Hierarchy: %v", err)
|
|
}
|
|
|
|
type node struct {
|
|
Attributes map[string]string `json:"attributes"`
|
|
Children []node `json:"children"`
|
|
}
|
|
var root node
|
|
if err := json.Unmarshal([]byte(dump), &root); err != nil {
|
|
t.Fatalf("unmarshal hierarchy: %v", err)
|
|
}
|
|
scrollableByID := map[string]string{}
|
|
var walk func(n node)
|
|
walk = func(n node) {
|
|
if id := n.Attributes["resource-id"]; id != "" {
|
|
scrollableByID[id] = n.Attributes["scrollable"]
|
|
}
|
|
for _, c := range n.Children {
|
|
walk(c)
|
|
}
|
|
}
|
|
walk(root)
|
|
|
|
for _, id := range []string{"overflowing", "sideways"} {
|
|
if scrollableByID[id] != "true" {
|
|
t.Errorf("%q: scrollable = %q, want \"true\"", id, scrollableByID[id])
|
|
}
|
|
}
|
|
if scrollableByID["fits"] != "" {
|
|
t.Errorf("%q: scrollable = %q, want absent", "fits", scrollableByID["fits"])
|
|
}
|
|
}
|
|
|
|
// TestRunCtx_CallerCancelPropagates confirms that cancelling the caller's
|
|
// context cancels the chromedp-bound context returned by runCtx. This is the
|
|
// channel by which step deadlines and Ctrl-C reach in-flight CDP calls.
|
|
func TestRunCtx_CallerCancelPropagates(t *testing.T) {
|
|
tabCtx, tabCancel := context.WithCancel(context.Background())
|
|
defer tabCancel()
|
|
d := &Driver{tabCtx: tabCtx}
|
|
|
|
callerCtx, callerCancel := context.WithCancel(context.Background())
|
|
derived, cancel := d.runCtx(callerCtx)
|
|
defer cancel()
|
|
|
|
callerCancel()
|
|
select {
|
|
case <-derived.Done():
|
|
case <-time.After(time.Second):
|
|
t.Fatal("derived ctx did not cancel after caller cancellation")
|
|
}
|
|
}
|
|
|
|
// TestRunCtx_TabCancelPropagates confirms the inverse: tearing down the tab
|
|
// also cancels any in-flight derived context.
|
|
func TestRunCtx_TabCancelPropagates(t *testing.T) {
|
|
tabCtx, tabCancel := context.WithCancel(context.Background())
|
|
d := &Driver{tabCtx: tabCtx}
|
|
|
|
derived, cancel := d.runCtx(context.Background())
|
|
defer cancel()
|
|
|
|
tabCancel()
|
|
select {
|
|
case <-derived.Done():
|
|
case <-time.After(time.Second):
|
|
t.Fatal("derived ctx did not cancel after tab cancellation")
|
|
}
|
|
}
|
|
|
|
// TestHierarchy_RootsAtDocumentElementWithoutHead pins the dump's root to html,
|
|
// because collectTargets in pkg/spec/src/web-runtime.ts walks
|
|
// querySelectorAll("*") and therefore sees html. A dump rooted at body hides
|
|
// page-level scrolling, which lives on html on a standard page, so the two
|
|
// enumerations disagree on exactly that candidate. The head subtree stays out:
|
|
// it is all zero-bounds, so it changes no eligible set, and it would otherwise
|
|
// carry script and title text into the trace.
|
|
func TestHierarchy_RootsAtDocumentElementWithoutHead(t *testing.T) {
|
|
const html = `<html><head><title>secret title</title>` +
|
|
`<script>window.marker = 1;</script></head>` +
|
|
`<body><div id="content">hello</div></body></html>`
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, "data:text/html,"+html, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
dump, err := d.Hierarchy(ctx)
|
|
if err != nil {
|
|
t.Fatalf("Hierarchy: %v", err)
|
|
}
|
|
|
|
type node struct {
|
|
Attributes map[string]string `json:"attributes"`
|
|
Children []node `json:"children"`
|
|
}
|
|
var root node
|
|
if err := json.Unmarshal([]byte(dump), &root); err != nil {
|
|
t.Fatalf("unmarshal hierarchy: %v", err)
|
|
}
|
|
if root.Attributes["tag"] != "html" {
|
|
t.Errorf("root tag: got %q, want html", root.Attributes["tag"])
|
|
}
|
|
if root.Attributes["sanderling-screen"] == "" {
|
|
t.Error("the root must still carry sanderling-screen")
|
|
}
|
|
|
|
tags := map[string]bool{}
|
|
var walk func(n node)
|
|
walk = func(n node) {
|
|
tags[n.Attributes["tag"]] = true
|
|
for _, child := range n.Children {
|
|
walk(child)
|
|
}
|
|
}
|
|
walk(root)
|
|
if !tags["body"] {
|
|
t.Error("body must appear under the root")
|
|
}
|
|
for _, tag := range []string{"head", "title", "script"} {
|
|
if tags[tag] {
|
|
t.Errorf("the head subtree must stay out of the dump, found %q", tag)
|
|
}
|
|
}
|
|
}
|
|
|
|
// TestLaunch_HonorsCallerDeadline points the driver at a listener that accepts
|
|
// the connection and never answers. Chrome has no page-load deadline of its
|
|
// own, so a Launch that ignored its caller context waited forever: an
|
|
// unattended campaign worker aimed at an unreachable target wedged with no
|
|
// diagnostic and did not even answer SIGTERM.
|
|
func TestLaunch_HonorsCallerDeadline(t *testing.T) {
|
|
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
|
if err != nil {
|
|
t.Fatalf("listen: %v", err)
|
|
}
|
|
defer listener.Close()
|
|
held := make(chan net.Conn, 8)
|
|
go func() {
|
|
for {
|
|
conn, err := listener.Accept()
|
|
if err != nil {
|
|
return
|
|
}
|
|
held <- conn
|
|
}
|
|
}()
|
|
defer func() {
|
|
close(held)
|
|
for conn := range held {
|
|
conn.Close()
|
|
}
|
|
}()
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
|
|
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
|
defer cancel()
|
|
done := make(chan error, 1)
|
|
go func() { done <- d.Launch(ctx, "http://"+listener.Addr().String(), false, nil) }()
|
|
select {
|
|
case err := <-done:
|
|
if err == nil {
|
|
t.Fatal("Launch returned nil against a target that never answers")
|
|
}
|
|
if !errors.Is(err, context.DeadlineExceeded) && !errors.Is(err, context.Canceled) {
|
|
t.Fatalf("Launch error = %v, want a context error", err)
|
|
}
|
|
case <-time.After(45 * time.Second):
|
|
t.Fatal("Launch ignored the caller deadline and blocked")
|
|
}
|
|
}
|
|
|
|
// TestLaunch_KeepsBrowserAliveAfterCallerContextEnds guards the trap that made
|
|
// Launch use the driver context in the first place: chromedp starts Chrome
|
|
// under whichever context runs first, so allocating under the caller's
|
|
// short-lived context would kill the browser as soon as Launch returned.
|
|
func TestLaunch_KeepsBrowserAliveAfterCallerContextEnds(t *testing.T) {
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
|
|
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
|
|
if err := d.Launch(ctx, "data:text/html,<body>alive</body>", false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
cancel()
|
|
|
|
var text string
|
|
if err := chromedp.Run(d.tabCtx,
|
|
chromedp.Evaluate(`document.body.textContent`, &text)); err != nil {
|
|
t.Fatalf("browser died with the caller context: %v", err)
|
|
}
|
|
if text != "alive" {
|
|
t.Errorf("body text = %q, want \"alive\"", text)
|
|
}
|
|
if err := d.Launch(context.Background(), "data:text/html,<body>again</body>", false, nil); err != nil {
|
|
t.Fatalf("second Launch after the first caller context ended: %v", err)
|
|
}
|
|
}
|
|
|
|
// TestInputText_ReplacesTextInsideAShadowRoot pins ReplacesTextOnInput's promise
|
|
// on the shape a canvas app actually has. Compose for Web draws its text fields
|
|
// on a canvas and routes typing through a hidden <input> INSIDE the shadow root
|
|
// it mounts, and document.activeElement stops at a shadow boundary: it names the
|
|
// host. The select-all therefore ran against a <div> with no select(), every
|
|
// InputText appended to the last, and a fuzzer typing twice into one field built
|
|
// up text it could never clear (observed on the folio wasm build as
|
|
// "0.0000001" -> "0.0000001\t-1").
|
|
func TestInputText_ReplacesTextInsideAShadowRoot(t *testing.T) {
|
|
const page = `<body><div id="app"></div><script>
|
|
const root = document.getElementById("app").attachShadow({mode: "open"});
|
|
root.innerHTML = ` + "`" + `
|
|
<style>
|
|
#surface { position: absolute; left: 0; top: 0; }
|
|
#a11y { position: absolute; left: 0; top: 0; pointer-events: none; }
|
|
#proxy { position: absolute; left: -9999px; }
|
|
</style>
|
|
<canvas id="surface" width="300" height="200"></canvas>
|
|
<div id="a11y"><div id="field">-</div></div>
|
|
<input id="proxy" type="text">` + "`" + `;
|
|
const proxy = root.getElementById("proxy");
|
|
const field = root.getElementById("field");
|
|
// The canvas owns the pointer (the a11y overlay is pointer-events: none)
|
|
// and hands focus to the proxy, exactly as a canvas app does.
|
|
root.getElementById("surface").addEventListener("click", function () { proxy.focus(); });
|
|
proxy.addEventListener("input", function () { field.textContent = proxy.value; });
|
|
</script></body>`
|
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
|
w.Header().Set("Content-Type", "text/html")
|
|
_, _ = w.Write([]byte(page))
|
|
}))
|
|
defer server.Close()
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, server.URL, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
if err := d.Tap(ctx, 40, 40); err != nil {
|
|
t.Fatalf("Tap: %v", err)
|
|
}
|
|
if err := d.InputText(ctx, "alpha"); err != nil {
|
|
t.Fatalf("InputText: %v", err)
|
|
}
|
|
if err := d.InputText(ctx, "beta"); err != nil {
|
|
t.Fatalf("InputText: %v", err)
|
|
}
|
|
|
|
var shown string
|
|
script := `document.getElementById("app").shadowRoot.getElementById("field").textContent`
|
|
if err := chromedp.Run(d.tabCtx, chromedp.Evaluate(script, &shown)); err != nil {
|
|
t.Fatalf("read field: %v", err)
|
|
}
|
|
if shown != "beta" {
|
|
t.Errorf("field holds %q, want %q; the second InputText appended instead of replacing", shown, "beta")
|
|
}
|
|
|
|
if err := d.EraseText(ctx, len("beta")); err != nil {
|
|
t.Fatalf("EraseText: %v", err)
|
|
}
|
|
if err := chromedp.Run(d.tabCtx, chromedp.Evaluate(script, &shown)); err != nil {
|
|
t.Fatalf("read field: %v", err)
|
|
}
|
|
if shown != "" {
|
|
t.Errorf("field holds %q after EraseText, want empty", shown)
|
|
}
|
|
}
|
|
|
|
// TestHierarchy_ScreenFallsBackToThePathname pins the route the goja host reads
|
|
// off the dump. Reading location.hash alone reported "/" on every step of a
|
|
// path-routed SPA (react-router's BrowserRouter, which the replay UI itself
|
|
// uses), so every screen looked like the same screen and no route-scoped
|
|
// property or action could tell them apart.
|
|
func TestHierarchy_ScreenFallsBackToThePathname(t *testing.T) {
|
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
|
w.Header().Set("Content-Type", "text/html")
|
|
_, _ = w.Write([]byte(`<body><div id="app">app</div></body>`))
|
|
}))
|
|
defer server.Close()
|
|
|
|
for _, testCase := range []struct {
|
|
name string
|
|
path string
|
|
want string
|
|
}{
|
|
{"path-routed", "/runs/20260101-120000/steps/7", "/runs/20260101-120000/steps/7"},
|
|
{"hash wins when present", "/runs/1#/detail", "/detail"},
|
|
{"root", "/", "/"},
|
|
} {
|
|
t.Run(testCase.name, func(t *testing.T) {
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, server.URL+testCase.path, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
dump, err := d.Hierarchy(ctx)
|
|
if err != nil {
|
|
t.Fatalf("Hierarchy: %v", err)
|
|
}
|
|
var root struct {
|
|
Attributes map[string]string `json:"attributes"`
|
|
}
|
|
if err := json.Unmarshal([]byte(dump), &root); err != nil {
|
|
t.Fatalf("unmarshal hierarchy: %v", err)
|
|
}
|
|
if got := root.Attributes["sanderling-screen"]; got != testCase.want {
|
|
t.Errorf("sanderling-screen: got %q, want %q", got, testCase.want)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestWaitForIdle_WaitsForWorkTheActionKickedOff pins the settle the runner
|
|
// relies on between acting and observing. WaitForIdle used to return the moment
|
|
// <body> existed, which is true before the app has reacted at all: measured on
|
|
// the folio wasm build, Compose's accessibility DOM lands ~136 ms after an
|
|
// InputText, so the next step read the pre-action text and typed into a field
|
|
// it believed was still empty.
|
|
func TestWaitForIdle_WaitsForWorkTheActionKickedOff(t *testing.T) {
|
|
const page = `<body><div id="app"></div><script>
|
|
const root = document.getElementById("app").attachShadow({mode: "open"});
|
|
root.innerHTML = '<button id="go" style="width:200px;height:80px">go</button><div id="out">pending</div>';
|
|
root.getElementById("go").addEventListener("click", function () {
|
|
setTimeout(function () { root.getElementById("out").textContent = "settled"; }, 100);
|
|
});
|
|
</script></body>`
|
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
|
w.Header().Set("Content-Type", "text/html")
|
|
_, _ = w.Write([]byte(page))
|
|
}))
|
|
defer server.Close()
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, server.URL, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
if err := d.Tap(ctx, 40, 40); err != nil {
|
|
t.Fatalf("Tap: %v", err)
|
|
}
|
|
if err := d.WaitForIdle(ctx, time.Second); err != nil {
|
|
t.Fatalf("WaitForIdle: %v", err)
|
|
}
|
|
|
|
var shown string
|
|
script := `document.getElementById("app").shadowRoot.getElementById("out").textContent`
|
|
if err := chromedp.Run(d.tabCtx, chromedp.Evaluate(script, &shown)); err != nil {
|
|
t.Fatalf("read: %v", err)
|
|
}
|
|
if shown != "settled" {
|
|
t.Errorf("observed %q; WaitForIdle returned before the tap's own work landed", shown)
|
|
}
|
|
}
|
|
|
|
// TestWaitForIdle_ReturnsOnABusyPage is the other half: a page that never stops
|
|
// mutating (an animation, a polling widget) must not hold the step loop open.
|
|
func TestWaitForIdle_ReturnsOnABusyPage(t *testing.T) {
|
|
const page = `<body><div id="tick">0</div><script>
|
|
let n = 0;
|
|
setInterval(function () { document.getElementById("tick").textContent = String(++n); }, 15);
|
|
</script></body>`
|
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
|
w.Header().Set("Content-Type", "text/html")
|
|
_, _ = w.Write([]byte(page))
|
|
}))
|
|
defer server.Close()
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, server.URL, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
start := time.Now()
|
|
if err := d.WaitForIdle(ctx, time.Second); err != nil {
|
|
t.Fatalf("WaitForIdle: %v", err)
|
|
}
|
|
if elapsed := time.Since(start); elapsed > 2*time.Second {
|
|
t.Errorf("WaitForIdle took %s on a busy page; it must return inside its budget", elapsed)
|
|
}
|
|
}
|
|
|
|
// TestWaitForIdle_WaitsOutARouteTransition covers the settle case a mutation
|
|
// observer cannot see. A canvas app splices the incoming screen's
|
|
// accessibility nodes in when its cross-fade STARTS and drops the outgoing
|
|
// screen's when it ends; between those two mutations the DOM is quiet with both
|
|
// routes live. Returning there hands the next step a tree naming the screen the
|
|
// app is leaving, which on the folio wasm build recorded a submit that had
|
|
// landed on Home as still being on the transaction screen: the route gate of an
|
|
// action-gated property then skipped the very step the action landed on.
|
|
//
|
|
// The page keeps mutating for 300 ms after the route splice, and the settle
|
|
// runs on the timeout production hands it (MinIdleTimeout). Both details are
|
|
// load-bearing. A quiet page reaches the transition check immediately, so it
|
|
// passes whether the transition window is anchored at the script start or at
|
|
// the end of the quiet period; churn is what pushes the check past a
|
|
// start-anchored deadline, which then finishes at once with two live screens.
|
|
// And a caller timeout below MinIdleTimeout cuts the whole settle off before
|
|
// the transition window can be spent, which is the same bug from the other end.
|
|
func TestWaitForIdle_WaitsOutARouteTransition(t *testing.T) {
|
|
const page = `<body><div id="app"></div><script>
|
|
const root = document.getElementById("app").attachShadow({mode: "open"});
|
|
root.innerHTML = '<button id="go" style="width:200px;height:80px">go</button>' +
|
|
'<div id="LedgerScreen">ledger</div><div id="spinner">0</div>';
|
|
root.getElementById("go").addEventListener("click", function () {
|
|
const incoming = document.createElement("div");
|
|
incoming.id = "HomeScreen";
|
|
incoming.textContent = "home";
|
|
root.appendChild(incoming);
|
|
let frame = 0;
|
|
const churn = setInterval(function () {
|
|
root.getElementById("spinner").textContent = String(++frame);
|
|
}, 30);
|
|
setTimeout(function () { clearInterval(churn); }, 300);
|
|
setTimeout(function () { root.getElementById("LedgerScreen").remove(); }, 1000);
|
|
});
|
|
</script></body>`
|
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
|
w.Header().Set("Content-Type", "text/html")
|
|
_, _ = w.Write([]byte(page))
|
|
}))
|
|
defer server.Close()
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, server.URL, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
if err := d.Tap(ctx, 40, 40); err != nil {
|
|
t.Fatalf("Tap: %v", err)
|
|
}
|
|
if err := d.WaitForIdle(ctx, d.MinIdleTimeout()); err != nil {
|
|
t.Fatalf("WaitForIdle: %v", err)
|
|
}
|
|
|
|
var live []string
|
|
script := `Array.from(document.getElementById("app").shadowRoot
|
|
.querySelectorAll('[id$="Screen"]')).map(e => e.id)`
|
|
if err := chromedp.Run(d.tabCtx, chromedp.Evaluate(script, &live)); err != nil {
|
|
t.Fatalf("read: %v", err)
|
|
}
|
|
if len(live) != 1 || live[0] != "HomeScreen" {
|
|
t.Errorf("live screens after the settle = %v, want [HomeScreen]; WaitForIdle "+
|
|
"returned mid-transition, so the next step verifies the outgoing route", live)
|
|
}
|
|
}
|
|
|
|
// TestWaitForIdle_BoundsTheTransitionWait is the other half of the transition
|
|
// wait: a page that shows two *Screen ids at rest is not mid-transition, it
|
|
// just matches the heuristic, and it must cost one bounded wait rather than the
|
|
// whole step budget on every step.
|
|
func TestWaitForIdle_BoundsTheTransitionWait(t *testing.T) {
|
|
const page = `<body><div id="HomeScreen">home</div><div id="LedgerScreen">ledger</div></body>`
|
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
|
w.Header().Set("Content-Type", "text/html")
|
|
_, _ = w.Write([]byte(page))
|
|
}))
|
|
defer server.Close()
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, server.URL, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
start := time.Now()
|
|
if err := d.WaitForIdle(ctx, 10*time.Second); err != nil {
|
|
t.Fatalf("WaitForIdle: %v", err)
|
|
}
|
|
if elapsed := time.Since(start); elapsed > transitionSettlePeriod+time.Second {
|
|
t.Errorf("WaitForIdle took %s on a page with two resting screens; the "+
|
|
"transition wait must be bounded by %s", elapsed, transitionSettlePeriod)
|
|
}
|
|
}
|
|
|
|
// TestEvaluateExtractors_WaitsOutARouteTransition covers the other sampler. A
|
|
// step reads the page twice: the hierarchy dump (which re-fetches while the
|
|
// tree looks transitional) and the spec's own extractors in V8. Sampling the
|
|
// extractors mid cross-fade reports the route the app is leaving, and an
|
|
// action-gated property then skips the one step its action can be judged on:
|
|
// on the folio wasm build a double-submit that landed on Home was extracted as
|
|
// still being on the transaction screen.
|
|
func TestEvaluateExtractors_WaitsOutARouteTransition(t *testing.T) {
|
|
const page = `<body><div id="app"></div><script>
|
|
const root = document.getElementById("app").attachShadow({mode: "open"});
|
|
root.innerHTML = '<div id="LedgerScreen">ledger</div>';
|
|
window.__sanderlingExtractors__ = function () {
|
|
return {0: {value: Array.from(root.querySelectorAll('[id$="Screen"]')).map(e => e.id).join(",")}};
|
|
};
|
|
window.startTransition = function () {
|
|
const incoming = document.createElement("div");
|
|
incoming.id = "HomeScreen";
|
|
root.appendChild(incoming);
|
|
setTimeout(function () { root.getElementById("LedgerScreen").remove(); }, 400);
|
|
};
|
|
</script></body>`
|
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
|
w.Header().Set("Content-Type", "text/html")
|
|
_, _ = w.Write([]byte(page))
|
|
}))
|
|
defer server.Close()
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, server.URL, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
if err := chromedp.Run(d.tabCtx, chromedp.Evaluate(`window.startTransition()`, nil)); err != nil {
|
|
t.Fatalf("start transition: %v", err)
|
|
}
|
|
|
|
values, err := d.EvaluateExtractors(ctx)
|
|
if err != nil {
|
|
t.Fatalf("EvaluateExtractors: %v", err)
|
|
}
|
|
if got := string(values[0]); got != `"HomeScreen"` {
|
|
t.Errorf("extractor read %s, want \"HomeScreen\"; the extractors sampled "+
|
|
"mid-transition, so the spec sees the route the app is leaving", got)
|
|
}
|
|
}
|
|
|
|
// TestSetLastAction_ReportsAPageThatCannotTakeIt covers the install the whole
|
|
// web path's action-gated properties hang off. A page without the setter is
|
|
// reachable: internal/testrun resolves the web runtime from
|
|
// node_modules/@sanderling/spec when no sibling checkout is present, and an
|
|
// older published runtime does not define it. Guarded as
|
|
// `setter && setter(...)`, that page returns undefined and chromedp reports
|
|
// success, so every step silently no-ops and every property gated on the last
|
|
// action goes vacuously true - a green run that checked nothing.
|
|
func TestSetLastAction_ReportsAPageThatCannotTakeIt(t *testing.T) {
|
|
const withSetter = `<body><script>
|
|
window.__lastActionSeen = null;
|
|
window.__sanderlingSetLastAction__ = function (value) { window.__lastActionSeen = value; };
|
|
</script></body>`
|
|
const withoutSetter = `<body><div id="app">no sanderling runtime here</div></body>`
|
|
pages := map[string]string{"/with": withSetter, "/without": withoutSetter}
|
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
w.Header().Set("Content-Type", "text/html")
|
|
_, _ = w.Write([]byte(pages[r.URL.Path]))
|
|
}))
|
|
defer server.Close()
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
|
|
if err := d.Launch(ctx, server.URL+"/with", false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
action := json.RawMessage(`{"kind":"Tap","on":"id:TxnSubmit"}`)
|
|
if err := d.SetLastAction(ctx, action); err != nil {
|
|
t.Fatalf("SetLastAction on a page that defines the setter: %v", err)
|
|
}
|
|
var seen map[string]string
|
|
if err := chromedp.Run(d.tabCtx,
|
|
chromedp.Evaluate(`window.__lastActionSeen`, &seen)); err != nil {
|
|
t.Fatalf("read installed action: %v", err)
|
|
}
|
|
if seen["on"] != "id:TxnSubmit" {
|
|
t.Errorf("the page received %v, want the action the runner applied", seen)
|
|
}
|
|
|
|
if err := d.Launch(ctx, server.URL+"/without", false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
if err := d.SetLastAction(ctx, action); err == nil {
|
|
t.Error("SetLastAction reported success on a page with no setter; " +
|
|
"a runtime that cannot take lastAction is indistinguishable from one that did")
|
|
}
|
|
}
|
|
|
|
// TestEvaluateExtractors_ReportsAMissingTable is the same failure on the other
|
|
// sampler. An empty override map is what a spec with no extractors returns, so
|
|
// treating a missing table as {} makes "this page has no sanderling runtime"
|
|
// read as an ordinary step - and the verifier then judges the run on goja's
|
|
// dump-derived values while believing they came from the page.
|
|
func TestEvaluateExtractors_ReportsAMissingTable(t *testing.T) {
|
|
const page = `<body><div id="app">no sanderling runtime here</div></body>`
|
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
|
w.Header().Set("Content-Type", "text/html")
|
|
_, _ = w.Write([]byte(page))
|
|
}))
|
|
defer server.Close()
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, server.URL, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
values, err := d.EvaluateExtractors(ctx)
|
|
if err == nil {
|
|
t.Errorf("EvaluateExtractors returned %v and no error on a page with no "+
|
|
"extractor table; a page that cannot be read must not read as empty", values)
|
|
}
|
|
}
|
|
|
|
// TestEvaluateExtractors_KeepsUndefinedApartFromNull covers the wire the page's
|
|
// readings cross. JSON has no undefined, so the web runtime wraps each reading
|
|
// in a {value} envelope: written straight into the table, an extractor that
|
|
// returned undefined lost its whole index to JSON.stringify and the host kept
|
|
// goja's dump-derived reading for it while the rest held the page's. An absent
|
|
// value has to arrive as an empty payload, which is what makes the verifier
|
|
// record undefined (the value the native host records for the same getter);
|
|
// arriving as JSON null would claim the getter returned null.
|
|
func TestEvaluateExtractors_KeepsUndefinedApartFromNull(t *testing.T) {
|
|
const page = `<body><script>
|
|
window.__sanderlingExtractors__ = function () {
|
|
return {0: {}, 1: {value: null}, 2: {value: {balance: 7}}};
|
|
};
|
|
</script></body>`
|
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
|
w.Header().Set("Content-Type", "text/html")
|
|
_, _ = w.Write([]byte(page))
|
|
}))
|
|
defer server.Close()
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, server.URL, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
|
|
values, err := d.EvaluateExtractors(ctx)
|
|
if err != nil {
|
|
t.Fatalf("EvaluateExtractors: %v", err)
|
|
}
|
|
if len(values) != 3 {
|
|
t.Fatalf("the page reported 3 readings, %d survived the wire: %v", len(values), values)
|
|
}
|
|
if got := values[0]; len(got) != 0 {
|
|
t.Errorf("the undefined reading arrived as %s, want an empty payload; "+
|
|
"the verifier records anything else as a value the getter never returned", got)
|
|
}
|
|
if got := string(values[1]); got != "null" {
|
|
t.Errorf("the null reading arrived as %s, want null", got)
|
|
}
|
|
if got := string(values[2]); got != `{"balance":7}` {
|
|
t.Errorf("the object reading arrived as %s, want {\"balance\":7}", got)
|
|
}
|
|
}
|
|
|
|
// TestEvaluateExtractors_RejectsAnUnenvelopedReading is the loud failure a page
|
|
// running an older @sanderling/spec produces. Its readings are bare values, and
|
|
// a bare value is indistinguishable from a reading whose getter returned that
|
|
// value, so accepting them silently puts the two engines on different bundles.
|
|
func TestEvaluateExtractors_RejectsAnUnenvelopedReading(t *testing.T) {
|
|
const page = `<body><script>
|
|
window.__sanderlingExtractors__ = function () { return {0: "home"}; };
|
|
</script></body>`
|
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
|
w.Header().Set("Content-Type", "text/html")
|
|
_, _ = w.Write([]byte(page))
|
|
}))
|
|
defer server.Close()
|
|
|
|
d := New()
|
|
defer d.Terminate(context.Background())
|
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
defer cancel()
|
|
if err := d.Launch(ctx, server.URL, false, nil); err != nil {
|
|
t.Fatalf("Launch: %v", err)
|
|
}
|
|
|
|
values, err := d.EvaluateExtractors(ctx)
|
|
if err == nil {
|
|
t.Fatalf("EvaluateExtractors accepted %v from a page whose readings are not "+
|
|
"enveloped; the page and the host are running different bundles", values)
|
|
}
|
|
if !strings.Contains(err.Error(), "different bundles") {
|
|
t.Errorf("EvaluateExtractors failed with %q, want it to name the bundle mismatch", err)
|
|
}
|
|
}
|