mirror of
https://github.com/priyanshujain/sanderling.git
synced 2026-10-02 11:07:10 +00:00
WIP: folio refactor - KotlinConf-style production-app shape (#47)
* feat(hierarchy): testTag alias resolves to resource-id and accessibilityIdentifier
Compose's testTag surfaces as resource-id on Android and as
accessibilityIdentifier on iOS. Selectors written as
{ testTag: "Foo" } now match either, so Sanderling specs can use the
same tag on both platforms.
Also rounds out the iOS identifier aliases so resource-id /
identifier / accessibilityIdentifier all resolve to one another.
* chore(folio): add gradle/libs.versions.toml
Centralises versions for all folio modules ahead of the module split.
Adds new entries for kotlinx-serialization, navigation3, Metro, KSP,
and the JetBrains lifecycle-viewmodel-compose multiplatform artifact.
* refactor(folio): introduce nested KotlinConf-style modules
Split the monolithic :composeApp into :core, :app:shared,
:app:ui-components, and :app:androidApp. The old module is still
present and remains the source of truth until the next commits remove
it; both compile in parallel to keep iOS/Android builds green during
the cut-over.
Highlights:
- :core - SQLDelight schema + LedgerStore + Repository (now an
injectable class, not a singleton object). Methods are suspend to
match generateAsync = true.
- :app:ui-components - design system primitives. IconButton/AppButton
APIs revised: label = real contentDescription, testTag = stable
selector. Drops the data-carrier description argument.
- :app:shared - per-screen ViewModels colocated with screens; pure
composables on (state, onEvent); LocalAppComponent CompositionLocal
for hand-rolled DI; @Serializable Route. Hosts the iOS framework
(baseName Shared).
- :app:androidApp - thin Android entry that constructs the
DriverFactory and hands it to App().
- gradle/libs.versions.toml centralises versions; settings.gradle.kts
enables type-safe project accessors.
Deferred to follow-up PRs (per the design discussion):
- Metro DI: hand-rolled AppComponent for now; Metro graphs are mostly
ceremony for an app this size and add KSP/version risk.
- Navigation3: kept the existing Navigator-as-backstack class,
injected rather than singleton; nav3 isn't shipping a stable
multiplatform artifact for commonMain consumption yet.
- :app:webApp + OPFS sqlite worker: web persistence is real new
wiring (custom worker on @sqlite.org/sqlite-wasm). Master's
WebLedgerStore + Snapshot is being removed by this PR; web stays
buildable as a klib but no app-level wasm binary lands here.
* refactor(folio): delete :composeApp and retarget tooling
Removes the old monolithic module now that :core / :app:shared /
:app:ui-components / :app:androidApp own the source. Updates:
- justfile install/uninstall recipes -> :app:androidApp
- iosApp/project.yml framework path -> ../app/shared/...,
baseName Shared (was ComposeApp); pre-build script invokes
:app:shared:linkDebugFrameworkIosSimulatorArm64
- iosApp/iosApp/iOSApp.swift -> import Shared
- README -> mentions SQLDelight unification, drops the
data-carrier contentDescription notes (now stale), no Layout
section per repo convention
* refactor(folio-spec): query testTag and identify items by visible text
Replaces every accessibilityText / descPrefix data-carrier read with
testTag selectors that resolve to resource-id (Android) or
accessibilityIdentifier (iOS) via the SDK's alias table.
- Routes detected via testTag (LoginScreen, HomeScreen, etc.)
- Account identity = visible account name (no synthetic id encoded
in semantics).
- Ledger row identity = joined text content of the row.
- Active account derived from route alone (not parsed from
contentDescription).
- Focused input read from native focused="true" attribute, not from
a custom focused_input data carrier.
* fix(folio): build green on Android assemble + iOS framework link
- Drop ksp/metro/navigation3 plugin aliases - not actually applied
by any module in this PR (deferred follow-up).
- import awaitAsOne from app.cash.sqldelight.async.coroutines for
the suspend single-row reads enabled by generateAsync = true.
- Drop kotlin.js.ExperimentalWasmJsInterop opt-in from common
compilerOptions (it isn't valid for android/jvm targets).
- :app:shared androidMain pulls in androidx.activity:activity-compose
for the BackHandler actual.
* fix(folio): testTagsAsResourceId at App root + JS-bridge regression test
App.kt sets testTagsAsResourceId=true on the root Box semantics so
Compose's testTag surfaces as Android resource-id (and equivalent on
iOS via accessibilityIdentifier). Without this, testTag stays in the
Compose semantics tree but never reaches the runtime hierarchy that
UIAutomator and Sanderling read.
Also adds TestStateAxObjectSelectorTestTagAlias as a regression
test for the {testTag: ...} object selector resolving through the
SDK alias to resource-id at the JS bridge layer.
* test(verifier): expose PredicateError latching across steps
The runner logs PredicateError once per step. The current implementation
latches the first error per thunk, so the log freezes on step 1 forever
even when later steps would observe different errors. This test fails
today and locks in the contract: PredicateError must reflect the most
recent step.
* fix(verifier): refresh predicate errors per step
EvaluateProperties short-circuits once an Always-property latches to
violated, so the underlying goja predicate stops being called and
formula.err keeps whatever it threw at step 1. The runner logs
PredicateError every step a property is violated, which made every
subsequent log line repeat the step-1 throw. That looks like the spec
runtime is seeing stale state, but it is just stale error reporting.
EvaluateProperties now invokes every registered predicate once per step
purely to refresh formula.err. Verdicts are unaffected. The thunk
itself stops latching so the new value wins on whichever path runs first.
* chore(folio): add Metro DI plugin (1.0.0-RC4) to versions catalog
Adds dev.zacsweers.metro plugin alias and applies it to :core
as a smoke test. Compiler-plugin only, no KSP required.
* chore(folio): apply Metro plugin to :app:shared and :app:androidApp
* feat(folio-core): annotate Repository and SqlLedgerStore with @Inject
* feat(folio-core): scope Repository and SqlLedgerStore as @SingleIn(AppScope)
Both are app-wide singletons so the SqlDelight-backed flows remain
shared across the graph.
* feat(folio): annotate ViewModels with Metro @Inject / @AssistedInject
LedgerViewModel and AddTransactionViewModel use @AssistedInject for
their accountId param plus a nested @AssistedFactory; the rest are
plain @Inject constructor classes.
* feat(folio): introduce Metro AppGraph in commonMain
Single shared @DependencyGraph(AppScope::class) that exposes
Repository, Navigator, and ViewModels. LedgerDatabase enters the
graph via @DependencyGraph.Factory.create(database) so the suspend
DriverFactory.create() can stay outside the DI surface.
@Binds wires SqlLedgerStore to LedgerStore; Navigator is provided
explicitly so its Route.Home start state stays in DI rather than
relying on a default-parameter being honored by the graph.
* fix(folio): expect/actual testTagsAsResourceId so iOS link succeeds
Compose's androidx.compose.ui.semantics.testTagsAsResourceId is
Android-only. Calling it directly from commonMain broke
linkDebugFrameworkIosSimulatorArm64. Replace with an expect Modifier
extension that wires the semantics on Android and is a no-op on
iOS / wasmJs.
* refactor(folio): replace AppComponent with Metro AppGraph in App.kt
App now takes a suspend graph builder; the platform constructs
LedgerDatabase off the suspend DriverFactory.create() before invoking
the Metro graph factory. Routes resolve VMs through LocalAppGraph
instead of the hand-rolled LocalAppComponent.
Drops the loading-state placeholder comment (the empty Box is enough).
* refactor(folio): resolve ViewModels through LocalAppGraph in routes
Each *Route composable now reads the AppGraph from CompositionLocal
and pulls its VM via the appropriate accessor or AssistedFactory.
* refactor(folio): build AppGraph from platform entry points
MainActivity (Android) and MainViewController (iOS) now own the
suspend DriverFactory.create() and feed the resulting LedgerDatabase
into Metro's createGraphFactory<AppGraph.Factory>().
* chore(folio): add navigation-compose 2.9.2 dependency
Adds the JetBrains KMP navigation-compose library to the shared
module. Used in subsequent commits to replace the hand-rolled
Navigator with a typed-route NavHost.
* refactor(folio): replace custom Navigator with NavHost backstack
Wraps androidx.navigation.NavHostController behind the existing
push/replace/back surface so call sites in ViewModels stay unchanged.
App.kt now wires a typed NavHost with @Serializable Route entries
and observes the controller's currentBackStackEntry to drive the
session-based Login/Home redirect.
* fix(folio-core): wire kotlinx-browser so wasmJs DriverFactory compiles
org.w3c.dom.Worker on wasmJs lives in kotlinx-browser, not the stdlib.
Pin 0.5.0 alongside the @sqlite.org/sqlite-wasm 3.53.0-build1 version
that the upcoming web app will depend on, and switch the worker
constructor to the module-worker form that webpack expects.
* feat(folio): scaffold :app:webApp wasmJs module
Compose Multiplatform target that depends on :app:shared and pulls
@sqlite.org/sqlite-wasm 3.53.0-build1 as the npm runtime for the
SQLDelight web worker.
* feat(folio-webApp): add main entrypoint and index.html
main.kt mirrors the iOS entry point: builds DriverFactory + AppGraph
factory, hooks browser back-gesture into WebBackGesture, then mounts
the shared App composable into ComposeViewport.
* feat(folio-webApp): OPFS-backed sqlite worker + webpack config
sqlite.worker.js implements the SQLDelight web-worker protocol
(exec/begin_transaction/end_transaction/rollback_transaction) on top
of @sqlite.org/sqlite-wasm. Prefers the OPFS SAH pool VFS for
persistent storage and falls back to in-memory when OPFS is
unavailable.
webpack.config.d/coopcoep.js sends COOP/COEP headers on the dev
server so cross-origin isolation is available, even though the SAH
pool itself does not require it. webpack.config.d/sqlite-wasm.js
enables asyncWebAssembly so webpack can bundle sqlite3.wasm via the
'new URL("sqlite3.wasm", import.meta.url)' reference inside the
sqlite-wasm package.
* chore(folio): add web/web-build just recipes and refresh yarn lock
Yarn lock picks up @sqlite.org/sqlite-wasm 3.53.0-build1.
* fix(folio-core): probe schema before create on wasmJs
Wasm SqlDriver doesn't auto-track user_version like the Android
driver, so awaitCreate() ran on every page load and tripped over
already-created tables. Read PRAGMA user_version, run
awaitCreate/awaitMigrate based on it, and self-heal pre-existing
tables with version 0 by stamping the current schema version.
* chore(folio-webApp): pin dev-server port and trim worker logging
webpack-dev-server now binds 8088 (or WEBAPP_PORT) so it doesn't
collide with the docs server on 8080. Drop the per-message reply
log; keep only the OPFS init line and error logging.
* chore(folio): nest iosApp under app/ for KotlinConf parity
Match KotlinConf-app's filesystem layout where every entry point (android,
ios, web, shared, ui-components) lives under app/. iosApp is still an Xcode
project, not a Gradle module, so settings.gradle.kts is unchanged.
* feat(folio): testTag identity for AccountName and ledger row cells
Replaces string-heuristic identity in the spec extractors with stable
testTags. AccountCard exposes AccountName; LedgerRow exposes TxnNote
and TxnDate. Spec extractors read those directly instead of filtering
visible text by "starts with $" / "matches digit".
* fix(folio-app): branch start destination on initial session
Read repository.session.value at first composition and pick
Route.Home or Route.Login as the NavHost startDestination. Avoids
the one-frame Home flash on cold start with no persisted session.
* refactor(folio-webApp): hard-fail when OPFS unavailable
Drops the silent in-memory fallback. The README claims OPFS
persistence; falling back without surfacing the degrade made data
loss invisible across reloads. Now the worker errors out and the
Kotlin DriverFactory rejects the create() call instead.
* docs(verifier): document extractor advancement and refresh invariants
Extractor previous/current advance only on PushSnapshot, never per
thunk-call. refreshPredicateErrors depends on this for safe re-entry.
Also flags that re-invoked predicates run outside their LTL gate, so
they must be side-effect-free reads.
* fix(hierarchy): populate ResourceID from accessibilityIdentifier
iOS Compose surfaces testTag as accessibilityIdentifier. Previously
only resource-id and identifier seeded element.ResourceID, leaving
element.id empty for iOS Compose nodes and forcing specs to walk
attrs to recover stable identifiers.
* refactor(folio-spec): use element.id for focused field tag
Now that ResourceID populates uniformly across Android/iOS Compose,
the spec can read element.id directly instead of probing attrs for
each platform's underlying field name.
* fix(folio-spec): pick account card via seeded from(), not Math.random
Math.random() breaks --seed reproducibility. The verifier's seeded
RNG flows through from(), so re-running a seed now produces the
same card pick sequence.
* docs(spec): fix README example to use scoped extractors
The previous snippet referenced `state.ax.find` inside an actions()
body where state is not in scope, and shadowed the imported actions
helper with an export of the same name.
* feat(hierarchy): add FindBySelectorPath for chained object selectors
Each selector in the chain is matched within the descendants of the
previous match. Returns the deepest match (or nil) for FindBySelectorPath
and every deepest match for FindAllBySelectorPath.
* feat(verifier): dispatch JS array selectors to FindBySelectorPath
`state.ax.find([{...}, {...}])` now walks each segment scoped under
the previous match. Strings and single objects keep their existing
single-shot lookup.
* feat(spec): expose SelectorPath in find/findAll signatures
* fix(spec): satisfy AccessibilityElement interface in Tap test fixture
* refactor(folio-spec): collapse chained finds into selector paths
* feat(spec): add keyedBy(element, tags) identity helper
Joins element.find({testTag: tag})?.text per tag with U+001F as the
delimiter so user-visible text can never collide with the separator.
Returns empty string for an undefined element.
* refactor(folio-spec): use keyedBy for ledger row identity
* feat(spec): add whenRoute action gating helper
whenRoute(route, allowedRoutes, body) wraps an actions() generator
that returns [] unless route.current matches one of the allowed
values. Accepts a single route or an array.
* test(spec): cover whenRoute matching, gating, and array routes
* refactor(folio-spec): gate addAccount and addTxn with whenRoute
* refactor(hierarchy): use maps.Copy for attribute merge
Linter flagged the manual loop after recent edits surfaced the hint.
* feat(verifier): dispatch setup generator before actions root
Setup is consulted every step; when it returns ErrNoAction the call falls
through to the existing actionGenerator retry loop. This lets specs split
deterministic preconditions (login, onboarding) out of the weighted action
pool while auto-reengaging if state regresses (e.g. logout under fuzz).
* docs(spec): document setup precondition action generator
* refactor(folio-spec): export login as setup, remove from action pool
Login is deterministic and yields no actions once the app is past the
login screen; sitting at weight 50 in the action pool wasted half of step
picks on a no-op. Promote it to setup so the runner only consults it
while it has work to do, and rebalance remaining weights to round numbers
(addAccount 50, addTxn 40, back 10).
This commit is contained in:
111 files changed
+2330
-1071
No files matched your search
@@ -7,6 +7,58 @@ import (
|
||||
"github.com/priyanshujain/sanderling/internal/hierarchy"
|
||||
)
|
||||
|
||||
// TestStateAxObjectSelectorTestTagAlias verifies that an object selector
|
||||
// `{ testTag: "X" }` resolves through the testTag alias to match an element
|
||||
// whose source attributes carry resource-id="X" (the Compose
|
||||
// testTagsAsResourceId=true case on Android).
|
||||
func TestStateAxObjectSelectorTestTagAlias(t *testing.T) {
|
||||
src := `{
|
||||
"attributes": {"class": "android.widget.LinearLayout"},
|
||||
"children": [
|
||||
{
|
||||
"attributes": {"resource-id": "LoginScreen", "class": "android.view.View"},
|
||||
"children": [
|
||||
{
|
||||
"attributes": {"resource-id": "LoginEmail", "class": "android.widget.EditText"},
|
||||
"children": []
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}`
|
||||
tree, err := hierarchy.Parse(src)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
verifier := newVerifier(t)
|
||||
mustLoad(t, verifier, `
|
||||
globalThis.loginRoot = __sanderling__.extract(state => {
|
||||
const r = state.ax.find({ testTag: "LoginScreen" });
|
||||
return r ? "matched" : "miss";
|
||||
});
|
||||
globalThis.loginEmailViaChain = __sanderling__.extract(state => {
|
||||
const r = state.ax.find({ testTag: "LoginScreen" });
|
||||
if (!r) return "outer-miss";
|
||||
const inner = r.find({ testTag: "LoginEmail" });
|
||||
return inner ? "inner-matched" : "inner-miss";
|
||||
});
|
||||
`)
|
||||
|
||||
if err := verifier.PushSnapshot(SnapshotInput{Snapshots: Snapshots{}, Tree: tree}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
root := verifier.runtime.GlobalObject().Get("loginRoot").ToObject(verifier.runtime).Get("current").String()
|
||||
if root != "matched" {
|
||||
t.Fatalf("loginRoot = %q, want matched", root)
|
||||
}
|
||||
chain := verifier.runtime.GlobalObject().Get("loginEmailViaChain").ToObject(verifier.runtime).Get("current").String()
|
||||
if chain != "inner-matched" {
|
||||
t.Fatalf("loginEmailViaChain = %q, want inner-matched", chain)
|
||||
}
|
||||
}
|
||||
|
||||
// TestStateAxFindWorks verifies that a Parse+PushSnapshot+extract round trip
|
||||
// actually lets the spec resolve selectors through state.ax.find.
|
||||
// Reads /tmp/live-dump.json (Maestro TreeNode JSON format); skipped if absent.
|
||||
|
||||
@@ -14,9 +14,10 @@ type extractorState struct {
|
||||
|
||||
type formulaState struct {
|
||||
predicate goja.Callable
|
||||
// err latches the first goja error returned by predicate. The thunk
|
||||
// returns false on error so the LTL evaluator marks the property
|
||||
// violated; PredicateError surfaces the underlying cause.
|
||||
// err holds the goja error from this thunk's most recent invocation, or
|
||||
// nil if the latest call succeeded. The thunk returns false on error so
|
||||
// the LTL evaluator marks the property violated; PredicateError surfaces
|
||||
// the underlying cause for the current step.
|
||||
err error
|
||||
}
|
||||
|
||||
|
||||
@@ -136,7 +136,8 @@ func nodeObject(runtime *goja.Runtime, node *hierarchy.Node, selector string) go
|
||||
return object
|
||||
}
|
||||
|
||||
// findNodeFromJS dispatches a JS value (string or object) to Tree-level node lookup.
|
||||
// findNodeFromJS dispatches a JS value (string, object, or array of objects)
|
||||
// to Tree-level node lookup.
|
||||
func findNodeFromJS(runtime *goja.Runtime, tree *hierarchy.Tree, arg goja.Value) *hierarchy.Node {
|
||||
if goja.IsUndefined(arg) || goja.IsNull(arg) {
|
||||
return nil
|
||||
@@ -147,6 +148,9 @@ func findNodeFromJS(runtime *goja.Runtime, tree *hierarchy.Tree, arg goja.Value)
|
||||
if s, ok := arg.Export().(string); ok {
|
||||
return tree.FindNode(s)
|
||||
}
|
||||
if path, ok := selectorPathFromJS(runtime, arg); ok {
|
||||
return tree.FindBySelectorPath(path)
|
||||
}
|
||||
sel := selectorFromJSObject(runtime, arg)
|
||||
if len(sel.Filters) == 0 {
|
||||
return nil
|
||||
@@ -162,6 +166,9 @@ func findAllNodesFromJS(runtime *goja.Runtime, tree *hierarchy.Tree, arg goja.Va
|
||||
if s, ok := arg.Export().(string); ok {
|
||||
return tree.FindAllNodes(s)
|
||||
}
|
||||
if path, ok := selectorPathFromJS(runtime, arg); ok {
|
||||
return tree.FindAllBySelectorPath(path)
|
||||
}
|
||||
sel := selectorFromJSObject(runtime, arg)
|
||||
if len(sel.Filters) == 0 {
|
||||
return nil
|
||||
@@ -177,6 +184,9 @@ func findNodeInSubtreeFromJS(runtime *goja.Runtime, node *hierarchy.Node, arg go
|
||||
if s, ok := arg.Export().(string); ok {
|
||||
return node.Find(s)
|
||||
}
|
||||
if path, ok := selectorPathFromJS(runtime, arg); ok {
|
||||
return node.FindBySelectorPath(path)
|
||||
}
|
||||
sel := selectorFromJSObject(runtime, arg)
|
||||
if len(sel.Filters) == 0 {
|
||||
return nil
|
||||
@@ -192,6 +202,9 @@ func findAllNodesInSubtreeFromJS(runtime *goja.Runtime, node *hierarchy.Node, ar
|
||||
if s, ok := arg.Export().(string); ok {
|
||||
return node.FindAll(s)
|
||||
}
|
||||
if path, ok := selectorPathFromJS(runtime, arg); ok {
|
||||
return node.FindAllBySelectorPath(path)
|
||||
}
|
||||
sel := selectorFromJSObject(runtime, arg)
|
||||
if len(sel.Filters) == 0 {
|
||||
return nil
|
||||
@@ -219,6 +232,37 @@ func selectorFromJSObject(runtime *goja.Runtime, arg goja.Value) hierarchy.Selec
|
||||
return sel
|
||||
}
|
||||
|
||||
// selectorPathFromJS recognizes a JS array of selector objects and converts it
|
||||
// into a Selector chain. Returns ok=false for non-arrays so callers fall
|
||||
// through to single-object dispatch.
|
||||
func selectorPathFromJS(runtime *goja.Runtime, arg goja.Value) ([]hierarchy.Selector, bool) {
|
||||
exported := arg.Export()
|
||||
slice, ok := exported.([]any)
|
||||
if !ok {
|
||||
return nil, false
|
||||
}
|
||||
obj := arg.ToObject(runtime)
|
||||
if obj == nil {
|
||||
return nil, false
|
||||
}
|
||||
path := make([]hierarchy.Selector, 0, len(slice))
|
||||
for index := range slice {
|
||||
entry := obj.Get(fmt.Sprintf("%d", index))
|
||||
if entry == nil || goja.IsUndefined(entry) || goja.IsNull(entry) {
|
||||
return nil, false
|
||||
}
|
||||
sel := selectorFromJSObject(runtime, entry)
|
||||
if len(sel.Filters) == 0 {
|
||||
return nil, false
|
||||
}
|
||||
path = append(path, sel)
|
||||
}
|
||||
if len(path) == 0 {
|
||||
return nil, false
|
||||
}
|
||||
return path, true
|
||||
}
|
||||
|
||||
// selectorStringFromJS returns a string representation of the selector argument
|
||||
// for tagging returned element objects (used by selectorOf to reconstruct the
|
||||
// selector when the element is passed back as an action target).
|
||||
|
||||
@@ -7,6 +7,9 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/dop251/goja"
|
||||
|
||||
"github.com/priyanshujain/sanderling/internal/hierarchy"
|
||||
"github.com/priyanshujain/sanderling/internal/ltl"
|
||||
)
|
||||
|
||||
@@ -176,6 +179,108 @@ func TestNextAction_EmptyGeneratorReturnsErrNoAction(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestNextAction_SetupTakesPrecedenceWhenYielding(t *testing.T) {
|
||||
verifier := newVerifier(t)
|
||||
mustLoad(t, verifier, `
|
||||
globalThis.setup = __sanderling__.actions(() => [__sanderling__.tap({ on: "id:setup" })]);
|
||||
globalThis.actions = __sanderling__.actions(() => [__sanderling__.tap({ on: "id:main" })]);
|
||||
`)
|
||||
_ = verifier.PushSnapshot(SnapshotInput{Snapshots: Snapshots{}})
|
||||
|
||||
action, err := verifier.NextAction()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if action.On != "id:setup" {
|
||||
t.Errorf("setup precedence: got %q, want id:setup", action.On)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNextAction_FallsThroughToActionsWhenSetupEmpty(t *testing.T) {
|
||||
verifier := newVerifier(t)
|
||||
mustLoad(t, verifier, `
|
||||
globalThis.setup = __sanderling__.actions(() => []);
|
||||
globalThis.actions = __sanderling__.actions(() => [__sanderling__.tap({ on: "id:main" })]);
|
||||
`)
|
||||
_ = verifier.PushSnapshot(SnapshotInput{Snapshots: Snapshots{}})
|
||||
|
||||
action, err := verifier.NextAction()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if action.On != "id:main" {
|
||||
t.Errorf("fallthrough: got %q, want id:main", action.On)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNextAction_SetupReengagesAfterRegression(t *testing.T) {
|
||||
verifier := newVerifier(t)
|
||||
mustLoad(t, verifier, `
|
||||
globalThis.loggedIn = __sanderling__.extract(state => state.snapshots["loggedIn"] === true);
|
||||
globalThis.setup = __sanderling__.actions(() => {
|
||||
if (loggedIn.current) return [];
|
||||
return [__sanderling__.tap({ on: "id:login" })];
|
||||
});
|
||||
globalThis.actions = __sanderling__.actions(() => [__sanderling__.tap({ on: "id:main" })]);
|
||||
`)
|
||||
|
||||
push := func(loggedIn bool) {
|
||||
raw := json.RawMessage(`false`)
|
||||
if loggedIn {
|
||||
raw = json.RawMessage(`true`)
|
||||
}
|
||||
if err := verifier.PushSnapshot(SnapshotInput{Snapshots: Snapshots{"loggedIn": raw}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
push(false)
|
||||
action, err := verifier.NextAction()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if action.On != "id:login" {
|
||||
t.Fatalf("step 1 (logged out): got %q, want id:login", action.On)
|
||||
}
|
||||
|
||||
push(true)
|
||||
action, err = verifier.NextAction()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if action.On != "id:main" {
|
||||
t.Fatalf("step 2 (logged in): got %q, want id:main", action.On)
|
||||
}
|
||||
|
||||
push(false)
|
||||
action, err = verifier.NextAction()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if action.On != "id:login" {
|
||||
t.Fatalf("step 3 (regressed): got %q, want id:login", action.On)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNextAction_NoSetupRegistered(t *testing.T) {
|
||||
verifier := newVerifier(t)
|
||||
mustLoad(t, verifier, `
|
||||
globalThis.actions = __sanderling__.actions(() => [__sanderling__.tap({ on: "id:main" })]);
|
||||
`)
|
||||
_ = verifier.PushSnapshot(SnapshotInput{Snapshots: Snapshots{}})
|
||||
|
||||
if verifier.setupGenerator != nil {
|
||||
t.Errorf("setupGenerator should be nil when spec does not export setup")
|
||||
}
|
||||
action, err := verifier.NextAction()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if action.On != "id:main" {
|
||||
t.Errorf("got %q, want id:main", action.On)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInputText_RoundTrip(t *testing.T) {
|
||||
verifier := newVerifier(t)
|
||||
mustLoad(t, verifier, `
|
||||
@@ -258,3 +363,142 @@ func TestLoad_AcceptsSpecWithoutPropertiesOrActions(t *testing.T) {
|
||||
t.Errorf("expected ErrNoAction, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestSelectorPath_ScopedDescent ensures the JS-side `find([{...}, {...}])`
|
||||
// shape walks each segment scoped under the previous match.
|
||||
func TestSelectorPath_ScopedDescent(t *testing.T) {
|
||||
const treeJSON = `{
|
||||
"attributes": {"resource-id": "rootView", "bounds": "[0,0,1080,2340]"},
|
||||
"children": [
|
||||
{
|
||||
"attributes": {"testTag": "HomeScreen", "bounds": "[0,0,540,2340]"},
|
||||
"children": [
|
||||
{
|
||||
"attributes": {"testTag": "AccountCard", "bounds": "[0,0,540,200]"},
|
||||
"children": [
|
||||
{"attributes": {"testTag": "AccountName", "text": "Checking", "bounds": "[10,10,200,40]"}, "children": []}
|
||||
]
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"attributes": {"testTag": "LedgerScreen", "bounds": "[540,0,1080,2340]"},
|
||||
"children": [
|
||||
{"attributes": {"testTag": "AccountName", "text": "Other", "bounds": "[600,10,800,40]"}, "children": []}
|
||||
]
|
||||
}
|
||||
]
|
||||
}`
|
||||
verifier := newVerifier(t)
|
||||
mustLoad(t, verifier, `
|
||||
globalThis.found = __sanderling__.extract(state =>
|
||||
state.ax.find([{ testTag: "HomeScreen" }, { testTag: "AccountCard" }, { testTag: "AccountName" }])
|
||||
);
|
||||
globalThis.foundUnreachable = __sanderling__.extract(state =>
|
||||
state.ax.find([{ testTag: "LedgerScreen" }, { testTag: "AccountCard" }])
|
||||
);
|
||||
globalThis.allInHome = __sanderling__.extract(state =>
|
||||
state.ax.findAll([{ testTag: "HomeScreen" }, { testTag: "AccountName" }])
|
||||
);
|
||||
`)
|
||||
tree, err := hierarchy.Parse(treeJSON)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := verifier.PushSnapshot(SnapshotInput{Snapshots: Snapshots{}, Tree: tree}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
found := verifier.runtime.GlobalObject().Get("found").ToObject(verifier.runtime).Get("current")
|
||||
if found == nil || goja.IsUndefined(found) {
|
||||
t.Fatal("expected path lookup to find AccountName under HomeScreen > AccountCard")
|
||||
}
|
||||
text := found.ToObject(verifier.runtime).Get("text")
|
||||
if text.String() != "Checking" {
|
||||
t.Fatalf("text = %q, want Checking", text.String())
|
||||
}
|
||||
unreachable := verifier.runtime.GlobalObject().Get("foundUnreachable").ToObject(verifier.runtime).Get("current")
|
||||
if !goja.IsUndefined(unreachable) {
|
||||
t.Fatalf("AccountCard is not under LedgerScreen, expected undefined, got %v", unreachable)
|
||||
}
|
||||
allInHome := verifier.runtime.GlobalObject().Get("allInHome").ToObject(verifier.runtime).Get("current")
|
||||
allObject := allInHome.ToObject(verifier.runtime)
|
||||
length := allObject.Get("length").ToInteger()
|
||||
if length != 1 {
|
||||
t.Fatalf("findAll path length = %d, want 1 (Checking only, not Other in LedgerScreen)", length)
|
||||
}
|
||||
}
|
||||
|
||||
// TestFrom_SeededReplayIsDeterministic guarantees `from()` over a per-step
|
||||
// dynamic array picks the same element under the same seed across runs. The
|
||||
// folio spec relies on this to replace Math.random() in account-card taps.
|
||||
func TestFrom_SeededReplayIsDeterministic(t *testing.T) {
|
||||
pickedSequence := func(seed uint64) []string {
|
||||
verifier := newVerifier(t, WithRand(rand.New(rand.NewPCG(seed, 0))))
|
||||
mustLoad(t, verifier, `
|
||||
globalThis.actions = __sanderling__.actions(() => {
|
||||
const cards = ["card_a", "card_b", "card_c", "card_d"];
|
||||
return [__sanderling__.tap({ on: __sanderling__.from(cards).generate() })];
|
||||
});
|
||||
`)
|
||||
_ = verifier.PushSnapshot(SnapshotInput{Snapshots: Snapshots{}})
|
||||
var picks []string
|
||||
for range 20 {
|
||||
action, err := verifier.NextAction()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
picks = append(picks, action.On)
|
||||
}
|
||||
return picks
|
||||
}
|
||||
first := pickedSequence(1234)
|
||||
second := pickedSequence(1234)
|
||||
for i := range first {
|
||||
if first[i] != second[i] {
|
||||
t.Fatalf("step %d: %q != %q (replay not deterministic)", i, first[i], second[i])
|
||||
}
|
||||
}
|
||||
other := pickedSequence(5678)
|
||||
identical := true
|
||||
for i := range first {
|
||||
if first[i] != other[i] {
|
||||
identical = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if identical {
|
||||
t.Fatal("expected different seeds to produce different pick sequences")
|
||||
}
|
||||
}
|
||||
|
||||
// PredicateError must reflect the most recent step's predicate result, not a
|
||||
// latched first-step error. The runner logs PredicateError once per step; if it
|
||||
// stays pinned to step 1 forever, downstream debugging looks frozen even though
|
||||
// the underlying state is changing.
|
||||
func TestPredicateError_ReflectsCurrentStepNotFirstStep(t *testing.T) {
|
||||
const spec = `
|
||||
globalThis.counter = __sanderling__.extract(state => state.snapshots["count"]);
|
||||
globalThis.properties = {
|
||||
reportsCounter: __sanderling__.always(() => { throw new Error("count=" + counter.current); }),
|
||||
};
|
||||
`
|
||||
verifier := newVerifier(t)
|
||||
mustLoad(t, verifier, spec)
|
||||
|
||||
for step := 1; step <= 3; step++ {
|
||||
raw := json.RawMessage([]byte{'"', byte('0' + step), '"'})
|
||||
if err := verifier.PushSnapshot(SnapshotInput{Snapshots: Snapshots{"count": raw}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = verifier.EvaluateProperties()
|
||||
|
||||
got := verifier.PredicateError("reportsCounter")
|
||||
if got == nil {
|
||||
t.Fatalf("step %d: PredicateError = nil, want non-nil", step)
|
||||
}
|
||||
want := "count=" + string(rune('0'+step))
|
||||
if !strings.Contains(got.Error(), want) {
|
||||
t.Errorf("step %d: PredicateError = %q, want to contain %q", step, got.Error(), want)
|
||||
}
|
||||
}
|
||||
}
|
||||
+54
-10
@@ -20,6 +20,7 @@ type Verifier struct {
|
||||
|
||||
properties map[string]int // property name -> formula-spec index
|
||||
actionGenerator goja.Value
|
||||
setupGenerator goja.Value
|
||||
|
||||
evaluators map[string]*ltl.Evaluator
|
||||
|
||||
@@ -56,8 +57,9 @@ func New(options ...Option) (*Verifier, error) {
|
||||
}
|
||||
|
||||
// Load executes the bundled spec source. The spec is expected to assign its
|
||||
// property formulas to globalThis.properties and its root action generator
|
||||
// to globalThis.actions.
|
||||
// property formulas to globalThis.properties, its root action generator to
|
||||
// globalThis.actions, and optionally a setup (precondition) action generator
|
||||
// to globalThis.setup.
|
||||
func (v *Verifier) Load(source string) error {
|
||||
if _, err := v.runtime.RunString(source); err != nil {
|
||||
return fmt.Errorf("run spec: %w", err)
|
||||
@@ -88,6 +90,10 @@ func (v *Verifier) Load(source string) error {
|
||||
v.actionGenerator = actionsValue
|
||||
}
|
||||
|
||||
if setupValue := v.runtime.GlobalObject().Get("setup"); setupValue != nil && !goja.IsUndefined(setupValue) && !goja.IsNull(setupValue) {
|
||||
v.setupGenerator = setupValue
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -216,6 +222,10 @@ func (v *Verifier) PushSnapshot(input SnapshotInput) error {
|
||||
if err := v.runtime.GlobalObject().Set("state", state); err != nil {
|
||||
return fmt.Errorf("set state: %w", err)
|
||||
}
|
||||
// Extractor previous/current advance exactly once per PushSnapshot.
|
||||
// Predicate thunks read these slots but never trigger advancement, so
|
||||
// invoking a thunk multiple times between snapshots is value-stable.
|
||||
// refreshPredicateErrors relies on this to safely re-call predicates.
|
||||
for index, extractor := range v.extractors {
|
||||
previous := extractor.handle.Get("current")
|
||||
_ = extractor.handle.Set("previous", previous)
|
||||
@@ -254,6 +264,7 @@ func (v *Verifier) EvaluateProperties() map[string]ltl.Verdict {
|
||||
for name, evaluator := range v.evaluators {
|
||||
verdicts[name] = evaluator.ObserveAt(stepTime)
|
||||
}
|
||||
v.refreshPredicateErrors()
|
||||
return verdicts
|
||||
}
|
||||
|
||||
@@ -273,12 +284,22 @@ func (v *Verifier) Residuals() map[string]ltl.Formula {
|
||||
return residuals
|
||||
}
|
||||
|
||||
// NextAction resolves the root action generator into a single Action.
|
||||
// Returns ErrNoAction when no branch of the generator produces one after a
|
||||
// small number of retries. Retrying avoids wedging when most branches of a
|
||||
// weighted generator produce no action on the current screen (e.g. a gated
|
||||
// login-phone generator when the app is already past login).
|
||||
// NextAction resolves an action for the current step. The setup generator,
|
||||
// when registered, runs first; if it yields an action, that wins. When setup
|
||||
// returns ErrNoAction (all branches empty) the call falls through to the
|
||||
// root action generator with the existing retry semantics. Setup is consulted
|
||||
// every step, so state regression (e.g. a logout under fuzz) automatically
|
||||
// re-engages the precondition.
|
||||
func (v *Verifier) NextAction() (Action, error) {
|
||||
if v.setupGenerator != nil {
|
||||
action, err := v.resolveGenerator(v.setupGenerator)
|
||||
if err == nil {
|
||||
return action, nil
|
||||
}
|
||||
if !errors.Is(err, ErrNoAction) {
|
||||
return Action{}, err
|
||||
}
|
||||
}
|
||||
if v.actionGenerator == nil {
|
||||
return Action{}, ErrNoAction
|
||||
}
|
||||
@@ -302,15 +323,38 @@ func (v *Verifier) formulaThunk(index int) func() bool {
|
||||
formula := v.formulas[index]
|
||||
result, err := formula.predicate(goja.Undefined())
|
||||
if err != nil {
|
||||
if formula.err == nil {
|
||||
formula.err = err
|
||||
}
|
||||
formula.err = err
|
||||
return false
|
||||
}
|
||||
formula.err = nil
|
||||
return result.ToBoolean()
|
||||
}
|
||||
}
|
||||
|
||||
// refreshPredicateErrors re-invokes every registered predicate so that
|
||||
// formula.err reflects the current step rather than a latched first-step
|
||||
// throw. EvaluateProperties short-circuits once a property has latched to
|
||||
// violated, so without this refresh the runner's per-step "predicate error"
|
||||
// log freezes on whatever the predicate threw at step 1. The refreshed errors
|
||||
// have no effect on verdicts.
|
||||
//
|
||||
// Invariant: predicates may be re-invoked here outside the LTL gate that
|
||||
// would normally skip them (e.g. an `implies` consequent whose antecedent is
|
||||
// false). They must therefore be side-effect-free reads of extractor state;
|
||||
// any spec that asserts internal preconditions inside a predicate could
|
||||
// surface a spurious error in the inspect UI without affecting verdicts.
|
||||
func (v *Verifier) refreshPredicateErrors() {
|
||||
for _, formula := range v.formulas {
|
||||
result, err := formula.predicate(goja.Undefined())
|
||||
if err != nil {
|
||||
formula.err = err
|
||||
continue
|
||||
}
|
||||
_ = result
|
||||
formula.err = nil
|
||||
}
|
||||
}
|
||||
|
||||
// PredicateError returns the first goja error raised by any thunk in the
|
||||
// named property's formula tree, or nil if none fired. Callers typically
|
||||
// consult this after EvaluateProperties reports a violation to distinguish
|
||||
|
||||
Reference in new issue
Block a user