follow-ups from the pr #73 review (#77)

* ci(folio): run gradle on jdk 21 for the metro plugin

the metro gradle plugin folio builds with publishes org.gradle.jvm.version 21
and java 21 class files, so every leg failed at the folio build on a 17
runtime. local builds pass on jdk 25, which is why only ci saw it.

* fix(build): clean pkg/spec/dist, not the dead spec-api path

* chore: point stale spec-api comments at pkg/spec

* fix(spec): publish src so an installed package carries the runtime entries

* fix(testrun): alias the installed spec package so one module graph loads

* fix(spec): export Direction, ScrollAction and LongPressAction from the entry

* docs(spec): cut the package readme to a description and doc links

* docs: say how the cli and spec package versions relate

* fix(verifier): report whether the last action was confirmed applied

Both hosts get applied: true when the runner saw the dispatch succeed and
applied: null when it could not, so an unconfirmed action stops arriving at
the spec as no action at all.

* fix(runner): an apply error leaves the action's fate unknown, not undone

A deadline that fires after the tap was dispatched leaves the effect
committed. Reporting nil made the spec see an effect with no action to cause
it, which is how the counting property convicts a healthy app.

* fix(release): stage the sidecar jar at the renamed embed path

* test(replay-ui): trace fixtures for the vacuity counts

one real green run, one run that rendered nothing, one that judges every property at least once.

* ci(replay-ui): count the steps each property judged

the exit code says no property returned false; it does not say any property was ever evaluated. this reads the trace and reports judged vs declined per property, and fails when the step page never rendered.

* test(replay-ui): cover the summary script from make test

* ci(replay-ui): summarise through the vacuity script

* docs(ci): explain the replay-ui judged/declined counts

* fix(verifier): encode element-valued extractors into the trace

An ax element exports with its find/findAll host functions attached, and
json.Marshal refuses the whole value over them: json: unsupported type:
func(goja.FunctionCall) goja.Value. The encoding failed, curr stayed nil,
and the goja hosts (ios, android) recorded null for every element-valued
extractor in both the per-step diff and the violation witness.

Apply the web host's sanitize rule before marshaling, so one rule encodes
an element on both hosts.

* test(verifier): pin element encoding to one rule on both hosts

* test(runner): assert an element reaches trace.jsonl and its witness

* feat(spec): give state.lastAction an applied field

Three states, not two: no action is a null lastAction, applied: true is an
action the runner confirmed, applied: null is one it dispatched and never
learned the fate of.

* fix(folio): do not attribute an effect to an unconfirmed action

submitChangesBalanceByTypedAmount and createdAccountHasNonZeroBalance both
convict by pinning an effect on the last action, so both decline unless the
runner saw it applied. The fixtures now say which fate they mean.

* test(folio): an unconfirmed submit belongs in the window

The count is an upper bound on the submits a window holds, so the tap that may
have landed counts and committedTransactionsExceedSubmits has nothing to
convict on.

* test(runner): a tap that lands under a failed apply is not a double submit

Drives the real folio counting predicates through the runner against a device
that commits the tap and then times out. The double-submit case is the control:
without it a green proves only that the property never fired.

* test(verifier): pin the three lastAction states on both hosts

The web page is handed the same applied field the goja object exposes, so a
property cannot read one thing on native and another on web.

* docs(spec-language): document the three lastAction states

pkg/spec/test/api.test.ts

@@ -29,6 +29,12 @@ import {
   weighted,
   whenRoute,
 } from "../src/index.ts";
+import type {
+  Action,
+  Direction,
+  LongPressAction,
+  ScrollAction,
+} from "../src/index.ts";
 import { setSamplerRng } from "../src/actions.ts";
 import { Pcg } from "../src/pcg.ts";
 import type { GeneratorNode } from "../src/action-tree.ts";
@@ -397,3 +403,18 @@ test("whenRoute body is skipped for a null route", () => {
   const node = whenRoute(route, ["home"], () => [Tap({ on: "id:x" })]);
   assert.deepEqual((node as { generate: () => unknown }).generate(), []);
 });
+
+// The package entry is the only module a spec author can import from, so every
+// member of the exported Action union, and the Direction needed to build a
+// Scroll, has to be reachable there rather than only from src/types.ts.
+test("index exports every action type a spec author annotates with", () => {
+  const direction: Direction = "down";
+  const scroll: ScrollAction = Scroll({ direction, in: "id:list" });
+  const longPress: LongPressAction = LongPress({ on: "id:row" });
+  const built: Action[] = [scroll, longPress];
+
+  assert.deepEqual(
+    built.map(action => action.kind),
+    ["Scroll", "LongPress"],
+  );
+});

pkg/spec/test/folio-new-account.test.ts

@@ -3,8 +3,12 @@ import { test } from "node:test";
 
 import { createdAccountHasNonZeroBalance } from "../../../examples/folio/sanderling/predicates.ts";
 
-const created = { kind: "Tap", on: "testTag:AddAccountScreen > testTag:AddAccountSubmit" };
-const idle = { kind: "Tap", on: "testTag:HomeScreen > testTag:AccountCard" };
+const created = {
+  kind: "Tap",
+  on: "testTag:AddAccountScreen > testTag:AddAccountSubmit",
+  applied: true as const,
+};
+const idle = { kind: "Tap", on: "testTag:HomeScreen > testTag:AccountCard", applied: true as const };
 
 const account = (name: string, balance: number | null) => ({ name, balance });
 
@@ -27,7 +31,7 @@ test("a double-tapped create is judged the same way", () => {
   assert.equal(
     createdAccountHasNonZeroBalance({
       route: "home",
-      lastAction: { kind: "DoubleTap", on: "id:AddAccountSubmit" },
+      lastAction: { kind: "DoubleTap", on: "id:AddAccountSubmit", applied: true },
       typedName: "Travel",
       before: [account("Checking", 0)],
       after: [account("Checking", 0), account("Travel", 5000)],
@@ -233,3 +237,20 @@ test("a card that was already there is not a card that was just created", () =>
     false,
   );
 });
+
+// The apply call failed with the gesture possibly already delivered, so nobody
+// knows whether that account was created. The card carrying the typed name may
+// be an older one that scrolled into view, and attributing it to a creation
+// that may never have happened is a conviction built on a guess.
+test("a create the runner could not confirm attributes nothing", () => {
+  assert.equal(
+    createdAccountHasNonZeroBalance({
+      route: "home",
+      lastAction: { ...created, applied: null },
+      typedName: "Travel",
+      before: [account("Checking", 0)],
+      after: [account("Checking", 0), account("Travel", 5000)],
+    }),
+    false,
+  );
+});

pkg/spec/test/folio-submit-balance-predicate.test.ts

@@ -12,7 +12,7 @@ test("single submit: delta matches typed amount", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 500,
       prevTotalBalance: 1000,
@@ -26,7 +26,7 @@ test("double submit: delta is twice the typed amount, fires", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 500,
       prevTotalBalance: 1000,
@@ -40,7 +40,7 @@ test("DoubleTap kind also caught when delta exceeds typed amount", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "DoubleTap", on: submitOn },
+      lastAction: { kind: "DoubleTap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 500,
       prevTotalBalance: 0,
@@ -54,7 +54,7 @@ test("wrong action kind: vacuous true even with mismatch", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "InputText", on: submitOn },
+      lastAction: { kind: "InputText", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 500,
       prevTotalBalance: 1000,
@@ -68,7 +68,7 @@ test("wrong target: vacuous true even with mismatch", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: "testTag:LoginScreen > testTag:LoginSubmit" },
+      lastAction: { kind: "Tap", on: "testTag:LoginScreen > testTag:LoginSubmit", applied: true },
       submitsInWindow: 1,
       typedAmount: 500,
       prevTotalBalance: 1000,
@@ -96,7 +96,7 @@ test("zero typedAmount: vacuous true", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 0,
       prevTotalBalance: 1000,
@@ -110,7 +110,7 @@ test("selector as object: coerced safely and TxnSubmit detected", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: { testTag: "TxnSubmit" } },
+      lastAction: { kind: "Tap", on: { testTag: "TxnSubmit" }, applied: true },
       submitsInWindow: 1,
       typedAmount: 500,
       prevTotalBalance: 0,
@@ -124,7 +124,7 @@ test("selector as object without TxnSubmit: vacuous true", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: { testTag: "LoginSubmit" } },
+      lastAction: { kind: "Tap", on: { testTag: "LoginSubmit" }, applied: true },
       submitsInWindow: 1,
       typedAmount: 500,
       prevTotalBalance: 0,
@@ -138,7 +138,7 @@ test("raw whole-dollar input: single submit clears", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: parseTypedAmount("50"),
       prevTotalBalance: 5000,
@@ -152,7 +152,7 @@ test("raw whole-dollar input: double submit fires", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: parseTypedAmount("50"),
       prevTotalBalance: 5000,
@@ -166,7 +166,7 @@ test("decimal input from empty prior balance clears", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: parseTypedAmount("5.50"),
       prevTotalBalance: 0,
@@ -180,7 +180,7 @@ test("DoubleTap kind with raw whole-dollar input fires", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "DoubleTap", on: submitOn },
+      lastAction: { kind: "DoubleTap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: parseTypedAmount("100"),
       prevTotalBalance: 0,
@@ -194,7 +194,7 @@ test("route gate: ledger landing with stale carrier is skipped", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "ledger",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 5000,
       prevTotalBalance: 0,
@@ -208,7 +208,7 @@ test("route gate: add-transaction landing with double-submit delta is skipped",
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "add-transaction",
-      lastAction: { kind: "DoubleTap", on: submitOn },
+      lastAction: { kind: "DoubleTap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 5000,
       prevTotalBalance: 0,
@@ -222,7 +222,7 @@ test("route gate: null route is skipped", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: null,
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 5000,
       prevTotalBalance: 0,
@@ -236,7 +236,7 @@ test("route gate: home landing with matching delta passes", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 5000,
       prevTotalBalance: 0,
@@ -250,7 +250,7 @@ test("route gate: home landing with double-insert delta fires", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 5000,
       prevTotalBalance: 0,
@@ -275,7 +275,7 @@ test("above 2^53 a healthy single submit is not reported", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 1600,
       prevTotalBalance: HUGE_BALANCE,
@@ -289,7 +289,7 @@ test("above 2^53 a double-submit delta is not reported either", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 1600,
       prevTotalBalance: HUGE_BALANCE,
@@ -303,7 +303,7 @@ test("an unreadable previous balance above 2^53 is not evidence", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 1600,
       prevTotalBalance: HUGE_BALANCE,
@@ -320,7 +320,7 @@ test("typed amount above 2^53 is not evidence", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 1e23,
       prevTotalBalance: 0,
@@ -336,7 +336,7 @@ test("boundary: a double submit landing exactly on MAX_SAFE_INTEGER still fires"
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 4503599627370495,
       prevTotalBalance: 0,
@@ -350,7 +350,7 @@ test("boundary: a single submit landing exactly on MAX_SAFE_INTEGER passes", ()
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 9007199254740991,
       prevTotalBalance: 0,
@@ -364,7 +364,7 @@ test("boundary: one cent past MAX_SAFE_INTEGER stops being evidence", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 4503599627370496,
       prevTotalBalance: 0,
@@ -381,7 +381,7 @@ test("a large but exact difference between safe balances still fires", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 500,
       prevTotalBalance: -9007199254740991,
@@ -397,7 +397,7 @@ test("21-digit typed amount with an unmoved balance is not a violation", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: parseTypedAmount("999999999999999999999"),
       prevTotalBalance: 220900,
@@ -417,7 +417,7 @@ test("freshness: two submits in the window is vacuous, not a conviction", () =>
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "DoubleTap", on: submitOn },
+      lastAction: { kind: "DoubleTap", on: submitOn, applied: true },
       submitsInWindow: 2,
       typedAmount: 19600,
       prevTotalBalance: 0,
@@ -431,7 +431,7 @@ test("freshness: two submits cannot convict even on a clean 2x delta", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 2,
       typedAmount: 500,
       prevTotalBalance: 1000,
@@ -448,7 +448,7 @@ test("freshness boundary: exactly one submit is the window that convicts", () =>
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "DoubleTap", on: submitOn },
+      lastAction: { kind: "DoubleTap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 19600,
       prevTotalBalance: 0,
@@ -462,7 +462,7 @@ test("freshness boundary: one submit with a healthy 1x delta still passes", () =
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 1,
       typedAmount: 19600,
       prevTotalBalance: 0,
@@ -476,7 +476,7 @@ test("freshness boundary: three submits is vacuous", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 3,
       typedAmount: 500,
       prevTotalBalance: 0,
@@ -493,7 +493,7 @@ test("freshness boundary: a window with no submit in it is vacuous", () => {
   assert.equal(
     submitChangesBalanceByTypedAmount({
       route: "home",
-      lastAction: { kind: "Tap", on: submitOn },
+      lastAction: { kind: "Tap", on: submitOn, applied: true },
       submitsInWindow: 0,
       typedAmount: 500,
       prevTotalBalance: 1000,
@@ -502,3 +502,21 @@ test("freshness boundary: a window with no submit in it is vacuous", () => {
     true,
   );
 });
+
+// applied: null is the runner saying it dispatched the tap and never learned
+// whether it landed. A submit that committed nothing leaves the balance where
+// it was, so demanding the typed amount of movement for it convicts an app that
+// did exactly what it should have.
+test("a submit the runner could not confirm demands no balance move", () => {
+  assert.equal(
+    submitChangesBalanceByTypedAmount({
+      route: "home",
+      lastAction: { kind: "Tap", on: submitOn, applied: null },
+      submitsInWindow: 1,
+      typedAmount: 500,
+      prevTotalBalance: 1000,
+      currTotalBalance: 1000,
+    }),
+    true,
+  );
+});

pkg/spec/test/folio-submit-window.test.ts

@@ -2,6 +2,7 @@ import assert from "node:assert/strict";
 import { test } from "node:test";
 
 import {
+  committedTransactionsExceedSubmits,
   countSubmitsInWindow,
   isTxnSubmitTap,
   readHomeTotalBalance,
@@ -149,3 +150,25 @@ test("an unreadable Home does not close the window", () => {
   assert.equal(trace[2]?.total, null);
   assert.equal(trace[3]?.submits, 2);
 });
+
+// The window is an upper bound on the submits it holds, so a submit whose
+// dispatch the runner could not confirm belongs in it: the tap may well have
+// landed, and a bound that leaves it out is one the transaction it committed
+// exceeds. That is the false conviction, a rise of one against a window of
+// zero, on the property carrying most of the detection on android.
+test("a submit the runner could not confirm still counts toward the window", () => {
+  const window = countSubmitsInWindow({
+    previousCount: 0,
+    lastAction: { kind: "Tap", on: submitOn, applied: null },
+    fresh: true,
+  });
+  assert.equal(window.reported, 1);
+  assert.equal(
+    committedTransactionsExceedSubmits({
+      countsBefore: { Travel: 3 },
+      countsAfter: { Travel: 4 },
+      submitsInWindow: window.reported,
+    }),
+    false,
+  );
+});

pkg/spec/test/folio-transition-frame.test.ts

@@ -94,7 +94,7 @@ test("the measured android transition chain no longer convicts at delta 0", () =
   const step = (
     tags: string[],
     totalText: string | undefined,
-    lastAction: { kind: string; on: string } | null,
+    lastAction: { kind: string; on: string; applied: true } | null,
   ) => {
     const route = routeOfFrame(SCREENS, frame(...tags));
     const reading = readHomeTotalBalance({ route, totalText, previousCarrier: carrier });
@@ -104,8 +104,12 @@ test("the measured android transition chain no longer convicts at delta 0", () =
     return { route, total: reading.value, submits: window.reported };
   };
 
-  const back = { kind: "DoubleTap", on: "id:BackButton" };
-  const phantomSubmit = { kind: "Tap", on: "testTag:AddTransactionScreen > testTag:TxnSubmit" };
+  const back = { kind: "DoubleTap", on: "id:BackButton", applied: true as const };
+  const phantomSubmit = {
+    kind: "Tap",
+    on: "testTag:AddTransactionScreen > testTag:TxnSubmit",
+    applied: true as const,
+  };
 
   const transition = step(["AddTransactionScreen", "HomeScreen"], "$86,911.00", back);
   assert.equal(transition.route, null);