feat: LTL operators, sampling, and default generators (#17)

* feat(ltl): add Now/Next/Eventually/Implies/Or/And/Not formulas

Replace the fold-with-latch evaluator with a residual-formula reducer.
Each Observe() instantiates a fresh obligation from the root (stripping
an outer Always), reduces each pending obligation against current state,
latches Violated on first failure, and surfaces Pending verdicts for
deferred obligations. Existing Always/Pure/Thunk tests continue to pass.

* feat(ltl): support relative duration for eventually().within()

* feat(proto): add Swipe, PressKey, RecentLogs RPCs

* feat(verifier,runner): formula handles, new action kinds, rich state

- verifier: add formula-spec registry; bindNow/bindNext/bindEventually with
  chainable .implies/.or/.and/.not and .within(n,unit) on eventually; bindFrom
  for uniform sampling. bindAlways keeps accepting plain predicates.
- verifier: store lastTree, lastAction, step time, logs, exceptions on the
  Verifier; SnapshotInput replaces the (snapshots, tree) pair. stateObject now
  produces state.lastAction/time/logs/exceptions matching the TS State type.
- verifier: make taps/swipes/waitOnce/pressKey built-in generators actually
  fire; taps picks a clickable, enabled element from the last hierarchy.
- agent: add exceptions field to Message wire format.
- driver: add Swipe/PressKey/RecentLogs to Driver interface; wire maestro
  client and mock driver. LogEntry exposed for runner consumption.
- runner: apply Swipe/PressKey/Wait actions; collect logcat and exceptions;
  pass lastAction and step time into PushSnapshot.

* feat(spec-api): LTL operators, new actions, richer State

- ltl.ts exports now/next/eventually; always overload accepts a Formula
- types.ts: Formula gains implies/or/and/not; EventuallyFormula adds .within;
  State gains lastAction/time/logs/exceptions; Swipe/PressKey/Wait action types
- actions.ts: Swipe/PressKey/Wait/from constructors; waitOnce + pressKey
  default generators
- tests exercise the chaining, sampling, and new actions through a recorded
  fake runtime

* feat(sidecar): add swipe, pressKey, recentLogs RPC handlers

* feat(sdk-android): capture uncaught exceptions

Install a default uncaught handler on Uatu.start, chained with any
existing handler so Android's crash reporter still runs. Expose
Uatu.reportError for callers to forward caught throwables. A bounded
circular buffer (default 50) drains into each STATE message's new
exceptions field. Protocol.kt serializes/deserializes the field,
matching the Go wire format added to internal/agent/protocol.go.

* feat(spec-api): add @uatu/spec/defaults/properties bundle

* feat(sample-app): exercise new LTL operators + defaults

spec.ts now imports eventually/next/now/from from @uatu/spec and
noUncaughtExceptions from @uatu/spec/defaults/properties. It declares
three properties that exercise the new surface:

- accountCountNonNegative: plain always() safety
- addAccountAdvances: always(now(x).implies(next(y)))
- eventuallyLoggedIn: eventually(p).within(30, "seconds")
- noUncaughtExceptions: imported default

The weighted actions root uses from() for random phone/name sampling
and entries for taps/swipes/waitOnce/pressKey built-ins.

SampleApplication gains a debug hook gated on the system property
uatu.inject_error so the e2e run can synthesize an Uatu.reportError and
verify noUncaughtExceptions violates.

cmd/uatu/test_run.go adds a subpath alias so specs importing
"@uatu/spec/defaults/properties" resolve against the in-tree source
when running from the uatu checkout. The spec-integration tests swap
the old click-counter fixtures for the new login hierarchy.

* feat(trace): record swipe/key/wait details + exceptions

trace.Step gains an Exceptions array so the trace captures the
class/message/stackTrace for each SDK-reported throwable in a step.
trace.Action gains FromX/FromY/ToX/ToY/Key/DurationMillis so the full
payload of Swipe/PressKey/Wait actions is visible in trace.jsonl.

sample-app's debug error hook now gates on ApplicationInfo.DEBUGGABLE
instead of a system property (adb setprop fails on non-rooted
emulators).
This commit is contained in:
pj authored and GitHub committed 2026-04-20 02:19:39 +07:00
1 parent 5b5594ee05
commit 7493945251
40 files changed
+2980 -262

No files matched your search

+189 -21
View File
@@ -1,12 +1,16 @@
package ltl
import "fmt"
import (
"fmt"
"time"
)
type Verdict int
const (
VerdictHolds Verdict = iota
VerdictViolated
VerdictPending
)
func (v Verdict) String() string {
@@ -15,46 +19,210 @@ func (v Verdict) String() string {
return "holds"
case VerdictViolated:
return "violated"
case VerdictPending:
return "pending"
default:
return fmt.Sprintf("verdict(%d)", int(v))
}
}
// Evaluator folds a formula across observed steps. v0.1 semantics:
// Always(P) is satisfied if P held at every observed step; once P is false,
// the verdict latches to Violated.
// Evaluator reduces a formula across observed steps using residual-formula
// semantics. Each step either resolves pending obligations (to holds or
// violated) or carries them forward as residuals. Once a single obligation
// violates, the overall verdict latches to Violated.
type Evaluator struct {
formula Formula
root Formula
pending []Formula
violated bool
}
func NewEvaluator(formula Formula) *Evaluator {
return &Evaluator{formula: formula}
return &Evaluator{root: formula}
}
// Observe evaluates the formula against the current state and returns the
// running verdict. Once Violated, subsequent calls keep returning Violated
// regardless of what later observations look like.
// running verdict. Uses the real wall clock for deadline-bound operators;
// callers that need reproducible time should use ObserveAt.
func (e *Evaluator) Observe() Verdict {
return e.ObserveAt(time.Now())
}
// ObserveAt is like Observe but takes the current step time explicitly.
func (e *Evaluator) ObserveAt(now time.Time) Verdict {
if e.violated {
return VerdictViolated
}
if !holdsAtCurrentStep(e.formula) {
e.violated = true
return VerdictViolated
fresh := rootObligation(e.root)
obligations := append(e.pending, fresh)
e.pending = e.pending[:0]
for _, obligation := range obligations {
result := reduce(obligation, now)
switch result.status {
case statusHolds:
// drop
case statusViolated:
e.violated = true
e.pending = nil
return VerdictViolated
case statusPending:
e.pending = append(e.pending, result.formula)
}
}
if len(e.pending) > 0 {
return VerdictPending
}
return VerdictHolds
}
func holdsAtCurrentStep(formula Formula) bool {
switch concrete := formula.(type) {
case AlwaysFormula:
return holdsAtCurrentStep(concrete.Inner)
case PureFormula:
return concrete.Value
case ThunkFormula:
return concrete.Func()
default:
panic(fmt.Sprintf("ltl: unsupported formula type %T", formula))
// rootObligation returns the formula to instantiate at each step. An outer
// Always is stripped so its inner is re-evaluated every step; any other root
// formula is itself re-instantiated each step (matching the v0.1 semantics
// where a bare Thunk is re-observed on every call).
func rootObligation(root Formula) Formula {
if always, ok := root.(AlwaysFormula); ok {
return always.Inner
}
return root
}
type residualStatus int
const (
statusHolds residualStatus = iota
statusViolated
statusPending
)
type reduceResult struct {
status residualStatus
formula Formula
}
func holds() reduceResult { return reduceResult{status: statusHolds} }
func violated() reduceResult { return reduceResult{status: statusViolated} }
func pending(f Formula) reduceResult {
return reduceResult{status: statusPending, formula: f}
}
func reduce(formula Formula, now time.Time) reduceResult {
switch concrete := formula.(type) {
case PureFormula:
if concrete.Value {
return holds()
}
return violated()
case ThunkFormula:
if concrete.Func() {
return holds()
}
return violated()
case NowFormula:
return reduce(concrete.Inner, now)
case NextFormula:
// Next defers the inner obligation to the following step without
// evaluating it now.
return pending(concrete.Inner)
case EventuallyFormula:
// First-reduction deadline resolution: if the formula was built with
// a relative duration, fix the absolute deadline to (now + duration)
// so subsequent reductions compare against a stable value.
if !concrete.HasDeadline && concrete.Duration > 0 {
concrete.Deadline = now.Add(concrete.Duration)
concrete.HasDeadline = true
}
innerResult := reduce(concrete.Inner, now)
if innerResult.status == statusHolds {
return holds()
}
if concrete.HasStepBound && concrete.StepBound <= 1 {
return violated()
}
if concrete.HasDeadline && !now.Before(concrete.Deadline) {
return violated()
}
next := concrete
if concrete.HasStepBound {
next.StepBound = concrete.StepBound - 1
}
return pending(next)
case ImpliesFormula:
antecedent := reduce(concrete.Antecedent, now)
switch antecedent.status {
case statusHolds:
return reduce(concrete.Consequent, now)
case statusViolated:
return holds()
case statusPending:
return pending(ImpliesFormula{
Antecedent: antecedent.formula,
Consequent: concrete.Consequent,
})
}
case OrFormula:
left := reduce(concrete.Left, now)
right := reduce(concrete.Right, now)
if left.status == statusHolds || right.status == statusHolds {
return holds()
}
if left.status == statusViolated && right.status == statusViolated {
return violated()
}
if left.status == statusViolated {
return pending(right.formula)
}
if right.status == statusViolated {
return pending(left.formula)
}
return pending(OrFormula{Left: left.formula, Right: right.formula})
case AndFormula:
left := reduce(concrete.Left, now)
right := reduce(concrete.Right, now)
if left.status == statusViolated || right.status == statusViolated {
return violated()
}
if left.status == statusHolds && right.status == statusHolds {
return holds()
}
if left.status == statusHolds {
return pending(right.formula)
}
if right.status == statusHolds {
return pending(left.formula)
}
return pending(AndFormula{Left: left.formula, Right: right.formula})
case NotFormula:
inner := reduce(concrete.Inner, now)
switch inner.status {
case statusHolds:
return violated()
case statusViolated:
return holds()
case statusPending:
return pending(NotFormula{Inner: inner.formula})
}
case AlwaysFormula:
innerResult := reduce(concrete.Inner, now)
if innerResult.status == statusViolated {
return violated()
}
next := AlwaysFormula{Inner: concrete.Inner}
if innerResult.status == statusHolds {
return pending(next)
}
return pending(AndFormula{Left: innerResult.formula, Right: next})
}
panic(fmt.Sprintf("ltl: unsupported formula type %T", formula))
}
+2 -1
View File
@@ -3,6 +3,7 @@ package ltl
import (
"strings"
"testing"
"time"
)
func observe(formula Formula, count int) []Verdict {
@@ -119,5 +120,5 @@ func TestObserve_PanicsOnUnknownFormulaType(t *testing.T) {
t.Errorf("expected panic on unsupported formula type")
}
}()
holdsAtCurrentStep(unsupportedFormula{})
reduce(unsupportedFormula{}, time.Now())
}
+115 -7
View File
@@ -1,10 +1,12 @@
package ltl
import "fmt"
import (
"fmt"
"strings"
"time"
)
// Formula is the AST of a temporal logic property. v0.1 supports only Always
// over Pure/Thunk leaves; eventually, next, and bounded operators are
// deferred to v0.2+.
// Formula is the AST of a temporal logic property.
type Formula interface {
isFormula()
describe() string
@@ -22,19 +24,125 @@ type ThunkFormula struct {
Func func() bool
}
// NowFormula marks its inner formula for evaluation at the current step only.
// Primarily used so that now(...).implies(...) parses unambiguously.
type NowFormula struct {
Inner Formula
}
// NextFormula obliges its inner formula to hold at the next step (not this one).
type NextFormula struct {
Inner Formula
}
// EventuallyFormula obliges its inner formula to hold at some step within the
// given bound. An unbounded eventually never triggers a violation within a
// finite run.
//
// When Duration is non-zero and Deadline is the zero time, the evaluator
// resolves the absolute deadline on first reduction using the observation
// time. This matches the "within N seconds of obligation instantiation"
// semantics used by nested Always(Eventually(...).within(...)) formulas.
type EventuallyFormula struct {
Inner Formula
StepBound int
HasStepBound bool
Duration time.Duration
Deadline time.Time
HasDeadline bool
}
type ImpliesFormula struct {
Antecedent Formula
Consequent Formula
}
type OrFormula struct {
Left Formula
Right Formula
}
type AndFormula struct {
Left Formula
Right Formula
}
type NotFormula struct {
Inner Formula
}
func Always(inner Formula) Formula { return AlwaysFormula{Inner: inner} }
func Pure(value bool) Formula { return PureFormula{Value: value} }
func Thunk(function func() bool) Formula { return ThunkFormula{Func: function} }
func (AlwaysFormula) isFormula() {}
func (PureFormula) isFormula() {}
func (ThunkFormula) isFormula() {}
func Now(inner Formula) Formula { return NowFormula{Inner: inner} }
func Next(inner Formula) Formula { return NextFormula{Inner: inner} }
func Eventually(inner Formula) Formula { return EventuallyFormula{Inner: inner} }
func EventuallyWithinSteps(inner Formula, steps int) Formula {
return EventuallyFormula{Inner: inner, StepBound: steps, HasStepBound: true}
}
func EventuallyBefore(inner Formula, deadline time.Time) Formula {
return EventuallyFormula{Inner: inner, Deadline: deadline, HasDeadline: true}
}
func EventuallyWithin(inner Formula, duration time.Duration) Formula {
return EventuallyFormula{Inner: inner, Duration: duration}
}
func Implies(antecedent, consequent Formula) Formula {
return ImpliesFormula{Antecedent: antecedent, Consequent: consequent}
}
func Or(left, right Formula) Formula { return OrFormula{Left: left, Right: right} }
func And(left, right Formula) Formula { return AndFormula{Left: left, Right: right} }
func Not(inner Formula) Formula { return NotFormula{Inner: inner} }
func (AlwaysFormula) isFormula() {}
func (PureFormula) isFormula() {}
func (ThunkFormula) isFormula() {}
func (NowFormula) isFormula() {}
func (NextFormula) isFormula() {}
func (EventuallyFormula) isFormula() {}
func (ImpliesFormula) isFormula() {}
func (OrFormula) isFormula() {}
func (AndFormula) isFormula() {}
func (NotFormula) isFormula() {}
func (a AlwaysFormula) describe() string { return "Always(" + a.Inner.describe() + ")" }
func (p PureFormula) describe() string { return fmt.Sprintf("Pure(%t)", p.Value) }
func (ThunkFormula) describe() string { return "Thunk(...)" }
func (n NowFormula) describe() string { return "Now(" + n.Inner.describe() + ")" }
func (n NextFormula) describe() string { return "Next(" + n.Inner.describe() + ")" }
func (e EventuallyFormula) describe() string {
parts := []string{e.Inner.describe()}
if e.HasStepBound {
parts = append(parts, fmt.Sprintf("steps=%d", e.StepBound))
}
if e.HasDeadline {
parts = append(parts, "deadline="+e.Deadline.Format(time.RFC3339Nano))
} else if e.Duration > 0 {
parts = append(parts, "within="+e.Duration.String())
}
return "Eventually(" + strings.Join(parts, ", ") + ")"
}
func (i ImpliesFormula) describe() string {
return "Implies(" + i.Antecedent.describe() + ", " + i.Consequent.describe() + ")"
}
func (o OrFormula) describe() string {
return "Or(" + o.Left.describe() + ", " + o.Right.describe() + ")"
}
func (a AndFormula) describe() string {
return "And(" + a.Left.describe() + ", " + a.Right.describe() + ")"
}
func (n NotFormula) describe() string { return "Not(" + n.Inner.describe() + ")" }
// Describe returns a debug-friendly representation of the formula.
func Describe(formula Formula) string { return formula.describe() }
+193
View File
@@ -0,0 +1,193 @@
package ltl
import (
"strings"
"testing"
"time"
)
func TestDescribe_NowNextEventually(t *testing.T) {
now := Always(Now(Pure(true)))
if got := Describe(now); !strings.Contains(got, "Now") || !strings.Contains(got, "Always") {
t.Errorf("Describe(Always(Now(Pure(true)))) = %q", got)
}
next := Always(Next(Pure(false)))
if got := Describe(next); !strings.Contains(got, "Next") {
t.Errorf("Describe next = %q", got)
}
ev := Always(EventuallyWithinSteps(Pure(true), 3))
if got := Describe(ev); !strings.Contains(got, "Eventually") || !strings.Contains(got, "steps=3") {
t.Errorf("Describe eventually = %q", got)
}
}
func TestDescribe_ImpliesOrAndNot(t *testing.T) {
implies := Implies(Pure(true), Pure(false))
if got := Describe(implies); !strings.Contains(got, "Implies") {
t.Errorf("Describe implies = %q", got)
}
or := Or(Pure(true), Pure(false))
if got := Describe(or); !strings.Contains(got, "Or") {
t.Errorf("Describe or = %q", got)
}
and := And(Pure(true), Pure(false))
if got := Describe(and); !strings.Contains(got, "And") {
t.Errorf("Describe and = %q", got)
}
not := Not(Pure(true))
if got := Describe(not); !strings.Contains(got, "Not") {
t.Errorf("Describe not = %q", got)
}
}
func TestAlways_Now_ViolatesImmediately(t *testing.T) {
evaluator := NewEvaluator(Always(Now(Pure(false))))
if got := evaluator.Observe(); got != VerdictViolated {
t.Errorf("step 1: got %v, want violated", got)
}
}
func TestAlways_Next_PendingThenViolated(t *testing.T) {
y := true
evaluator := NewEvaluator(Always(Next(Thunk(func() bool { return y }))))
if got := evaluator.Observe(); got != VerdictPending {
t.Errorf("step 1: got %v, want pending", got)
}
y = false
if got := evaluator.Observe(); got != VerdictViolated {
t.Errorf("step 2: got %v, want violated", got)
}
}
func TestAlways_Next_StaysPendingWhileInnerHolds(t *testing.T) {
evaluator := NewEvaluator(Always(Next(Thunk(func() bool { return true }))))
for index := range 3 {
if got := evaluator.ObserveAt(time.Unix(int64(index), 0)); got != VerdictPending {
t.Errorf("step %d: got %v, want pending", index+1, got)
}
}
}
func TestAlways_NowImpliesEventuallyWithin_ViolatesWhenYLate(t *testing.T) {
// always(now(() => x).implies(eventually(() => y).within(3, "steps")))
// x = true only at step 1; y = true only at step 4.
xValues := []bool{true, false, false, false, false}
yValues := []bool{false, false, false, true, true}
step := 0
predX := Thunk(func() bool { return xValues[step] })
predY := Thunk(func() bool { return yValues[step] })
formula := Always(Implies(Now(predX), EventuallyWithinSteps(predY, 3)))
evaluator := NewEvaluator(formula)
verdicts := make([]Verdict, 0, 5)
for range 5 {
verdicts = append(verdicts, evaluator.Observe())
step++
}
// Step 1: X true, eventually(Y, 3) spawned pending. Pending.
// Step 2: pending eventually decrements (Y false). Pending.
// Step 3: eventually bound exhausted (Y still false). Violated.
if verdicts[0] != VerdictPending {
t.Errorf("step 1: got %v, want pending", verdicts[0])
}
if verdicts[1] != VerdictPending {
t.Errorf("step 2: got %v, want pending", verdicts[1])
}
if verdicts[2] != VerdictViolated {
t.Errorf("step 3: got %v, want violated", verdicts[2])
}
}
func TestAlways_NowImpliesEventuallyWithin_HoldsWhenYInBound(t *testing.T) {
// Same formula, y = true at step 3 (within the 3-step bound).
xValues := []bool{true, false, false}
yValues := []bool{false, false, true}
step := 0
predX := Thunk(func() bool { return xValues[step] })
predY := Thunk(func() bool { return yValues[step] })
formula := Always(Implies(Now(predX), EventuallyWithinSteps(predY, 3)))
evaluator := NewEvaluator(formula)
verdicts := make([]Verdict, 0, 3)
for range 3 {
verdicts = append(verdicts, evaluator.Observe())
step++
}
if verdicts[0] != VerdictPending {
t.Errorf("step 1: got %v, want pending", verdicts[0])
}
if verdicts[1] != VerdictPending {
t.Errorf("step 2: got %v, want pending", verdicts[1])
}
if verdicts[2] != VerdictHolds {
t.Errorf("step 3: got %v, want holds", verdicts[2])
}
}
func TestEventually_DeadlineViolation(t *testing.T) {
base := time.Unix(0, 0)
deadline := base.Add(1 * time.Second)
formula := Always(EventuallyBefore(Pure(false), deadline))
evaluator := NewEvaluator(formula)
// Well before deadline: pending.
if got := evaluator.ObserveAt(base.Add(100 * time.Millisecond)); got != VerdictPending {
t.Errorf("pre-deadline: got %v, want pending", got)
}
// At or past deadline: violated.
if got := evaluator.ObserveAt(base.Add(2 * time.Second)); got != VerdictViolated {
t.Errorf("post-deadline: got %v, want violated", got)
}
}
func TestEventually_RelativeDurationResolvesOnFirstReduce(t *testing.T) {
base := time.Unix(0, 0)
// One-shot Eventually (not wrapped in Always) with a 1s relative deadline.
evaluator := NewEvaluator(EventuallyWithin(Pure(false), 1*time.Second))
if got := evaluator.ObserveAt(base); got != VerdictPending {
t.Errorf("creation step: got %v, want pending", got)
}
if got := evaluator.ObserveAt(base.Add(500 * time.Millisecond)); got != VerdictPending {
t.Errorf("mid-window: got %v, want pending", got)
}
if got := evaluator.ObserveAt(base.Add(2 * time.Second)); got != VerdictViolated {
t.Errorf("past-window: got %v, want violated", got)
}
}
func TestOr_OneBranchHolds(t *testing.T) {
evaluator := NewEvaluator(Always(Or(Pure(false), Pure(true))))
if got := evaluator.Observe(); got != VerdictHolds {
t.Errorf("or(false,true): got %v, want holds", got)
}
}
func TestAnd_OneBranchViolatesLatches(t *testing.T) {
evaluator := NewEvaluator(Always(And(Pure(true), Pure(false))))
if got := evaluator.Observe(); got != VerdictViolated {
t.Errorf("and(true,false): got %v, want violated", got)
}
}
func TestNot_InvertsPure(t *testing.T) {
holds := NewEvaluator(Always(Not(Pure(false))))
if got := holds.Observe(); got != VerdictHolds {
t.Errorf("not(false): got %v, want holds", got)
}
violates := NewEvaluator(Always(Not(Pure(true))))
if got := violates.Observe(); got != VerdictViolated {
t.Errorf("not(true): got %v, want violated", got)
}
}
func TestVerdict_StringPending(t *testing.T) {
if got := VerdictPending.String(); got != "pending" {
t.Errorf("VerdictPending.String() = %q", got)
}
}