fix(runner,sample-app): surface silent errors and demo a failing property (#15)

* fix(runner): surface non-deadline WaitForIdle errors

Previously the WaitForIdle return value was discarded entirely, hiding
real driver failures (gRPC transport errors, sidecar crashes) behind
the expected deadline-exceeded case. Log non-deadline errors so they
are visible without changing control flow.

* chore(sample-app): drop unused uptime_millis extractor

Registered in SampleApplication but never consumed by spec.ts.

* fix(sample-app): drop trivial appIsRunning property

app_state was hardcoded to 'running' so the property was a tautology
that could never fail. Removing both the extractor and the property
is the simplest fix; demo-grade properties that can fail land next.

* feat(sample-app): add Reset button that zeroes clickCount

Pairs with the next commit's tap-reset action so the fuzzer can
violate clickCountNeverDecreases and demonstrate uatu actually
finding a property violation.

* feat(sample-app): add tap-reset action to exercise Reset button

Weighted at 10/122, fuzzer reaches it within a short run. Pairs with
the Reset button to demonstrate uatu detecting the
clickCountNeverDecreases violation.

* fix(runner): filter WaitForIdle errors via context state, not errors.Is

errors.Is(err, context.DeadlineExceeded) misses gRPC's wrapped
status.DeadlineExceeded, so every step under the maestro driver
logged a spurious warning. Check idleCtx.Err() instead — captures
both deadline-fired and parent-canceled cases regardless of how the
driver wraps them.

* chore(sample-app): tune action weights so demo violates in ~30s

Prior weights left tap-reset rare enough that short demo runs missed
the violation by chance. Bumped to 30/107, with typeUsername reduced
since username noise doesn't help exercise clickCount.

* refactor(runner): route warnings through slog

Adds Options.Logger (defaults to slog.Default()) and converts the
three warning sites that were using fmt.Printf. Progress line stays
on Printf since it's user-facing UI, not a log. Makes the warnings
testable via a capturing handler.

* test(runner): assert WaitForIdle driver errors are logged

Captures slog output via TextHandler into a buffer and asserts the
warning message + injected error text appear when the mock driver
returns a non-context error from WaitForIdle. Guards against a
regression of the silent-error swallow.
This commit is contained in:
pj authored and GitHub committed 2026-04-18 18:48:25 +07:00
1 parent 00f66ba48d
commit 69002b07b1
5 files changed
+67 -13

No files matched your search

+12 -3
View File
@@ -5,6 +5,7 @@ import (
"encoding/json"
"errors"
"fmt"
"log/slog"
"time"
"github.com/priyanshujain/uatu/internal/agent"
@@ -24,6 +25,7 @@ type Options struct {
Driver driver.Driver
Verifier *verifier.Verifier
TraceWriter *trace.Writer
Logger *slog.Logger
}
type Summary struct {
@@ -46,6 +48,10 @@ func Run(ctx context.Context, options Options) (Summary, error) {
if err := validate(options); err != nil {
return Summary{}, err
}
logger := options.Logger
if logger == nil {
logger = slog.Default()
}
summary := Summary{StartTime: time.Now()}
deadline := summary.StartTime.Add(options.Duration)
@@ -64,7 +70,7 @@ func Run(ctx context.Context, options Options) (Summary, error) {
// a hierarchy that matches the snapshots captured a moment later.
tree, hierarchyErr := fetchHierarchy(ctx, options.Driver)
if hierarchyErr != nil {
fmt.Printf("warning: step %d hierarchy: %v\n", stepIndex, hierarchyErr)
logger.Warn("hierarchy fetch failed", "step", stepIndex, "err", hierarchyErr)
}
treeSize := 0
if tree != nil {
@@ -81,7 +87,7 @@ func Run(ctx context.Context, options Options) (Summary, error) {
}
screen, screenErr := screenFromSnapshot(snapshot.Snapshots)
if screenErr != nil {
fmt.Printf("warning: step %d screen: %v\n", stepIndex, screenErr)
logger.Warn("screen snapshot decode failed", "step", stepIndex, "err", screenErr)
}
fmt.Printf("step %d: screen=%q hierarchy=%d nodes\n",
stepIndex, screen, treeSize)
@@ -126,7 +132,10 @@ func Run(ctx context.Context, options Options) (Summary, error) {
}
idleCtx, idleCancel := context.WithTimeout(ctx, options.IdleTimeout)
_ = options.Driver.WaitForIdle(idleCtx, options.IdleTimeout)
idleErr := options.Driver.WaitForIdle(idleCtx, options.IdleTimeout)
if idleErr != nil && idleCtx.Err() == nil {
logger.Warn("wait_for_idle failed", "step", stepIndex, "err", idleErr)
}
idleCancel()
}