mirror of
https://github.com/priyanshujain/sanderling.git
synced 2026-10-02 11:07:10 +00:00
feat: uatu inspect UI (web trace explorer) (#23)
* feat(trace): extend Step/Action/Meta schema for inspect UI
Add Step.Hierarchy, Step.Residuals, Action.Selector/ResolvedBounds/TapPoint,
Meta.EndedAt and JSON tags on hierarchy.Element/Bounds/Tree so trace.jsonl
can drive the upcoming uatu inspect web UI.
* test(trace): cover EndedAt + new step fields round-trip
* feat(ltl): MarshalJSON for Formula AST + Evaluator.Residual()
Each Formula concrete type now serializes to a closed-set residual node
(true/false/not/and/or/implies/always/now/next/eventually/predicate/error)
that mirrors the TS spec API surface. Evaluator.Residual() folds pending
obligations into a single Formula so the runner can stamp one residual
per property per step into trace.jsonl.
* feat(runner): stamp residuals, hierarchy, selector targets, ended_at
Each Step now carries the captured hierarchy, per-property residual ASTs,
and (for Tap/InputText) the selector + resolved bounds + tap point. The
test_run command writes meta.ended_at on graceful shutdown so the inspect
UI can distinguish completed runs from in-progress ones.
* feat(inspect): scaffold embed dist for SPA assets
Stage 2 stub for the inspect server. Real web bundle gets wired in
Stage 4 (Makefile copies web/dist into internal/inspect/dist).
* chore(web): ignore web/ build output in root .gitignore
* chore(web): add bun + vite + vitest scaffold config
* feat(web): monochrome design tokens, typography, app shell CSS
* chore(web): placeholder for self-hosted JetBrains Mono fonts
* feat(web): index.html entry with style links and root mount
* feat(web): typescript types mirroring run/step trace schema
* feat(web): typed fetchers for runs/steps/screenshots
* feat(web): App shell with router and run/step routes
* feat(inspect): runs scan, lazy step parse, mtime-aware cache
* feat(web): RunList route with table, loading, and error states
* feat(web): RunDetail route shell with three placeholder panels
* feat(inspect): fsnotify-backed runs watcher with debounce
* fix(web): use jest-dom/vitest entry so matchers register
* test(web): cover listRuns happy path and error response
* test(web): render RunList with mocked fetch and assert row
* chore(web): commit bun lockfile
* feat(inspect): http handlers for runs/steps/screenshots/SSE
* test(inspect): cover handlers, screenshot whitelist, SSE, dev proxy
* feat(cmd): add 'uatu inspect' subcommand
* fix(web): align TS types with snake_case wire format
Go inspect server serializes RunSummary, StepSummary, Step, Meta with
snake_case JSON tags (matching the on-disk trace.jsonl/meta.json). Update
the TS types and consumers to match so API responses parse without
runtime undefined fields. Action keeps resolvedBounds/tapPoint as camelCase
because those keys were defined that way in the trace schema.
* feat(web): add ActionList panel for run-detail step navigation
* feat(web): add SnapshotTable panel with diff highlighting
Renders snapshots dictionary as a flat sorted dotted-path tree.
Changed leaves get data-changed plus a hover title with the previous value.
* test(web): cover SnapshotTable rendering and diff behavior
Eight cases: empty state, sort order, dotted-path expansion,
changed/unchanged/missing-previous flagging, and inline-vs-expanded arrays.
* feat(web): add Screenshot panel with bounds and tap overlays
Center column of run-detail page. Renders the device screenshot
scaled to fit, with an SVG overlay drawing resolvedBounds as a
violation-colored rect, tapPoint as a contrast ring, and swipes
as an arrow. Falls back to a placeholder when src is missing or
the image fails to load.
* fix(web): guard scrollIntoView call for jsdom compatibility
* test(web): cover Screenshot panel rendering and overlays
* test(web): cover ActionList rendering, selection, keyboard, and markers
* feat(web): add ExceptionsPanel component
* test(web): add ExceptionsPanel tests
* feat(web): add Timeline panel with property swimlanes
Renders SVG swimlanes per property with violated/pending/holds cells,
action-marker dots, click-to-seek, and a selected-step highlight bar.
* test(web): cover Timeline empty state, cells, status, click, highlight
* feat(web): add ResidualNode recursive AST renderer
* test(web): cover ResidualNode operators, predicate, and error chip
* feat(web): add ViolationsPanel with status badges and jump button
* test(web): cover ViolationsPanel rows, status grouping, and jump button
* test(web): register testing-library cleanup globally
All six panel test files added local afterEach(cleanup); centralize it in
the shared setup so future tests inherit DOM isolation by default.
* feat(web): hooks for url/keyboard/theme/sse
* feat(web): wire all panels into run-detail with phone-dominant grid
ActionList left, Screenshot center, Snapshots/Properties/Exceptions
stacked right, Timeline bottom. URL-synced step index (useStep), keyboard
shortcuts (j/k/arrows/g/G/.), light+dark theme toggle stored in
localStorage, SSE auto-refresh on the run index.
* test(web): add three reference run fixtures (clean, violation, exception)
* build: web targets in Makefile + bun in CI; docs(inspect)
- Makefile: web-build/web-dev/inspect-dev/test-web targets; uatu and
install now depend on web-build so the binary embeds the latest SPA.
- ci.yml: setup-bun + cache; existing make test now runs web typecheck +
vitest as part of the full suite.
- docs/manual/inspect.md: panel reference, keyboard shortcuts, URLs.
- docs/manual/cli.md: document uatu inspect.
- README: link to inspect docs.
* feat(runner): capture a screenshot per step
The driver already exposes Screenshot(ctx), but the runner never called
it. Each step now writes <run>/screenshots/step-NNNNN.png right after
the trace line, using the same failure-is-a-warning posture as other
best-effort observability hooks. Makes the inspect UI's center panel
actually useful.
* feat(inspect): include action_label in StepSummary
Tap/InputText/Swipe/PressKey/Wait each get a short human-readable
label (selector, quoted text, swipe direction, key name, duration) so
the action list panel can render readable rows instead of just 'Tap'
with no target.
* test(inspect): accept either #app or #root in SPA shell fallback
* feat(web): render action_label and screen in ActionList rows
Step rows now show 'Tap id:save', 'InputText "alice"', 'Swipe up',
'PressKey back', etc. Steps with no action fall back to
'observe @ <screen>' so the list reads as a flow instead of a wall
of '--' placeholders.
* feat(sidecar): implement screencap for android driver backend
Was stubbed to return an empty byte array, which made the runner's
per-step screenshot capture a no-op. Shell out to 'adb exec-out
screencap -p' and stream the PNG bytes back. Width/height stay zero
because the PNG header carries them; the Go side can parse if needed.
* feat(proto): add Metrics RPC for per-step CPU and memory capture
* feat(driver): Metrics(bundleID) returns cpu_percent + heap/total bytes
* feat(sidecar): implement Metrics RPC via adb top + /proc/<pid>/status
* feat(runner): capture metrics + before/after screenshots per step
Each step now writes step-NNNNN.png (before applyAction) and
step-NNNNN-after.png (after the action + wait-for-idle). The runner
samples Driver.Metrics(bundleID) before writing the trace line and
stamps Step.Metrics with cpu_percent, heap_bytes, total_memory_bytes
so the inspect UI can chart CPU and heap over the run.
* fix(runner,sidecar): measure CPU across step via /proc stat delta
'top -d 0.3 -n 2' measures CPU in a 300ms window that coincides with
the SDK-paused app, always reporting 0%. Switch to reading
/proc/<pid>/stat utime+stime and computing the delta between successive
calls; the natural step cadence gives a 2-5s measurement window that
captures the action response and render cycle. Also moved the sample
to before snapshotStep so the delta starts before the SDK pause.
* feat(web): add Metrics type for per-step cpu and memory
* refactor(web): replace --accent-change with --accent-positive token
* refactor(web): recolor chip-progress as neutral outlined chip
* refactor(web): use neutral border for changed snapshot rows
* feat(web): add MetricsChart panel with HEAP and CPU lanes
SVG-based time-series chart rendering heap bytes and CPU percent per
step across two stacked lanes, with a shared step axis below. Lines are
monochrome; a vertical highlight marks the selected step; per-step hit
rects make any click seek to that step.
* feat(web): revamp ActionList with tag targets, elapsed time, and expandable rows
Render selector-based Tap actions as <tag/> markup, show zero-padded MM:SS.mmm
elapsed time per row, and expand the active row with Position/Content sub-rows
when a full Step is available. Adds formatActionRow/formatElapsed helpers and
covers both with unit tests.
* fix(runner): stop copying Tap selector into action.text
The 'Content' inspect row should show the user-supplied text for
InputText actions and stay empty for Taps. Previously the runner copied
action.On into traceAction.Text for both, so the inspect UI showed the
selector as the tap's 'Content'.
* fix(web): use text-muted for swipe arrow after accent-change removal
* fix(web): snapshot values truncate with ellipsis + title tooltip
Long JSON values were breaking one character per line due to
overflow-wrap:anywhere in a narrow column. Switch to single-line ellipsis
with the full value exposed via the title attribute on hover.
* feat(web): state-before/after columns + metrics chart at bottom
RunDetail now renders a four-column grid:
actions | state-before | state-after | side (exceptions + timeline)
with MetricsChart spanning the bottom row. Each state column shows its
own screenshot (step-NNNNN.png vs step-NNNNN-after.png), snapshot table,
and violations panel. ActionList now receives runStartMillis and the
selected Step so the active row can expand Position/Content sub-rows.
* fix(web): skip zero-value ticks + add exception markers to metrics
HEAP '0B' and CPU '100%' labels overlapped at the lane boundary. Drop
the bottom-of-range tick on both lanes (baseline is implied) and widen
LANE_GAP so the remaining labels have breathing room. Accept an
exceptionStepIndices prop and draw a dashed red vertical line at each
to surface exception spikes directly on the CPU/heap chart.
* fix(web): let action body column shrink below its content
Required minmax(0, 1fr) so the row grid honours the column's min-size of
0 instead of the implicit 'auto', preventing the action-list from
overflowing its parent when the target string is long.
* feat(web): bigger state screenshots + single properties row
Collapse snapshots into a summary chip ('SNAPSHOTS · N violations') so
the screenshot fills its state card. Deduplicate ViolationsPanel —
show it once in a new full-width 'properties' row between the state
cards and the timeline. Drop the right sidebar; exceptions now surface
as dashed markers on the metrics chart with the ExceptionsPanel only
rendering when there are actual exceptions to report.
* feat(web): add minimal Tabs component
Monochrome tab strip with underline-on-active. Used by state-before
and state-after cards to swap between Screenshot, Snapshots, Properties.
Pane scrolls internally so the outer grid stays fixed-height.
* feat(web): fold timeline into MetricsChart as STEPS lane
Adds a thin per-step status row above HEAP showing violated (red),
pending (dim gray) or holds (green-tinted). Extends highlight +
exception markers to span the status lane. Frees a whole row in the
detail grid so the page can fit in 100vh.
* refactor(web): tabbed state cards, drop standalone Timeline panel
State-before/after now use Tabs (Screenshot / Snapshots / Properties,
default Screenshot). Removes the dedicated timeline row; status lane
lives on the metrics chart. Banner is gone from the shell.
* feat(web): lock app shell to 100vh with no page scroll
html/body/#root fill the viewport, body gets overflow:hidden, and the
detail grid uses minmax(0, 1fr) rows so inner panels own their scroll.
Tightens toolbar + panel padding for a denser feel.
* feat(web): arrow-key nav + badges on Tabs (WAI-ARIA tablist)
Roving tabindex, ArrowLeft/Right/Up/Down/Home/End navigation, explicit
aria-selected/aria-controls/id wiring, and support for an optional
badge inside each tab (used for violation counts).
* feat(web): ViolationsPanel supports violationsOnly filter
* feat(web): ActionList arrow-key nav + listbox semantics + smaller font
Promote the list to role=listbox with role=option rows; roving tabindex
lets ArrowUp/Down (and Home/End) seek between steps with focus. Font
size dropped to 11px and padding tightened so long selector-tag labels
fit in the 340px actions column.
* fix(web): useKeyboardNav yields arrow keys to tablist/listbox targets
Previously pressing ArrowRight on a focused tab switched tabs AND
advanced the step. Skip arrow handling when the event target is inside
an element with an arrow-owning ARIA role.
* feat(web): fourth 'Violations' tab + wider actions + shorter metrics
Adds a Violations tab to each state card showing only violated properties
(with count badge on the tab label when > 0). Actions column widened
from 280px to 340px, bottom metrics strip trimmed from 220px to 140px
with tighter lane heights, so the whole page still fits in 100vh with
no scrollbar.
* feat(web): compact RunDetail layout using 1px borders instead of panel padding
* refactor(inspect): simplify MetricsChart to HEAP+CPU with time axis
Drop the STEPS status lane and per-sample circle markers, switch the
x-axis from step indices to mm:ss clock time, trim y-axis ticks to
min/max with compact units, rotate lane labels into the left gutter,
and replace the thin playhead line with a wider dotted red band.
Traces stay grayscale; red appears only on the playhead pattern.
* fix(web): RunList rows no longer stretch to fill viewport height
Tables inherited flex: 1 1 auto from .app-main > * and distributed extra
vertical space across rows. Override with flex: 0 0 auto + align-self.
* misc changes
* fix(web): hoist useState above early return in MetricsChart
Calling useState after an unconditional early return violates React's
Rules of Hooks: the empty-samples branch renders 0 hooks while the
populated branch calls 1. On the initial null->loaded transition of
history the hook count changes and React throws.
* fix(web): subscribe to named SSE event instead of 'message'
Server emits 'event: runs.changed' frames; the WHATWG EventSource spec
dispatches those as events of type 'runs.changed', not 'message'. The
listener registered on 'message' was never fired, so RunList never
auto-refreshed on run create/finish/delete.
* fix(inspect): unsubscribe SSE clients on disconnect
Watcher.Subscribe appended to a slice with no matching removal path,
so every closed EventSource connection leaked its channel. Over a
long-running server the slice grew unbounded and every fs event paid
O(N) iterating dead channels. Add Unsubscribe + defer it in
handleEvents.
Unsubscribe does not close the channel: broadcast snapshots the
slice without holding the mutex, so a concurrent close would race
with its non-blocking send.
* fix(trace): rename resolvedBounds/tapPoint to snake_case
Every other json tag in the trace schema (from_x, duration_millis,
bundle_sha256, etc.) uses snake_case. The two new Action fields
introduced with the inspect UI broke that pattern. Rename them
before the format ships to external consumers.
* chore(web): drop vitest and remove UI tests from CI
No UI tests wanted in web. Removes vitest, jsdom, testing-library
devDeps and the vitest.setup.ts + vite.config.ts test block.
Makefile test-web becomes web-typecheck (typecheck only).
Fixes CI failure where `vitest run` exits 1 with no test files.
* chore(make): dedupe sidecar embed and drop recursive make
Make $(SIDECAR_JAR) the real recipe and $(SIDECAR_EMBED) a file
target, so uatu/install/inspect-dev share one copy step and
sidecar/release-cli just depend on the jar instead of re-invoking make.
This commit is contained in:
77 files changed
+6505
-73
No files matched your search
@@ -30,6 +30,11 @@ type Driver interface {
|
||||
|
||||
WaitForIdle(ctx context.Context, duration time.Duration) error
|
||||
Health(ctx context.Context) (Health, error)
|
||||
// Metrics samples the app's CPU and memory at the time of the call.
|
||||
// CPUPercent is percent of a single core (multi-core apps can exceed
|
||||
// 100). HeapBytes is resident set size; TotalMemoryBytes includes
|
||||
// native allocations.
|
||||
Metrics(ctx context.Context, bundleID string) (Metrics, error)
|
||||
}
|
||||
|
||||
type LogEntry struct {
|
||||
@@ -50,3 +55,9 @@ type Health struct {
|
||||
Version string
|
||||
Platform string
|
||||
}
|
||||
|
||||
type Metrics struct {
|
||||
CPUPercent float64
|
||||
HeapBytes int64
|
||||
TotalMemoryBytes int64
|
||||
}
|
||||
@@ -153,4 +153,16 @@ func (c *Client) Health(ctx context.Context) (driver.Health, error) {
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (c *Client) Metrics(ctx context.Context, bundleID string) (driver.Metrics, error) {
|
||||
response, err := c.stub.Metrics(ctx, &driverpb.MetricsRequest{BundleId: bundleID})
|
||||
if err != nil {
|
||||
return driver.Metrics{}, err
|
||||
}
|
||||
return driver.Metrics{
|
||||
CPUPercent: response.GetCpuPercent(),
|
||||
HeapBytes: response.GetHeapBytes(),
|
||||
TotalMemoryBytes: response.GetTotalMemoryBytes(),
|
||||
}, nil
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*Client)(nil)
|
||||
@@ -23,6 +23,7 @@ const (
|
||||
ActionRecentLogs ActionKind = "recent_logs"
|
||||
ActionWaitForIdle ActionKind = "wait_for_idle"
|
||||
ActionHealth ActionKind = "health"
|
||||
ActionMetrics ActionKind = "metrics"
|
||||
)
|
||||
|
||||
type Action struct {
|
||||
@@ -53,6 +54,7 @@ type Driver struct {
|
||||
ImageData driver.Image
|
||||
HealthInfo driver.Health
|
||||
LogEntries []driver.LogEntry
|
||||
MetricsData driver.Metrics
|
||||
Failures map[ActionKind]error
|
||||
}
|
||||
|
||||
@@ -208,4 +210,14 @@ func (d *Driver) Health(ctx context.Context) (driver.Health, error) {
|
||||
return d.HealthInfo, nil
|
||||
}
|
||||
|
||||
func (d *Driver) Metrics(ctx context.Context, bundleID string) (driver.Metrics, error) {
|
||||
if err := d.failure(ActionMetrics); err != nil {
|
||||
return driver.Metrics{}, err
|
||||
}
|
||||
d.record(Action{Kind: ActionMetrics, BundleID: bundleID})
|
||||
d.mutex.Lock()
|
||||
defer d.mutex.Unlock()
|
||||
return d.MetricsData, nil
|
||||
}
|
||||
|
||||
var _ driver.Driver = (*Driver)(nil)
|
||||
@@ -19,7 +19,10 @@ import (
|
||||
|
||||
// Bounds is an inclusive rectangle in device pixels.
|
||||
type Bounds struct {
|
||||
Left, Top, Right, Bottom int
|
||||
Left int `json:"left"`
|
||||
Top int `json:"top"`
|
||||
Right int `json:"right"`
|
||||
Bottom int `json:"bottom"`
|
||||
}
|
||||
|
||||
// Center returns the center point of the bounds.
|
||||
@@ -35,22 +38,22 @@ func (b Bounds) Height() int { return b.Bottom - b.Top }
|
||||
|
||||
// Element is a flattened view of one uiautomator node.
|
||||
type Element struct {
|
||||
ResourceID string
|
||||
Text string
|
||||
Description string
|
||||
Class string
|
||||
Package string
|
||||
Clickable bool
|
||||
Enabled bool
|
||||
Checked bool
|
||||
Focused bool
|
||||
Selected bool
|
||||
Bounds Bounds
|
||||
ResourceID string `json:"resourceId,omitempty"`
|
||||
Text string `json:"text,omitempty"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Class string `json:"class,omitempty"`
|
||||
Package string `json:"package,omitempty"`
|
||||
Clickable bool `json:"clickable,omitempty"`
|
||||
Enabled bool `json:"enabled,omitempty"`
|
||||
Checked bool `json:"checked,omitempty"`
|
||||
Focused bool `json:"focused,omitempty"`
|
||||
Selected bool `json:"selected,omitempty"`
|
||||
Bounds Bounds `json:"bounds"`
|
||||
}
|
||||
|
||||
// Tree is a flat collection of every node in a hierarchy dump, in pre-order.
|
||||
type Tree struct {
|
||||
Elements []*Element
|
||||
Elements []*Element `json:"elements"`
|
||||
}
|
||||
|
||||
// Parse parses a uiautomator-style XML dump.
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
package inspect
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"io/fs"
|
||||
)
|
||||
|
||||
//go:embed all:dist
|
||||
var distFS embed.FS
|
||||
|
||||
// Assets returns the embedded SPA bundle rooted at the dist directory.
|
||||
// In Stage 2 this contains a stub index.html. Stage 4 wires the real
|
||||
// bundle in via Makefile (copy web/dist -> internal/inspect/dist).
|
||||
func Assets() fs.FS {
|
||||
sub, err := fs.Sub(distFS, "dist")
|
||||
if err != nil {
|
||||
panic("inspect: dist embed missing: " + err.Error())
|
||||
}
|
||||
return sub
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
package inspect
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httputil"
|
||||
"net/url"
|
||||
)
|
||||
|
||||
// DevTarget is the hardcoded Vite dev server address. The --dev flag
|
||||
// enables a reverse proxy to this address for non-API requests.
|
||||
const DevTarget = "http://127.0.0.1:5173"
|
||||
|
||||
func newDevProxy(target string) (http.Handler, error) {
|
||||
parsed, err := url.Parse(target)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
proxy := httputil.NewSingleHostReverseProxy(parsed)
|
||||
return proxy, nil
|
||||
}
|
||||
Vendored
Whitespace-only changes.
Vendored
+13
@@ -0,0 +1,13 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>uatu inspect</title>
|
||||
<script type="module" crossorigin src="/assets/index-DFpYQDqN.js"></script>
|
||||
<link rel="stylesheet" crossorigin href="/assets/index-BSJUf7yI.css">
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,429 @@
|
||||
package inspect
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/priyanshujain/uatu/internal/trace"
|
||||
)
|
||||
|
||||
// Maximum size of a single trace.jsonl line. Hierarchies and snapshots
|
||||
// can be large; 16 MiB is enough headroom for realistic traces.
|
||||
const maxScanTokenSize = 16 * 1024 * 1024
|
||||
|
||||
// RunSummary is the lightweight per-run record returned by Scan and the
|
||||
// /api/runs handler. Keep this in lockstep with the JSON shape consumed
|
||||
// by the SPA's run list view.
|
||||
type RunSummary struct {
|
||||
ID string `json:"id"`
|
||||
StartedAt time.Time `json:"started_at"`
|
||||
EndedAt *time.Time `json:"ended_at,omitempty"`
|
||||
SpecPath string `json:"spec_path"`
|
||||
Seed int64 `json:"seed"`
|
||||
Platform string `json:"platform"`
|
||||
BundleID string `json:"bundle_id"`
|
||||
DurationMillis int64 `json:"duration_millis"`
|
||||
StepCount int `json:"step_count"`
|
||||
ViolationCount int `json:"violation_count"`
|
||||
InProgress bool `json:"in_progress"`
|
||||
}
|
||||
|
||||
// StepSummary is the slim per-step record used to render the step list
|
||||
// and timeline. Heavy fields (hierarchy, snapshots, residuals) are sent
|
||||
// only via the per-step endpoint.
|
||||
type StepSummary struct {
|
||||
Index int `json:"index"`
|
||||
Timestamp time.Time `json:"timestamp"`
|
||||
Screen string `json:"screen,omitempty"`
|
||||
ActionKind string `json:"action_kind,omitempty"`
|
||||
ActionLabel string `json:"action_label,omitempty"`
|
||||
HasViolations bool `json:"has_violations"`
|
||||
HasExceptions bool `json:"has_exceptions"`
|
||||
}
|
||||
|
||||
// RunDetail is the full /api/runs/{id} payload: meta + slim step list.
|
||||
type RunDetail struct {
|
||||
RunSummary
|
||||
Meta trace.Meta `json:"meta"`
|
||||
Steps []StepSummary `json:"steps"`
|
||||
}
|
||||
|
||||
// Run is a cached parse of one run directory. Step lookups re-read the
|
||||
// JSONL file from disk; only the line offsets are kept in memory.
|
||||
type Run struct {
|
||||
ID string
|
||||
Directory string
|
||||
Meta trace.Meta
|
||||
Summary RunSummary
|
||||
Steps []StepSummary
|
||||
tracePath string
|
||||
traceMtime time.Time
|
||||
offsets []int64
|
||||
}
|
||||
|
||||
// IsRunDirectory reports whether dir looks like a single run (has meta.json).
|
||||
func IsRunDirectory(directory string) bool {
|
||||
_, err := os.Stat(filepath.Join(directory, "meta.json"))
|
||||
return err == nil
|
||||
}
|
||||
|
||||
// Scan walks runsDirectory and returns one RunSummary per child directory
|
||||
// that contains meta.json. Results are sorted by StartedAt descending.
|
||||
func Scan(runsDirectory string) ([]RunSummary, error) {
|
||||
entries, err := os.ReadDir(runsDirectory)
|
||||
if err != nil {
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return []RunSummary{}, nil
|
||||
}
|
||||
return nil, fmt.Errorf("read runs dir: %w", err)
|
||||
}
|
||||
summaries := make([]RunSummary, 0, len(entries))
|
||||
for _, entry := range entries {
|
||||
if !entry.IsDir() {
|
||||
continue
|
||||
}
|
||||
runDirectory := filepath.Join(runsDirectory, entry.Name())
|
||||
summary, err := summarize(runDirectory, entry.Name())
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
summaries = append(summaries, summary)
|
||||
}
|
||||
sort.Slice(summaries, func(i, j int) bool {
|
||||
return summaries[i].StartedAt.After(summaries[j].StartedAt)
|
||||
})
|
||||
return summaries, nil
|
||||
}
|
||||
|
||||
func summarize(runDirectory, id string) (RunSummary, error) {
|
||||
meta, err := readMeta(runDirectory)
|
||||
if err != nil {
|
||||
return RunSummary{}, err
|
||||
}
|
||||
stepCount, violationCount, err := tallyTrace(filepath.Join(runDirectory, "trace.jsonl"))
|
||||
if err != nil {
|
||||
return RunSummary{}, err
|
||||
}
|
||||
return buildSummary(id, meta, stepCount, violationCount), nil
|
||||
}
|
||||
|
||||
func buildSummary(id string, meta trace.Meta, stepCount, violationCount int) RunSummary {
|
||||
summary := RunSummary{
|
||||
ID: id,
|
||||
StartedAt: meta.StartedAt,
|
||||
EndedAt: meta.EndedAt,
|
||||
SpecPath: meta.SpecPath,
|
||||
Seed: meta.Seed,
|
||||
Platform: meta.Platform,
|
||||
BundleID: meta.BundleID,
|
||||
StepCount: stepCount,
|
||||
ViolationCount: violationCount,
|
||||
InProgress: meta.EndedAt == nil,
|
||||
}
|
||||
if meta.EndedAt != nil {
|
||||
summary.DurationMillis = meta.EndedAt.Sub(meta.StartedAt).Milliseconds()
|
||||
}
|
||||
return summary
|
||||
}
|
||||
|
||||
func readMeta(runDirectory string) (trace.Meta, error) {
|
||||
body, err := os.ReadFile(filepath.Join(runDirectory, "meta.json"))
|
||||
if err != nil {
|
||||
return trace.Meta{}, fmt.Errorf("read meta: %w", err)
|
||||
}
|
||||
var meta trace.Meta
|
||||
if err := json.Unmarshal(body, &meta); err != nil {
|
||||
return trace.Meta{}, fmt.Errorf("decode meta: %w", err)
|
||||
}
|
||||
return meta, nil
|
||||
}
|
||||
|
||||
func tallyTrace(tracePath string) (steps, violations int, err error) {
|
||||
file, err := os.Open(tracePath)
|
||||
if err != nil {
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return 0, 0, nil
|
||||
}
|
||||
return 0, 0, fmt.Errorf("open trace: %w", err)
|
||||
}
|
||||
defer file.Close()
|
||||
scanner := bufio.NewScanner(file)
|
||||
scanner.Buffer(make([]byte, 64*1024), maxScanTokenSize)
|
||||
for scanner.Scan() {
|
||||
line := scanner.Bytes()
|
||||
if len(line) == 0 {
|
||||
continue
|
||||
}
|
||||
var partial struct {
|
||||
Violations []string `json:"violations,omitempty"`
|
||||
}
|
||||
if err := json.Unmarshal(line, &partial); err != nil {
|
||||
return 0, 0, fmt.Errorf("decode step: %w", err)
|
||||
}
|
||||
steps++
|
||||
violations += len(partial.Violations)
|
||||
}
|
||||
if err := scanner.Err(); err != nil {
|
||||
return 0, 0, fmt.Errorf("scan trace: %w", err)
|
||||
}
|
||||
return steps, violations, nil
|
||||
}
|
||||
|
||||
// Cache holds parsed Run records keyed by id. Open returns a fresh parse
|
||||
// when the underlying trace.jsonl mtime changes.
|
||||
type Cache struct {
|
||||
root string
|
||||
mutex sync.Mutex
|
||||
runs map[string]*Run
|
||||
}
|
||||
|
||||
func NewCache(runsDirectory string) *Cache {
|
||||
return &Cache{root: runsDirectory, runs: map[string]*Run{}}
|
||||
}
|
||||
|
||||
func (c *Cache) Root() string { return c.root }
|
||||
|
||||
// Open parses (or returns a cached parse of) the run named id.
|
||||
func (c *Cache) Open(id string) (*Run, error) {
|
||||
if !validRunID(id) {
|
||||
return nil, fs.ErrNotExist
|
||||
}
|
||||
runDirectory := filepath.Join(c.root, id)
|
||||
tracePath := filepath.Join(runDirectory, "trace.jsonl")
|
||||
traceInfo, traceErr := os.Stat(tracePath)
|
||||
|
||||
c.mutex.Lock()
|
||||
defer c.mutex.Unlock()
|
||||
if cached, ok := c.runs[id]; ok {
|
||||
if traceErr == nil && cached.traceMtime.Equal(traceInfo.ModTime()) {
|
||||
return cached, nil
|
||||
}
|
||||
}
|
||||
run, err := parseRun(runDirectory, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c.runs[id] = run
|
||||
return run, nil
|
||||
}
|
||||
|
||||
func parseRun(runDirectory, id string) (*Run, error) {
|
||||
meta, err := readMeta(runDirectory)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tracePath := filepath.Join(runDirectory, "trace.jsonl")
|
||||
steps, offsets, violationCount, traceMtime, err := scanSteps(tracePath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
summary := buildSummary(id, meta, len(steps), violationCount)
|
||||
return &Run{
|
||||
ID: id,
|
||||
Directory: runDirectory,
|
||||
Meta: meta,
|
||||
Summary: summary,
|
||||
Steps: steps,
|
||||
tracePath: tracePath,
|
||||
traceMtime: traceMtime,
|
||||
offsets: offsets,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func scanSteps(tracePath string) ([]StepSummary, []int64, int, time.Time, error) {
|
||||
file, err := os.Open(tracePath)
|
||||
if err != nil {
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
return []StepSummary{}, nil, 0, time.Time{}, nil
|
||||
}
|
||||
return nil, nil, 0, time.Time{}, fmt.Errorf("open trace: %w", err)
|
||||
}
|
||||
defer file.Close()
|
||||
info, err := file.Stat()
|
||||
if err != nil {
|
||||
return nil, nil, 0, time.Time{}, fmt.Errorf("stat trace: %w", err)
|
||||
}
|
||||
reader := bufio.NewReaderSize(file, 64*1024)
|
||||
steps := []StepSummary{}
|
||||
offsets := []int64{}
|
||||
violationCount := 0
|
||||
var offset int64
|
||||
for {
|
||||
lineStart := offset
|
||||
line, err := reader.ReadBytes('\n')
|
||||
offset += int64(len(line))
|
||||
trimmed := line
|
||||
if len(trimmed) > 0 && trimmed[len(trimmed)-1] == '\n' {
|
||||
trimmed = trimmed[:len(trimmed)-1]
|
||||
}
|
||||
if len(trimmed) > 0 {
|
||||
summary, partial, decodeErr := decodeStepSummary(trimmed)
|
||||
if decodeErr != nil {
|
||||
return nil, nil, 0, time.Time{}, decodeErr
|
||||
}
|
||||
steps = append(steps, summary)
|
||||
offsets = append(offsets, lineStart)
|
||||
violationCount += partial
|
||||
}
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
}
|
||||
return steps, offsets, violationCount, info.ModTime(), nil
|
||||
}
|
||||
|
||||
func decodeStepSummary(line []byte) (StepSummary, int, error) {
|
||||
var partial struct {
|
||||
Index int `json:"step"`
|
||||
Timestamp time.Time `json:"timestamp"`
|
||||
Screen string `json:"screen,omitempty"`
|
||||
Action *struct {
|
||||
Kind string `json:"kind"`
|
||||
X int `json:"x,omitempty"`
|
||||
Y int `json:"y,omitempty"`
|
||||
FromX int `json:"from_x,omitempty"`
|
||||
FromY int `json:"from_y,omitempty"`
|
||||
ToX int `json:"to_x,omitempty"`
|
||||
ToY int `json:"to_y,omitempty"`
|
||||
Key string `json:"key,omitempty"`
|
||||
Text string `json:"text,omitempty"`
|
||||
Selector string `json:"selector,omitempty"`
|
||||
DurationMillis int `json:"duration_millis,omitempty"`
|
||||
} `json:"action,omitempty"`
|
||||
Exceptions []json.RawMessage `json:"exceptions,omitempty"`
|
||||
Violations []string `json:"violations,omitempty"`
|
||||
}
|
||||
if err := json.Unmarshal(line, &partial); err != nil {
|
||||
return StepSummary{}, 0, fmt.Errorf("decode step: %w", err)
|
||||
}
|
||||
summary := StepSummary{
|
||||
Index: partial.Index,
|
||||
Timestamp: partial.Timestamp,
|
||||
Screen: partial.Screen,
|
||||
HasViolations: len(partial.Violations) > 0,
|
||||
HasExceptions: len(partial.Exceptions) > 0,
|
||||
}
|
||||
if partial.Action != nil {
|
||||
summary.ActionKind = partial.Action.Kind
|
||||
switch partial.Action.Kind {
|
||||
case "Tap":
|
||||
if partial.Action.Selector != "" {
|
||||
summary.ActionLabel = partial.Action.Selector
|
||||
} else if partial.Action.Text != "" {
|
||||
summary.ActionLabel = partial.Action.Text
|
||||
} else if partial.Action.X != 0 || partial.Action.Y != 0 {
|
||||
summary.ActionLabel = fmt.Sprintf("(%d,%d)", partial.Action.X, partial.Action.Y)
|
||||
}
|
||||
case "InputText":
|
||||
summary.ActionLabel = fmt.Sprintf("%q", partial.Action.Text)
|
||||
case "Swipe":
|
||||
summary.ActionLabel = swipeDirectionLabel(
|
||||
partial.Action.FromX, partial.Action.FromY,
|
||||
partial.Action.ToX, partial.Action.ToY,
|
||||
)
|
||||
case "PressKey":
|
||||
summary.ActionLabel = partial.Action.Key
|
||||
case "Wait":
|
||||
if partial.Action.DurationMillis > 0 {
|
||||
summary.ActionLabel = fmt.Sprintf("%dms", partial.Action.DurationMillis)
|
||||
}
|
||||
}
|
||||
}
|
||||
return summary, len(partial.Violations), nil
|
||||
}
|
||||
|
||||
func swipeDirectionLabel(fromX, fromY, toX, toY int) string {
|
||||
dx := toX - fromX
|
||||
dy := toY - fromY
|
||||
absX := dx
|
||||
if absX < 0 {
|
||||
absX = -absX
|
||||
}
|
||||
absY := dy
|
||||
if absY < 0 {
|
||||
absY = -absY
|
||||
}
|
||||
if absY >= absX {
|
||||
if dy < 0 {
|
||||
return "up"
|
||||
}
|
||||
return "down"
|
||||
}
|
||||
if dx < 0 {
|
||||
return "left"
|
||||
}
|
||||
return "right"
|
||||
}
|
||||
|
||||
// Step decodes the full Step record at index n (1-based, matching trace.Step.Index).
|
||||
func (c *Cache) Step(run *Run, index int) (trace.Step, error) {
|
||||
position := -1
|
||||
for i, summary := range run.Steps {
|
||||
if summary.Index == index {
|
||||
position = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if position == -1 {
|
||||
return trace.Step{}, fs.ErrNotExist
|
||||
}
|
||||
file, err := os.Open(run.tracePath)
|
||||
if err != nil {
|
||||
return trace.Step{}, fmt.Errorf("open trace: %w", err)
|
||||
}
|
||||
defer file.Close()
|
||||
if _, err := file.Seek(run.offsets[position], 0); err != nil {
|
||||
return trace.Step{}, fmt.Errorf("seek trace: %w", err)
|
||||
}
|
||||
reader := bufio.NewReaderSize(file, 64*1024)
|
||||
line, err := reader.ReadBytes('\n')
|
||||
if err != nil && len(line) == 0 {
|
||||
return trace.Step{}, fmt.Errorf("read step line: %w", err)
|
||||
}
|
||||
if len(line) > 0 && line[len(line)-1] == '\n' {
|
||||
line = line[:len(line)-1]
|
||||
}
|
||||
var step trace.Step
|
||||
if err := json.Unmarshal(line, &step); err != nil {
|
||||
return trace.Step{}, fmt.Errorf("decode step: %w", err)
|
||||
}
|
||||
return step, nil
|
||||
}
|
||||
|
||||
// Detail returns the /api/runs/{id} payload.
|
||||
func (c *Cache) Detail(id string) (RunDetail, error) {
|
||||
run, err := c.Open(id)
|
||||
if err != nil {
|
||||
return RunDetail{}, err
|
||||
}
|
||||
return RunDetail{
|
||||
RunSummary: run.Summary,
|
||||
Meta: run.Meta,
|
||||
Steps: run.Steps,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func validRunID(id string) bool {
|
||||
if id == "" || id == "." || id == ".." {
|
||||
return false
|
||||
}
|
||||
for _, r := range id {
|
||||
switch {
|
||||
case r >= 'a' && r <= 'z':
|
||||
case r >= 'A' && r <= 'Z':
|
||||
case r >= '0' && r <= '9':
|
||||
case r == '-' || r == '_' || r == '.':
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
@@ -0,0 +1,218 @@
|
||||
package inspect
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/priyanshujain/uatu/internal/trace"
|
||||
)
|
||||
|
||||
func TestScan_OrdersByStartedAtDescendingAndCountsViolations(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
older := time.Date(2026, 4, 17, 10, 0, 0, 0, time.UTC)
|
||||
newer := time.Date(2026, 4, 17, 12, 0, 0, 0, time.UTC)
|
||||
writeRun(t, root, "older", trace.Meta{StartedAt: older, EndedAt: timePointer(older.Add(2 * time.Second))}, []trace.Step{
|
||||
{Index: 1, Timestamp: older, Violations: []string{"propA"}},
|
||||
{Index: 2, Timestamp: older.Add(time.Second)},
|
||||
})
|
||||
writeRun(t, root, "newer", trace.Meta{StartedAt: newer, EndedAt: timePointer(newer.Add(time.Second))}, []trace.Step{
|
||||
{Index: 1, Timestamp: newer, Violations: []string{"propA", "propB"}},
|
||||
})
|
||||
|
||||
summaries, err := Scan(root)
|
||||
if err != nil {
|
||||
t.Fatalf("Scan: %v", err)
|
||||
}
|
||||
if len(summaries) != 2 {
|
||||
t.Fatalf("len(summaries) = %d, want 2", len(summaries))
|
||||
}
|
||||
if summaries[0].ID != "newer" {
|
||||
t.Errorf("first id = %q, want newer", summaries[0].ID)
|
||||
}
|
||||
if summaries[0].ViolationCount != 2 {
|
||||
t.Errorf("newer violations = %d, want 2", summaries[0].ViolationCount)
|
||||
}
|
||||
if summaries[1].ViolationCount != 1 {
|
||||
t.Errorf("older violations = %d, want 1", summaries[1].ViolationCount)
|
||||
}
|
||||
if summaries[1].StepCount != 2 {
|
||||
t.Errorf("older steps = %d, want 2", summaries[1].StepCount)
|
||||
}
|
||||
if summaries[0].DurationMillis != 1000 {
|
||||
t.Errorf("newer duration = %d, want 1000", summaries[0].DurationMillis)
|
||||
}
|
||||
}
|
||||
|
||||
func TestScan_MissingDirectoryReturnsEmpty(t *testing.T) {
|
||||
summaries, err := Scan(filepath.Join(t.TempDir(), "does-not-exist"))
|
||||
if err != nil {
|
||||
t.Fatalf("Scan: %v", err)
|
||||
}
|
||||
if len(summaries) != 0 {
|
||||
t.Errorf("len(summaries) = %d, want 0", len(summaries))
|
||||
}
|
||||
}
|
||||
|
||||
func TestScan_MissingEndedAtSurfacesInProgress(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
writeRun(t, root, "live", trace.Meta{StartedAt: time.Now().UTC()}, []trace.Step{
|
||||
{Index: 1, Timestamp: time.Now().UTC()},
|
||||
})
|
||||
summaries, err := Scan(root)
|
||||
if err != nil {
|
||||
t.Fatalf("Scan: %v", err)
|
||||
}
|
||||
if len(summaries) != 1 || !summaries[0].InProgress {
|
||||
t.Errorf("expected in_progress=true, got %+v", summaries)
|
||||
}
|
||||
if summaries[0].EndedAt != nil {
|
||||
t.Errorf("ended_at should be nil, got %v", summaries[0].EndedAt)
|
||||
}
|
||||
}
|
||||
|
||||
func TestScan_EmptyTraceTreatedAsZeroSteps(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
directory := filepath.Join(root, "empty")
|
||||
if err := os.MkdirAll(directory, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
meta := trace.Meta{StartedAt: time.Now().UTC()}
|
||||
body, _ := json.Marshal(meta)
|
||||
if err := os.WriteFile(filepath.Join(directory, "meta.json"), body, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
summaries, err := Scan(root)
|
||||
if err != nil {
|
||||
t.Fatalf("Scan: %v", err)
|
||||
}
|
||||
if len(summaries) != 1 {
|
||||
t.Fatalf("len = %d", len(summaries))
|
||||
}
|
||||
if summaries[0].StepCount != 0 {
|
||||
t.Errorf("step_count = %d, want 0", summaries[0].StepCount)
|
||||
}
|
||||
if !summaries[0].InProgress {
|
||||
t.Error("expected in_progress")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCacheStep_LazyDecodeReturnsFullStep(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
startedAt := time.Now().UTC()
|
||||
steps := []trace.Step{
|
||||
{Index: 1, Timestamp: startedAt, Screen: "A"},
|
||||
{Index: 2, Timestamp: startedAt.Add(time.Second), Screen: "B", Action: &trace.Action{Kind: "tap"}},
|
||||
{Index: 3, Timestamp: startedAt.Add(2 * time.Second), Screen: "C", Violations: []string{"prop1"}},
|
||||
}
|
||||
writeRun(t, root, "r1", trace.Meta{StartedAt: startedAt, EndedAt: timePointer(startedAt.Add(3 * time.Second))}, steps)
|
||||
|
||||
cache := NewCache(root)
|
||||
run, err := cache.Open("r1")
|
||||
if err != nil {
|
||||
t.Fatalf("Open: %v", err)
|
||||
}
|
||||
if len(run.Steps) != 3 {
|
||||
t.Fatalf("steps = %d, want 3", len(run.Steps))
|
||||
}
|
||||
if !run.Steps[2].HasViolations {
|
||||
t.Error("step 3 should HasViolations")
|
||||
}
|
||||
if run.Steps[1].ActionKind != "tap" {
|
||||
t.Errorf("step 2 action = %q, want tap", run.Steps[1].ActionKind)
|
||||
}
|
||||
|
||||
for _, target := range []int{1, 2, 3} {
|
||||
step, err := cache.Step(run, target)
|
||||
if err != nil {
|
||||
t.Fatalf("Step(%d): %v", target, err)
|
||||
}
|
||||
if step.Index != target {
|
||||
t.Errorf("Step(%d).Index = %d", target, step.Index)
|
||||
}
|
||||
}
|
||||
if _, err := cache.Step(run, 99); err == nil {
|
||||
t.Error("expected error for out-of-range step")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecodeStepSummary_ActionLabelPerKind(t *testing.T) {
|
||||
cases := []struct {
|
||||
line string
|
||||
wantKind string
|
||||
wantLabel string
|
||||
}{
|
||||
{`{"step":1,"timestamp":"2026-04-20T10:00:00Z","action":{"kind":"Tap","selector":"id:save"}}`, "Tap", "id:save"},
|
||||
{`{"step":2,"timestamp":"2026-04-20T10:00:01Z","action":{"kind":"Tap","x":140,"y":220}}`, "Tap", "(140,220)"},
|
||||
{`{"step":3,"timestamp":"2026-04-20T10:00:02Z","action":{"kind":"InputText","text":"alice"}}`, "InputText", `"alice"`},
|
||||
{`{"step":4,"timestamp":"2026-04-20T10:00:03Z","action":{"kind":"Swipe","from_x":10,"from_y":500,"to_x":10,"to_y":50}}`, "Swipe", "up"},
|
||||
{`{"step":5,"timestamp":"2026-04-20T10:00:04Z","action":{"kind":"Swipe","from_x":100,"from_y":50,"to_x":600,"to_y":50}}`, "Swipe", "right"},
|
||||
{`{"step":6,"timestamp":"2026-04-20T10:00:05Z","action":{"kind":"PressKey","key":"back"}}`, "PressKey", "back"},
|
||||
{`{"step":7,"timestamp":"2026-04-20T10:00:06Z","action":{"kind":"Wait","duration_millis":500}}`, "Wait", "500ms"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
summary, _, err := decodeStepSummary([]byte(tc.line))
|
||||
if err != nil {
|
||||
t.Fatalf("decode %s: %v", tc.line, err)
|
||||
}
|
||||
if summary.ActionKind != tc.wantKind {
|
||||
t.Errorf("kind = %q, want %q (line=%s)", summary.ActionKind, tc.wantKind, tc.line)
|
||||
}
|
||||
if summary.ActionLabel != tc.wantLabel {
|
||||
t.Errorf("label = %q, want %q (line=%s)", summary.ActionLabel, tc.wantLabel, tc.line)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCacheOpen_RejectsTraversalIDs(t *testing.T) {
|
||||
cache := NewCache(t.TempDir())
|
||||
for _, id := range []string{"", ".", "..", "../etc", "a/b", "a\\b"} {
|
||||
if _, err := cache.Open(id); err == nil {
|
||||
t.Errorf("Open(%q) should fail", id)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsRunDirectory(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
writeRun(t, root, "x", trace.Meta{StartedAt: time.Now().UTC()}, nil)
|
||||
if !IsRunDirectory(filepath.Join(root, "x")) {
|
||||
t.Error("expected true for run dir")
|
||||
}
|
||||
if IsRunDirectory(root) {
|
||||
t.Error("expected false for parent dir")
|
||||
}
|
||||
}
|
||||
|
||||
func writeRun(t *testing.T, root, id string, meta trace.Meta, steps []trace.Step) {
|
||||
t.Helper()
|
||||
directory := filepath.Join(root, id)
|
||||
if err := os.MkdirAll(directory, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
metaBody, err := json.MarshalIndent(meta, "", " ")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(directory, "meta.json"), metaBody, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if steps == nil {
|
||||
return
|
||||
}
|
||||
file, err := os.Create(filepath.Join(directory, "trace.jsonl"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer file.Close()
|
||||
encoder := json.NewEncoder(file)
|
||||
for _, step := range steps {
|
||||
if err := encoder.Encode(step); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func timePointer(t time.Time) *time.Time { return &t }
|
||||
@@ -0,0 +1,281 @@
|
||||
package inspect
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"net/http"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// ServerOptions configures a new Server.
|
||||
type ServerOptions struct {
|
||||
RunsDirectory string
|
||||
DevTarget string
|
||||
}
|
||||
|
||||
// Server holds the HTTP handlers for `uatu inspect`.
|
||||
type Server struct {
|
||||
options ServerOptions
|
||||
cache *Cache
|
||||
watcher *Watcher
|
||||
assets http.Handler
|
||||
dev http.Handler
|
||||
}
|
||||
|
||||
// NewServer constructs a Server. When options.DevTarget is non-empty the
|
||||
// server reverse-proxies non-API GETs to it; otherwise it serves embedded
|
||||
// assets from the dist FS.
|
||||
func NewServer(options ServerOptions) (*Server, error) {
|
||||
server := &Server{
|
||||
options: options,
|
||||
cache: NewCache(options.RunsDirectory),
|
||||
watcher: NewWatcher(options.RunsDirectory),
|
||||
assets: spaHandler(Assets()),
|
||||
}
|
||||
if options.DevTarget != "" {
|
||||
proxy, err := newDevProxy(options.DevTarget)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("dev proxy: %w", err)
|
||||
}
|
||||
server.dev = proxy
|
||||
}
|
||||
return server, nil
|
||||
}
|
||||
|
||||
// Watcher exposes the runs-directory watcher so callers can run it under their
|
||||
// own context.
|
||||
func (s *Server) Watcher() *Watcher { return s.watcher }
|
||||
|
||||
// Handler returns the root HTTP handler.
|
||||
func (s *Server) Handler() http.Handler {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/api/runs", s.handleRunsList)
|
||||
mux.HandleFunc("/api/runs/", s.handleRunsTree)
|
||||
mux.HandleFunc("/api/events", s.handleEvents)
|
||||
mux.HandleFunc("/", s.handleAssets)
|
||||
return mux
|
||||
}
|
||||
|
||||
func (s *Server) handleRunsList(responseWriter http.ResponseWriter, request *http.Request) {
|
||||
if request.Method != http.MethodGet {
|
||||
http.Error(responseWriter, "method not allowed", http.StatusMethodNotAllowed)
|
||||
return
|
||||
}
|
||||
summaries, err := Scan(s.options.RunsDirectory)
|
||||
if err != nil {
|
||||
http.Error(responseWriter, err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
writeJSON(responseWriter, http.StatusOK, summaries)
|
||||
}
|
||||
|
||||
var stepPathPattern = regexp.MustCompile(`^([a-zA-Z0-9._-]+)/steps/([^/]+)$`)
|
||||
var screenshotPathPattern = regexp.MustCompile(`^([a-zA-Z0-9._-]+)/screenshots/([a-zA-Z0-9._-]+\.png)$`)
|
||||
var runDetailPathPattern = regexp.MustCompile(`^([a-zA-Z0-9._-]+)/?$`)
|
||||
|
||||
func (s *Server) handleRunsTree(responseWriter http.ResponseWriter, request *http.Request) {
|
||||
if request.Method != http.MethodGet {
|
||||
http.Error(responseWriter, "method not allowed", http.StatusMethodNotAllowed)
|
||||
return
|
||||
}
|
||||
rest := strings.TrimPrefix(request.URL.Path, "/api/runs/")
|
||||
if rest == "" {
|
||||
s.handleRunsList(responseWriter, request)
|
||||
return
|
||||
}
|
||||
if match := stepPathPattern.FindStringSubmatch(rest); match != nil {
|
||||
s.serveStep(responseWriter, match[1], match[2])
|
||||
return
|
||||
}
|
||||
if match := screenshotPathPattern.FindStringSubmatch(rest); match != nil {
|
||||
s.serveScreenshot(responseWriter, request, match[1], match[2])
|
||||
return
|
||||
}
|
||||
if match := runDetailPathPattern.FindStringSubmatch(rest); match != nil {
|
||||
s.serveDetail(responseWriter, match[1])
|
||||
return
|
||||
}
|
||||
http.NotFound(responseWriter, request)
|
||||
}
|
||||
|
||||
func (s *Server) serveDetail(responseWriter http.ResponseWriter, id string) {
|
||||
detail, err := s.cache.Detail(id)
|
||||
if err != nil {
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
http.Error(responseWriter, "run not found", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
http.Error(responseWriter, err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
writeJSON(responseWriter, http.StatusOK, detail)
|
||||
}
|
||||
|
||||
func (s *Server) serveStep(responseWriter http.ResponseWriter, id, indexText string) {
|
||||
index, err := strconv.Atoi(indexText)
|
||||
if err != nil {
|
||||
http.Error(responseWriter, "step index must be numeric", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
run, err := s.cache.Open(id)
|
||||
if err != nil {
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
http.Error(responseWriter, "run not found", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
http.Error(responseWriter, err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
step, err := s.cache.Step(run, index)
|
||||
if err != nil {
|
||||
if errors.Is(err, fs.ErrNotExist) {
|
||||
http.Error(responseWriter, "step not found", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
http.Error(responseWriter, err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
writeJSON(responseWriter, http.StatusOK, step)
|
||||
}
|
||||
|
||||
func (s *Server) serveScreenshot(responseWriter http.ResponseWriter, request *http.Request, id, name string) {
|
||||
if !validRunID(id) {
|
||||
http.Error(responseWriter, "run not found", http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
full := filepath.Join(s.options.RunsDirectory, id, "screenshots", name)
|
||||
http.ServeFile(responseWriter, request, full)
|
||||
}
|
||||
|
||||
func (s *Server) handleEvents(responseWriter http.ResponseWriter, request *http.Request) {
|
||||
flusher, ok := responseWriter.(http.Flusher)
|
||||
if !ok {
|
||||
http.Error(responseWriter, "streaming unsupported", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
responseWriter.Header().Set("Content-Type", "text/event-stream")
|
||||
responseWriter.Header().Set("Cache-Control", "no-cache")
|
||||
responseWriter.Header().Set("Connection", "keep-alive")
|
||||
responseWriter.WriteHeader(http.StatusOK)
|
||||
flusher.Flush()
|
||||
|
||||
subscription := s.watcher.Subscribe()
|
||||
defer s.watcher.Unsubscribe(subscription)
|
||||
heartbeat := time.NewTicker(15 * time.Second)
|
||||
defer heartbeat.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-request.Context().Done():
|
||||
return
|
||||
case _, ok := <-subscription:
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
fmt.Fprint(responseWriter, "event: runs.changed\ndata: {\"type\":\"runs.changed\"}\n\n")
|
||||
flusher.Flush()
|
||||
case <-heartbeat.C:
|
||||
fmt.Fprint(responseWriter, ": ping\n\n")
|
||||
flusher.Flush()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) handleAssets(responseWriter http.ResponseWriter, request *http.Request) {
|
||||
if strings.HasPrefix(request.URL.Path, "/api/") {
|
||||
http.NotFound(responseWriter, request)
|
||||
return
|
||||
}
|
||||
if s.dev != nil {
|
||||
s.dev.ServeHTTP(responseWriter, request)
|
||||
return
|
||||
}
|
||||
s.assets.ServeHTTP(responseWriter, request)
|
||||
}
|
||||
|
||||
// spaHandler serves files from assets, falling back to index.html for
|
||||
// unknown paths so the SPA router can take over.
|
||||
func spaHandler(assets fs.FS) http.Handler {
|
||||
fileServer := http.FileServer(http.FS(assets))
|
||||
return http.HandlerFunc(func(responseWriter http.ResponseWriter, request *http.Request) {
|
||||
clean := strings.TrimPrefix(path.Clean(request.URL.Path), "/")
|
||||
if clean == "" {
|
||||
serveIndex(responseWriter, request, assets)
|
||||
return
|
||||
}
|
||||
file, err := assets.Open(clean)
|
||||
if err != nil {
|
||||
serveIndex(responseWriter, request, assets)
|
||||
return
|
||||
}
|
||||
file.Close()
|
||||
fileServer.ServeHTTP(responseWriter, request)
|
||||
})
|
||||
}
|
||||
|
||||
func serveIndex(responseWriter http.ResponseWriter, request *http.Request, assets fs.FS) {
|
||||
file, err := assets.Open("index.html")
|
||||
if err != nil {
|
||||
http.Error(responseWriter, "index.html missing from embedded assets", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
defer file.Close()
|
||||
body, err := readAll(file)
|
||||
if err != nil {
|
||||
http.Error(responseWriter, err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
responseWriter.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
_, _ = responseWriter.Write(body)
|
||||
}
|
||||
|
||||
func readAll(file fs.File) ([]byte, error) {
|
||||
const initialCapacity = 4 * 1024
|
||||
buffer := make([]byte, 0, initialCapacity)
|
||||
chunk := make([]byte, 4*1024)
|
||||
for {
|
||||
read, err := file.Read(chunk)
|
||||
if read > 0 {
|
||||
buffer = append(buffer, chunk[:read]...)
|
||||
}
|
||||
if err != nil {
|
||||
if errors.Is(err, fs.ErrInvalid) {
|
||||
return nil, err
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
return buffer, nil
|
||||
}
|
||||
|
||||
func writeJSON(responseWriter http.ResponseWriter, status int, payload any) {
|
||||
responseWriter.Header().Set("Content-Type", "application/json")
|
||||
responseWriter.WriteHeader(status)
|
||||
encoder := json.NewEncoder(responseWriter)
|
||||
_ = encoder.Encode(payload)
|
||||
}
|
||||
|
||||
// ResolveRunsDirectory takes the optional positional argument and returns
|
||||
// (runsDirectory, deepLinkID, error). When argument is "" it falls back
|
||||
// to ./runs. When argument is a single run directory (has meta.json), the
|
||||
// parent becomes runsDirectory and the basename becomes the deep-link id.
|
||||
func ResolveRunsDirectory(argument string) (string, string, error) {
|
||||
if argument == "" {
|
||||
return "./runs", "", nil
|
||||
}
|
||||
if IsRunDirectory(argument) {
|
||||
cleaned := filepath.Clean(argument)
|
||||
parent := filepath.Dir(cleaned)
|
||||
base := filepath.Base(cleaned)
|
||||
return parent, base, nil
|
||||
}
|
||||
return argument, "", nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,315 @@
|
||||
package inspect
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/priyanshujain/uatu/internal/trace"
|
||||
)
|
||||
|
||||
func newFixtureServer(t *testing.T) (*Server, string) {
|
||||
t.Helper()
|
||||
root := t.TempDir()
|
||||
startedAt := time.Date(2026, 4, 17, 18, 0, 0, 0, time.UTC)
|
||||
endedAt := startedAt.Add(5 * time.Second)
|
||||
writeRun(t, root, "run-a", trace.Meta{
|
||||
StartedAt: startedAt,
|
||||
EndedAt: &endedAt,
|
||||
SpecPath: "spec.ts",
|
||||
Seed: 42,
|
||||
Platform: "android",
|
||||
BundleID: "com.example",
|
||||
}, []trace.Step{
|
||||
{Index: 1, Timestamp: startedAt, Screen: "Home"},
|
||||
{Index: 2, Timestamp: startedAt.Add(time.Second), Screen: "Home", Action: &trace.Action{Kind: "tap"}},
|
||||
{Index: 3, Timestamp: startedAt.Add(2 * time.Second), Screen: "Cart", Violations: []string{"propA"}},
|
||||
})
|
||||
writeRun(t, root, "run-b", trace.Meta{
|
||||
StartedAt: startedAt.Add(time.Hour),
|
||||
}, []trace.Step{
|
||||
{Index: 1, Timestamp: startedAt.Add(time.Hour)},
|
||||
})
|
||||
|
||||
screenshotsDirectory := filepath.Join(root, "run-a", "screenshots")
|
||||
if err := os.MkdirAll(screenshotsDirectory, 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
pngBody := []byte{0x89, 'P', 'N', 'G', 0, 1, 2, 3}
|
||||
if err := os.WriteFile(filepath.Join(screenshotsDirectory, "step-00001.png"), pngBody, 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
server, err := NewServer(ServerOptions{RunsDirectory: root})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return server, root
|
||||
}
|
||||
|
||||
func TestHandleRunsList_OrdersAndCountsViolations(t *testing.T) {
|
||||
server, _ := newFixtureServer(t)
|
||||
recorder := httptest.NewRecorder()
|
||||
request := httptest.NewRequest(http.MethodGet, "/api/runs", nil)
|
||||
server.Handler().ServeHTTP(recorder, request)
|
||||
if recorder.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d", recorder.Code)
|
||||
}
|
||||
var summaries []RunSummary
|
||||
if err := json.Unmarshal(recorder.Body.Bytes(), &summaries); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(summaries) != 2 {
|
||||
t.Fatalf("len = %d", len(summaries))
|
||||
}
|
||||
if summaries[0].ID != "run-b" {
|
||||
t.Errorf("first id = %q, want run-b (most recent)", summaries[0].ID)
|
||||
}
|
||||
if summaries[1].ViolationCount != 1 {
|
||||
t.Errorf("run-a violation count = %d, want 1", summaries[1].ViolationCount)
|
||||
}
|
||||
if !summaries[0].InProgress {
|
||||
t.Error("run-b should be in_progress (no ended_at)")
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleRunDetail_DecodesMetaAndStepSummaries(t *testing.T) {
|
||||
server, _ := newFixtureServer(t)
|
||||
recorder := httptest.NewRecorder()
|
||||
request := httptest.NewRequest(http.MethodGet, "/api/runs/run-a", nil)
|
||||
server.Handler().ServeHTTP(recorder, request)
|
||||
if recorder.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, body=%s", recorder.Code, recorder.Body.String())
|
||||
}
|
||||
var detail RunDetail
|
||||
if err := json.Unmarshal(recorder.Body.Bytes(), &detail); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if detail.Meta.BundleID != "com.example" {
|
||||
t.Errorf("bundle = %q", detail.Meta.BundleID)
|
||||
}
|
||||
if len(detail.Steps) != 3 {
|
||||
t.Fatalf("steps len = %d", len(detail.Steps))
|
||||
}
|
||||
if detail.Steps[1].ActionKind != "tap" {
|
||||
t.Errorf("step 2 action = %q", detail.Steps[1].ActionKind)
|
||||
}
|
||||
if !detail.Steps[2].HasViolations {
|
||||
t.Error("step 3 should HasViolations")
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleStep_ReturnsFullStep(t *testing.T) {
|
||||
server, _ := newFixtureServer(t)
|
||||
recorder := httptest.NewRecorder()
|
||||
request := httptest.NewRequest(http.MethodGet, "/api/runs/run-a/steps/3", nil)
|
||||
server.Handler().ServeHTTP(recorder, request)
|
||||
if recorder.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d", recorder.Code)
|
||||
}
|
||||
var step trace.Step
|
||||
if err := json.Unmarshal(recorder.Body.Bytes(), &step); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if step.Index != 3 || step.Screen != "Cart" {
|
||||
t.Errorf("step = %+v", step)
|
||||
}
|
||||
if len(step.Violations) != 1 {
|
||||
t.Errorf("violations = %v", step.Violations)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleStep_ErrorCases(t *testing.T) {
|
||||
server, _ := newFixtureServer(t)
|
||||
cases := []struct {
|
||||
name string
|
||||
path string
|
||||
wantStatus int
|
||||
}{
|
||||
{"unknown run", "/api/runs/nope", http.StatusNotFound},
|
||||
{"non-numeric step", "/api/runs/run-a/steps/abc", http.StatusBadRequest},
|
||||
{"out-of-range step", "/api/runs/run-a/steps/999", http.StatusNotFound},
|
||||
}
|
||||
for _, testCase := range cases {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
recorder := httptest.NewRecorder()
|
||||
request := httptest.NewRequest(http.MethodGet, testCase.path, nil)
|
||||
server.Handler().ServeHTTP(recorder, request)
|
||||
if recorder.Code != testCase.wantStatus {
|
||||
t.Errorf("status = %d, want %d, body=%s", recorder.Code, testCase.wantStatus, recorder.Body.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestScreenshot_ServesWhitelistedPNG(t *testing.T) {
|
||||
server, _ := newFixtureServer(t)
|
||||
recorder := httptest.NewRecorder()
|
||||
request := httptest.NewRequest(http.MethodGet, "/api/runs/run-a/screenshots/step-00001.png", nil)
|
||||
server.Handler().ServeHTTP(recorder, request)
|
||||
if recorder.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d", recorder.Code)
|
||||
}
|
||||
if recorder.Body.Len() == 0 {
|
||||
t.Error("empty body")
|
||||
}
|
||||
}
|
||||
|
||||
func TestScreenshot_RejectsTraversalAndBadNames(t *testing.T) {
|
||||
server, _ := newFixtureServer(t)
|
||||
cases := []string{
|
||||
"/api/runs/run-a/screenshots/../meta.json",
|
||||
"/api/runs/run-a/screenshots/..%2Fmeta.json",
|
||||
"/api/runs/run-a/screenshots/step-00001.txt",
|
||||
"/api/runs/run-a/screenshots/.png",
|
||||
}
|
||||
for _, path := range cases {
|
||||
t.Run(path, func(t *testing.T) {
|
||||
recorder := httptest.NewRecorder()
|
||||
request := httptest.NewRequest(http.MethodGet, path, nil)
|
||||
server.Handler().ServeHTTP(recorder, request)
|
||||
if recorder.Code == http.StatusOK {
|
||||
t.Errorf("status = %d (should not be 200) body=%s", recorder.Code, recorder.Body.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSSE_ReturnsWhenContextCanceled(t *testing.T) {
|
||||
server, _ := newFixtureServer(t)
|
||||
httpServer := httptest.NewServer(server.Handler())
|
||||
defer httpServer.Close()
|
||||
|
||||
context, cancel := context.WithCancel(context.Background())
|
||||
request, err := http.NewRequestWithContext(context, http.MethodGet, httpServer.URL+"/api/events", nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
response, err := http.DefaultClient.Do(request)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer response.Body.Close()
|
||||
if response.StatusCode != http.StatusOK {
|
||||
t.Fatalf("status = %d", response.StatusCode)
|
||||
}
|
||||
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
_, _ = io.Copy(io.Discard, response.Body)
|
||||
close(done)
|
||||
}()
|
||||
cancel()
|
||||
select {
|
||||
case <-done:
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Fatal("SSE handler did not exit after context cancel")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDevProxy_ForwardsRequestBodyUnchanged(t *testing.T) {
|
||||
received := make(chan string, 1)
|
||||
upstream := httptest.NewServer(http.HandlerFunc(func(responseWriter http.ResponseWriter, request *http.Request) {
|
||||
body, err := io.ReadAll(request.Body)
|
||||
if err != nil {
|
||||
t.Errorf("upstream read: %v", err)
|
||||
}
|
||||
received <- string(body)
|
||||
responseWriter.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
defer upstream.Close()
|
||||
|
||||
root := t.TempDir()
|
||||
server, err := NewServer(ServerOptions{RunsDirectory: root, DevTarget: upstream.URL})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
httpServer := httptest.NewServer(server.Handler())
|
||||
defer httpServer.Close()
|
||||
|
||||
body := strings.NewReader("hello world")
|
||||
request, err := http.NewRequest(http.MethodPost, httpServer.URL+"/anything", body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
response, err := http.DefaultClient.Do(request)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
response.Body.Close()
|
||||
if response.StatusCode != http.StatusOK {
|
||||
t.Errorf("status = %d", response.StatusCode)
|
||||
}
|
||||
select {
|
||||
case got := <-received:
|
||||
if got != "hello world" {
|
||||
t.Errorf("upstream body = %q, want %q", got, "hello world")
|
||||
}
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Fatal("upstream never received the request")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAssets_FallbackToIndexHTML(t *testing.T) {
|
||||
server, _ := newFixtureServer(t)
|
||||
recorder := httptest.NewRecorder()
|
||||
request := httptest.NewRequest(http.MethodGet, "/runs/some-id-that-only-the-spa-knows", nil)
|
||||
server.Handler().ServeHTTP(recorder, request)
|
||||
if recorder.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d", recorder.Code)
|
||||
}
|
||||
body := recorder.Body.String()
|
||||
if !strings.Contains(body, "<div id=\"app\"></div>") && !strings.Contains(body, "<div id=\"root\"></div>") {
|
||||
t.Errorf("expected SPA shell with #app or #root, got %q", body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAssets_API404DoesNotFallThrough(t *testing.T) {
|
||||
server, _ := newFixtureServer(t)
|
||||
recorder := httptest.NewRecorder()
|
||||
request := httptest.NewRequest(http.MethodGet, "/api/wat", nil)
|
||||
server.Handler().ServeHTTP(recorder, request)
|
||||
if recorder.Code != http.StatusNotFound {
|
||||
t.Errorf("status = %d, want 404", recorder.Code)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveRunsDirectory(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
writeRun(t, root, "r1", trace.Meta{StartedAt: time.Now().UTC()}, nil)
|
||||
|
||||
runsDirectory, deepLink, err := ResolveRunsDirectory("")
|
||||
if err != nil || runsDirectory != "./runs" || deepLink != "" {
|
||||
t.Errorf("default = (%q, %q, %v)", runsDirectory, deepLink, err)
|
||||
}
|
||||
runsDirectory, deepLink, err = ResolveRunsDirectory(root)
|
||||
if err != nil || runsDirectory != root || deepLink != "" {
|
||||
t.Errorf("multi-run dir = (%q, %q, %v)", runsDirectory, deepLink, err)
|
||||
}
|
||||
singleRun := filepath.Join(root, "r1")
|
||||
runsDirectory, deepLink, err = ResolveRunsDirectory(singleRun)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if filepath.Clean(runsDirectory) != filepath.Clean(root) || deepLink != "r1" {
|
||||
t.Errorf("single-run dir = (%q, %q)", runsDirectory, deepLink)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDevProxy_ParsesTarget(t *testing.T) {
|
||||
if _, err := newDevProxy(":://bad-url"); err == nil {
|
||||
t.Error("expected parse error for invalid URL")
|
||||
}
|
||||
parsed, err := url.Parse(DevTarget)
|
||||
if err != nil || parsed.Host != "127.0.0.1:5173" {
|
||||
t.Errorf("DevTarget parsed wrong: %v %q", err, parsed.Host)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,147 @@
|
||||
package inspect
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io/fs"
|
||||
"os"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/fsnotify/fsnotify"
|
||||
)
|
||||
|
||||
const watcherDebounce = 200 * time.Millisecond
|
||||
|
||||
// Watcher reports coalesced runs.changed events from the runs directory.
|
||||
// Subscribe returns a channel that receives one event per debounce window.
|
||||
// The watcher tolerates a missing runs directory by polling for it to appear.
|
||||
type Watcher struct {
|
||||
directory string
|
||||
debounce time.Duration
|
||||
mutex sync.Mutex
|
||||
subscribers []chan struct{}
|
||||
closed bool
|
||||
}
|
||||
|
||||
func NewWatcher(directory string) *Watcher {
|
||||
return &Watcher{directory: directory, debounce: watcherDebounce}
|
||||
}
|
||||
|
||||
func (w *Watcher) Subscribe() <-chan struct{} {
|
||||
w.mutex.Lock()
|
||||
defer w.mutex.Unlock()
|
||||
channel := make(chan struct{}, 4)
|
||||
if w.closed {
|
||||
close(channel)
|
||||
return channel
|
||||
}
|
||||
w.subscribers = append(w.subscribers, channel)
|
||||
return channel
|
||||
}
|
||||
|
||||
// Unsubscribe removes a channel previously returned by Subscribe. The channel
|
||||
// is not closed because broadcast snapshots subscribers without holding the
|
||||
// mutex and a concurrent close would race with its non-blocking send.
|
||||
// Safe to call multiple times; unknown channels are ignored.
|
||||
func (w *Watcher) Unsubscribe(subscription <-chan struct{}) {
|
||||
w.mutex.Lock()
|
||||
defer w.mutex.Unlock()
|
||||
for index, channel := range w.subscribers {
|
||||
if (<-chan struct{})(channel) != subscription {
|
||||
continue
|
||||
}
|
||||
last := len(w.subscribers) - 1
|
||||
w.subscribers[index] = w.subscribers[last]
|
||||
w.subscribers[last] = nil
|
||||
w.subscribers = w.subscribers[:last]
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// Run blocks until ctx is canceled, watching directory for create/remove/rename
|
||||
// events and emitting one notification per debounce window to all subscribers.
|
||||
func (w *Watcher) Run(ctx context.Context) error {
|
||||
watcher, err := fsnotify.NewWatcher()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer watcher.Close()
|
||||
|
||||
if err := watchOrWaitForDirectory(ctx, watcher, w.directory); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
var pending bool
|
||||
timer := time.NewTimer(w.debounce)
|
||||
if !timer.Stop() {
|
||||
<-timer.C
|
||||
}
|
||||
defer timer.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
w.shutdown()
|
||||
return nil
|
||||
case event, ok := <-watcher.Events:
|
||||
if !ok {
|
||||
w.shutdown()
|
||||
return nil
|
||||
}
|
||||
if event.Op&(fsnotify.Create|fsnotify.Remove|fsnotify.Rename) == 0 {
|
||||
continue
|
||||
}
|
||||
if !pending {
|
||||
pending = true
|
||||
timer.Reset(w.debounce)
|
||||
}
|
||||
case <-watcher.Errors:
|
||||
// Drop transient errors; SSE is best-effort.
|
||||
case <-timer.C:
|
||||
if pending {
|
||||
pending = false
|
||||
w.broadcast()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (w *Watcher) broadcast() {
|
||||
w.mutex.Lock()
|
||||
subscribers := append([]chan struct{}(nil), w.subscribers...)
|
||||
w.mutex.Unlock()
|
||||
for _, channel := range subscribers {
|
||||
select {
|
||||
case channel <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (w *Watcher) shutdown() {
|
||||
w.mutex.Lock()
|
||||
defer w.mutex.Unlock()
|
||||
w.closed = true
|
||||
for _, channel := range w.subscribers {
|
||||
close(channel)
|
||||
}
|
||||
w.subscribers = nil
|
||||
}
|
||||
|
||||
func watchOrWaitForDirectory(ctx context.Context, watcher *fsnotify.Watcher, directory string) error {
|
||||
for {
|
||||
err := watcher.Add(directory)
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
if !errors.Is(err, fs.ErrNotExist) && !errors.Is(err, os.ErrNotExist) {
|
||||
return err
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil
|
||||
case <-time.After(time.Second):
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
package inspect
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestWatcher_UnsubscribeRemovesChannel(t *testing.T) {
|
||||
w := NewWatcher(t.TempDir())
|
||||
first := w.Subscribe()
|
||||
second := w.Subscribe()
|
||||
third := w.Subscribe()
|
||||
|
||||
if count := len(w.subscribers); count != 3 {
|
||||
t.Fatalf("expected 3 subscribers, got %d", count)
|
||||
}
|
||||
|
||||
w.Unsubscribe(second)
|
||||
|
||||
if count := len(w.subscribers); count != 2 {
|
||||
t.Fatalf("expected 2 subscribers after Unsubscribe, got %d", count)
|
||||
}
|
||||
|
||||
// broadcast should still notify remaining subscribers
|
||||
w.broadcast()
|
||||
select {
|
||||
case <-first:
|
||||
default:
|
||||
t.Error("first subscriber did not receive broadcast")
|
||||
}
|
||||
select {
|
||||
case <-third:
|
||||
default:
|
||||
t.Error("third subscriber did not receive broadcast")
|
||||
}
|
||||
}
|
||||
|
||||
func TestWatcher_UnsubscribeUnknownChannelIsNoop(t *testing.T) {
|
||||
w := NewWatcher(t.TempDir())
|
||||
existing := w.Subscribe()
|
||||
|
||||
stranger := make(chan struct{})
|
||||
w.Unsubscribe(stranger)
|
||||
|
||||
if count := len(w.subscribers); count != 1 {
|
||||
t.Fatalf("expected 1 subscriber after no-op Unsubscribe, got %d", count)
|
||||
}
|
||||
|
||||
w.broadcast()
|
||||
select {
|
||||
case <-existing:
|
||||
default:
|
||||
t.Error("existing subscriber did not receive broadcast")
|
||||
}
|
||||
}
|
||||
@@ -77,6 +77,26 @@ func (e *Evaluator) ObserveAt(now time.Time) Verdict {
|
||||
return VerdictHolds
|
||||
}
|
||||
|
||||
// Residual returns a single Formula describing what the evaluator still has
|
||||
// to prove after the most recent ObserveAt. PureFormula{true} means the
|
||||
// property holds for the run so far; PureFormula{false} means it has latched
|
||||
// to violated. When obligations are still pending, they are folded together
|
||||
// with AndFormula in the order they were registered so the JSON AST reflects
|
||||
// the same order the evaluator processes them in.
|
||||
func (e *Evaluator) Residual() Formula {
|
||||
if e.violated {
|
||||
return PureFormula{Value: false}
|
||||
}
|
||||
if len(e.pending) == 0 {
|
||||
return PureFormula{Value: true}
|
||||
}
|
||||
combined := e.pending[0]
|
||||
for _, formula := range e.pending[1:] {
|
||||
combined = AndFormula{Left: combined, Right: formula}
|
||||
}
|
||||
return combined
|
||||
}
|
||||
|
||||
// rootObligation returns the formula to instantiate at each step. An outer
|
||||
// Always is stripped so its inner is re-evaluated every step; any other root
|
||||
// formula is itself re-instantiated each step (matching the v0.1 semantics
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package ltl
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
@@ -12,6 +13,26 @@ type Formula interface {
|
||||
describe() string
|
||||
}
|
||||
|
||||
// PredicateLabel lets a ThunkFormula carry a human-readable name for the
|
||||
// closure it wraps. Verifier wires this in when the spec gives the predicate
|
||||
// a property name; otherwise it stays empty and serializes without a name.
|
||||
type PredicateLabel interface {
|
||||
PredicateName() string
|
||||
}
|
||||
|
||||
// ErrorFormula represents a thunk that threw during evaluation. The verifier
|
||||
// substitutes one of these into the residual when MarshalJSON would otherwise
|
||||
// have to encode an opaque thunk that already errored. It exists so that the
|
||||
// inspect UI can render "predicate threw" inline.
|
||||
type ErrorFormula struct {
|
||||
Message string
|
||||
}
|
||||
|
||||
func (ErrorFormula) isFormula() {}
|
||||
func (e ErrorFormula) describe() string {
|
||||
return fmt.Sprintf("Error(%q)", e.Message)
|
||||
}
|
||||
|
||||
type AlwaysFormula struct {
|
||||
Inner Formula
|
||||
}
|
||||
@@ -146,3 +167,104 @@ func (n NotFormula) describe() string { return "Not(" + n.Inner.describe() + ")"
|
||||
|
||||
// Describe returns a debug-friendly representation of the formula.
|
||||
func Describe(formula Formula) string { return formula.describe() }
|
||||
|
||||
// withinNode mirrors the optional `within` clause attached to bounded
|
||||
// Eventually nodes in the JSON AST.
|
||||
type withinNode struct {
|
||||
Amount int64 `json:"amount"`
|
||||
Unit string `json:"unit"`
|
||||
}
|
||||
|
||||
func (a AlwaysFormula) MarshalJSON() ([]byte, error) {
|
||||
return json.Marshal(struct {
|
||||
Op string `json:"op"`
|
||||
Arg Formula `json:"arg"`
|
||||
}{"always", a.Inner})
|
||||
}
|
||||
|
||||
func (n NowFormula) MarshalJSON() ([]byte, error) {
|
||||
return json.Marshal(struct {
|
||||
Op string `json:"op"`
|
||||
Arg Formula `json:"arg"`
|
||||
}{"now", n.Inner})
|
||||
}
|
||||
|
||||
func (n NextFormula) MarshalJSON() ([]byte, error) {
|
||||
return json.Marshal(struct {
|
||||
Op string `json:"op"`
|
||||
Arg Formula `json:"arg"`
|
||||
}{"next", n.Inner})
|
||||
}
|
||||
|
||||
func (n NotFormula) MarshalJSON() ([]byte, error) {
|
||||
return json.Marshal(struct {
|
||||
Op string `json:"op"`
|
||||
Arg Formula `json:"arg"`
|
||||
}{"not", n.Inner})
|
||||
}
|
||||
|
||||
func (e EventuallyFormula) MarshalJSON() ([]byte, error) {
|
||||
payload := struct {
|
||||
Op string `json:"op"`
|
||||
Arg Formula `json:"arg"`
|
||||
Within *withinNode `json:"within,omitempty"`
|
||||
}{Op: "eventually", Arg: e.Inner}
|
||||
switch {
|
||||
case e.HasStepBound:
|
||||
payload.Within = &withinNode{Amount: int64(e.StepBound), Unit: "steps"}
|
||||
case e.Duration > 0:
|
||||
payload.Within = &withinNode{Amount: e.Duration.Milliseconds(), Unit: "milliseconds"}
|
||||
case e.HasDeadline:
|
||||
payload.Within = &withinNode{Amount: e.Deadline.UnixMilli(), Unit: "deadline"}
|
||||
}
|
||||
return json.Marshal(payload)
|
||||
}
|
||||
|
||||
func (a AndFormula) MarshalJSON() ([]byte, error) {
|
||||
return json.Marshal(struct {
|
||||
Op string `json:"op"`
|
||||
Left Formula `json:"left"`
|
||||
Right Formula `json:"right"`
|
||||
}{"and", a.Left, a.Right})
|
||||
}
|
||||
|
||||
func (o OrFormula) MarshalJSON() ([]byte, error) {
|
||||
return json.Marshal(struct {
|
||||
Op string `json:"op"`
|
||||
Left Formula `json:"left"`
|
||||
Right Formula `json:"right"`
|
||||
}{"or", o.Left, o.Right})
|
||||
}
|
||||
|
||||
func (i ImpliesFormula) MarshalJSON() ([]byte, error) {
|
||||
return json.Marshal(struct {
|
||||
Op string `json:"op"`
|
||||
Left Formula `json:"left"`
|
||||
Right Formula `json:"right"`
|
||||
}{"implies", i.Antecedent, i.Consequent})
|
||||
}
|
||||
|
||||
func (p PureFormula) MarshalJSON() ([]byte, error) {
|
||||
if p.Value {
|
||||
return []byte(`{"op":"true"}`), nil
|
||||
}
|
||||
return []byte(`{"op":"false"}`), nil
|
||||
}
|
||||
|
||||
func (t ThunkFormula) MarshalJSON() ([]byte, error) {
|
||||
payload := struct {
|
||||
Op string `json:"op"`
|
||||
Name string `json:"name,omitempty"`
|
||||
}{Op: "predicate"}
|
||||
if labeled, ok := any(t).(PredicateLabel); ok {
|
||||
payload.Name = labeled.PredicateName()
|
||||
}
|
||||
return json.Marshal(payload)
|
||||
}
|
||||
|
||||
func (e ErrorFormula) MarshalJSON() ([]byte, error) {
|
||||
return json.Marshal(struct {
|
||||
Op string `json:"op"`
|
||||
Message string `json:"message"`
|
||||
}{"error", e.Message})
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
package ltl
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -191,3 +192,75 @@ func TestVerdict_StringPending(t *testing.T) {
|
||||
t.Errorf("VerdictPending.String() = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMarshalJSON_AlwaysImpliesEventually(t *testing.T) {
|
||||
formula := Always(Implies(Now(Pure(true)), EventuallyWithinSteps(Pure(false), 3)))
|
||||
body, err := json.Marshal(formula)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := `{"op":"always","arg":{"op":"implies","left":{"op":"now","arg":{"op":"true"}},"right":{"op":"eventually","arg":{"op":"false"},"within":{"amount":3,"unit":"steps"}}}}`
|
||||
if string(body) != want {
|
||||
t.Errorf("marshal mismatch:\n got: %s\nwant: %s", body, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMarshalJSON_AndOrNot(t *testing.T) {
|
||||
formula := And(Or(Pure(true), Pure(false)), Not(Pure(true)))
|
||||
body, _ := json.Marshal(formula)
|
||||
want := `{"op":"and","left":{"op":"or","left":{"op":"true"},"right":{"op":"false"}},"right":{"op":"not","arg":{"op":"true"}}}`
|
||||
if string(body) != want {
|
||||
t.Errorf("and/or/not marshal mismatch:\n got: %s\nwant: %s", body, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMarshalJSON_EventuallyMillisecondsAndDeadline(t *testing.T) {
|
||||
body, _ := json.Marshal(EventuallyWithin(Pure(true), 250*time.Millisecond))
|
||||
if !strings.Contains(string(body), `"unit":"milliseconds"`) || !strings.Contains(string(body), `"amount":250`) {
|
||||
t.Errorf("milliseconds within wrong: %s", body)
|
||||
}
|
||||
deadline := time.UnixMilli(1700000000000)
|
||||
body, _ = json.Marshal(EventuallyBefore(Pure(true), deadline))
|
||||
if !strings.Contains(string(body), `"unit":"deadline"`) || !strings.Contains(string(body), `"amount":1700000000000`) {
|
||||
t.Errorf("deadline within wrong: %s", body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMarshalJSON_NextAndThunkAndError(t *testing.T) {
|
||||
body, _ := json.Marshal(Next(Pure(true)))
|
||||
if string(body) != `{"op":"next","arg":{"op":"true"}}` {
|
||||
t.Errorf("next marshal wrong: %s", body)
|
||||
}
|
||||
body, _ = json.Marshal(Thunk(func() bool { return true }))
|
||||
if string(body) != `{"op":"predicate"}` {
|
||||
t.Errorf("thunk marshal wrong: %s", body)
|
||||
}
|
||||
body, _ = json.Marshal(ErrorFormula{Message: "bad"})
|
||||
if string(body) != `{"op":"error","message":"bad"}` {
|
||||
t.Errorf("error marshal wrong: %s", body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResidual_HoldsViolatedPending(t *testing.T) {
|
||||
holdsEval := NewEvaluator(Always(Pure(true)))
|
||||
holdsEval.Observe()
|
||||
if got := holdsEval.Residual(); got != (PureFormula{Value: true}) {
|
||||
t.Errorf("holds residual = %v, want true", got)
|
||||
}
|
||||
|
||||
violEval := NewEvaluator(Always(Now(Pure(false))))
|
||||
violEval.Observe()
|
||||
if got := violEval.Residual(); got != (PureFormula{Value: false}) {
|
||||
t.Errorf("violated residual = %v, want false", got)
|
||||
}
|
||||
|
||||
pendingEval := NewEvaluator(Always(Next(Pure(true))))
|
||||
pendingEval.Observe()
|
||||
body, err := json.Marshal(pendingEval.Residual())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(body), `"op":"and"`) && !strings.Contains(string(body), `"op":"true"`) {
|
||||
t.Errorf("pending residual unexpected: %s", body)
|
||||
}
|
||||
}
|
||||
+111
-3
@@ -21,6 +21,7 @@ type Options struct {
|
||||
SnapshotTimeout time.Duration
|
||||
IdleTimeout time.Duration
|
||||
|
||||
BundleID string
|
||||
Connection *agent.Conn
|
||||
Driver driver.Driver
|
||||
Verifier *verifier.Verifier
|
||||
@@ -79,6 +80,11 @@ func Run(ctx context.Context, options Options) (Summary, error) {
|
||||
treeSize = len(tree.Elements)
|
||||
}
|
||||
|
||||
// Sample metrics now, while the app is still running freely.
|
||||
// Measuring after snapshotStep would see the SDK-paused app and
|
||||
// report CPU=0 for every step.
|
||||
metrics := captureMetrics(ctx, options, logger, stepIndex)
|
||||
|
||||
snapshot, err := snapshotStep(ctx, options)
|
||||
if err != nil {
|
||||
return summary, fmt.Errorf("step %d snapshot: %w", stepIndex, err)
|
||||
@@ -116,11 +122,16 @@ func Run(ctx context.Context, options Options) (Summary, error) {
|
||||
nextAction, nextErr := options.Verifier.NextAction()
|
||||
var traceAction *trace.Action
|
||||
if nextErr == nil {
|
||||
traceAction = traceActionFor(nextAction)
|
||||
traceAction = traceActionFor(nextAction, tree)
|
||||
} else if !errors.Is(nextErr, verifier.ErrNoAction) {
|
||||
return summary, fmt.Errorf("step %d next action: %w", stepIndex, nextErr)
|
||||
}
|
||||
|
||||
residuals, residualErr := encodeResiduals(options.Verifier.Residuals())
|
||||
if residualErr != nil {
|
||||
logger.Warn("residual encode failed", "step", stepIndex, "err", residualErr)
|
||||
}
|
||||
|
||||
step := trace.Step{
|
||||
Index: stepIndex,
|
||||
Timestamp: stepStart,
|
||||
@@ -129,10 +140,14 @@ func Run(ctx context.Context, options Options) (Summary, error) {
|
||||
Action: traceAction,
|
||||
Exceptions: traceExceptions(exceptions),
|
||||
Violations: violations,
|
||||
Hierarchy: tree,
|
||||
Residuals: residuals,
|
||||
Metrics: metrics,
|
||||
}
|
||||
if err := options.TraceWriter.WriteStep(step); err != nil {
|
||||
return summary, fmt.Errorf("step %d trace: %w", stepIndex, err)
|
||||
}
|
||||
captureScreenshot(ctx, options, logger, stepIndex, false)
|
||||
summary.Steps = stepIndex
|
||||
if len(violations) > 0 {
|
||||
summary.Violations = append(summary.Violations, ViolationRecord{
|
||||
@@ -157,6 +172,9 @@ func Run(ctx context.Context, options Options) (Summary, error) {
|
||||
|
||||
idleCtx, idleCancel := context.WithTimeout(ctx, options.IdleTimeout)
|
||||
idleErr := options.Driver.WaitForIdle(idleCtx, options.IdleTimeout)
|
||||
if nextErr == nil {
|
||||
captureScreenshot(ctx, options, logger, stepIndex, true)
|
||||
}
|
||||
if idleErr != nil && idleCtx.Err() == nil {
|
||||
logger.Warn("wait_for_idle failed", "step", stepIndex, "err", idleErr)
|
||||
}
|
||||
@@ -334,13 +352,16 @@ func fetchHierarchy(ctx context.Context, drv driver.Driver) (*hierarchy.Tree, er
|
||||
return hierarchy.Parse(xmlText)
|
||||
}
|
||||
|
||||
func traceActionFor(action verifier.Action) *trace.Action {
|
||||
func traceActionFor(action verifier.Action, tree *hierarchy.Tree) *trace.Action {
|
||||
traceAction := &trace.Action{Kind: string(action.Kind), X: action.X, Y: action.Y}
|
||||
switch action.Kind {
|
||||
case verifier.ActionKindTap:
|
||||
traceAction.Text = action.On
|
||||
traceAction.Selector = action.On
|
||||
stampSelectorTarget(traceAction, action, tree)
|
||||
case verifier.ActionKindInputText:
|
||||
traceAction.Text = action.Text
|
||||
traceAction.Selector = action.On
|
||||
stampSelectorTarget(traceAction, action, tree)
|
||||
case verifier.ActionKindSwipe:
|
||||
traceAction.FromX = action.FromX
|
||||
traceAction.FromY = action.FromY
|
||||
@@ -357,6 +378,93 @@ func traceActionFor(action verifier.Action) *trace.Action {
|
||||
return traceAction
|
||||
}
|
||||
|
||||
// stampSelectorTarget mirrors applyAction's coordinate-resolution rule so the
|
||||
// trace records the same point the runner taps. When the spec passed an ax
|
||||
// element directly, action.X/Y are already populated and we use them; when the
|
||||
// spec passed a string selector, we resolve it against the captured hierarchy.
|
||||
func stampSelectorTarget(traceAction *trace.Action, action verifier.Action, tree *hierarchy.Tree) {
|
||||
if action.X > 0 && action.Y > 0 {
|
||||
traceAction.TapPoint = &trace.PointRecord{X: action.X, Y: action.Y}
|
||||
return
|
||||
}
|
||||
if tree == nil || action.On == "" {
|
||||
return
|
||||
}
|
||||
element := tree.Find(action.On)
|
||||
if element == nil {
|
||||
return
|
||||
}
|
||||
bounds := element.Bounds
|
||||
traceAction.ResolvedBounds = &trace.BoundsRecord{
|
||||
X: bounds.Left,
|
||||
Y: bounds.Top,
|
||||
Width: bounds.Width(),
|
||||
Height: bounds.Height(),
|
||||
}
|
||||
x, y := bounds.Center()
|
||||
if x > 0 && y > 0 {
|
||||
traceAction.TapPoint = &trace.PointRecord{X: x, Y: y}
|
||||
}
|
||||
}
|
||||
|
||||
func captureMetrics(ctx context.Context, options Options, logger *slog.Logger, stepIndex int) *trace.Metrics {
|
||||
if options.BundleID == "" {
|
||||
return nil
|
||||
}
|
||||
sample, err := options.Driver.Metrics(ctx, options.BundleID)
|
||||
if err != nil {
|
||||
logger.Warn("metrics capture failed", "step", stepIndex, "err", err)
|
||||
return nil
|
||||
}
|
||||
if sample.CPUPercent == 0 && sample.HeapBytes == 0 && sample.TotalMemoryBytes == 0 {
|
||||
return nil
|
||||
}
|
||||
return &trace.Metrics{
|
||||
CPUPercent: sample.CPUPercent,
|
||||
HeapBytes: sample.HeapBytes,
|
||||
TotalMemoryBytes: sample.TotalMemoryBytes,
|
||||
}
|
||||
}
|
||||
|
||||
func captureScreenshot(ctx context.Context, options Options, logger *slog.Logger, stepIndex int, after bool) {
|
||||
image, err := options.Driver.Screenshot(ctx)
|
||||
if err != nil {
|
||||
logger.Warn("screenshot capture failed", "step", stepIndex, "after", after, "err", err)
|
||||
return
|
||||
}
|
||||
if len(image.PNG) == 0 {
|
||||
return
|
||||
}
|
||||
var writeErr error
|
||||
if after {
|
||||
writeErr = options.TraceWriter.WriteScreenshotAfter(stepIndex, image.PNG)
|
||||
} else {
|
||||
writeErr = options.TraceWriter.WriteScreenshot(stepIndex, image.PNG)
|
||||
}
|
||||
if writeErr != nil {
|
||||
logger.Warn("screenshot write failed", "step", stepIndex, "after", after, "err", writeErr)
|
||||
}
|
||||
}
|
||||
|
||||
func encodeResiduals(residuals map[string]ltl.Formula) (map[string]json.RawMessage, error) {
|
||||
if len(residuals) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
encoded := make(map[string]json.RawMessage, len(residuals))
|
||||
var firstErr error
|
||||
for name, formula := range residuals {
|
||||
body, err := json.Marshal(formula)
|
||||
if err != nil {
|
||||
if firstErr == nil {
|
||||
firstErr = err
|
||||
}
|
||||
continue
|
||||
}
|
||||
encoded[name] = body
|
||||
}
|
||||
return encoded, firstErr
|
||||
}
|
||||
|
||||
func traceExceptions(exceptions []verifier.Exception) []trace.Exception {
|
||||
if len(exceptions) == 0 {
|
||||
return nil
|
||||
|
||||
@@ -310,6 +310,55 @@ func TestScreenFromSnapshot(t *testing.T) {
|
||||
})
|
||||
}
|
||||
|
||||
func TestRunner_StampsHierarchyResolvedBoundsAndResiduals(t *testing.T) {
|
||||
snapshots := []map[string]json.RawMessage{
|
||||
{"balance": json.RawMessage(`100`)},
|
||||
{"balance": json.RawMessage(`200`)},
|
||||
}
|
||||
state := newHarness(t, snapshots)
|
||||
state.startSDK(t)
|
||||
state.acceptConnection(t)
|
||||
|
||||
state.mock.HierarchyJSON = `<?xml version="1.0"?>
|
||||
<hierarchy>
|
||||
<node resource-id="com.fixture:id/next" clickable="true" enabled="true" bounds="[40,80][240,160]"/>
|
||||
</hierarchy>`
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if _, err := Run(ctx, Options{
|
||||
Duration: 100 * time.Millisecond,
|
||||
SnapshotTimeout: 2 * time.Second,
|
||||
IdleTimeout: 50 * time.Millisecond,
|
||||
Connection: state.conn,
|
||||
Driver: state.mock,
|
||||
Verifier: state.verifier,
|
||||
TraceWriter: state.writer,
|
||||
}); err != nil {
|
||||
t.Fatalf("Run: %v", err)
|
||||
}
|
||||
body, err := os.ReadFile(filepath.Join(state.writer.Directory(), "trace.jsonl"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
text := string(body)
|
||||
if !strings.Contains(text, `"selector":"id:next"`) {
|
||||
t.Errorf("expected selector in trace: %s", text)
|
||||
}
|
||||
if !strings.Contains(text, `"resolved_bounds":{"x":40,"y":80,"width":200,"height":80}`) {
|
||||
t.Errorf("expected resolved_bounds in trace: %s", text)
|
||||
}
|
||||
if !strings.Contains(text, `"tap_point":{"x":140,"y":120}`) {
|
||||
t.Errorf("expected tap_point in trace: %s", text)
|
||||
}
|
||||
if !strings.Contains(text, `"hierarchy":{"elements":`) {
|
||||
t.Errorf("expected hierarchy in trace: %s", text)
|
||||
}
|
||||
if !strings.Contains(text, `"residuals":{`) {
|
||||
t.Errorf("expected residuals in trace: %s", text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunner_LogsWaitForIdleDriverErrors(t *testing.T) {
|
||||
snapshots := []map[string]json.RawMessage{
|
||||
{"balance": json.RawMessage(`100`)},
|
||||
|
||||
+56
-19
@@ -8,6 +8,8 @@ import (
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/priyanshujain/uatu/internal/hierarchy"
|
||||
)
|
||||
|
||||
type Step struct {
|
||||
@@ -18,19 +20,43 @@ type Step struct {
|
||||
Action *Action `json:"action,omitempty"`
|
||||
Exceptions []Exception `json:"exceptions,omitempty"`
|
||||
Violations []string `json:"violations,omitempty"`
|
||||
Hierarchy *hierarchy.Tree `json:"hierarchy,omitempty"`
|
||||
Residuals map[string]json.RawMessage `json:"residuals,omitempty"`
|
||||
Metrics *Metrics `json:"metrics,omitempty"`
|
||||
}
|
||||
|
||||
type Metrics struct {
|
||||
CPUPercent float64 `json:"cpu_percent"`
|
||||
HeapBytes int64 `json:"heap_bytes,omitempty"`
|
||||
TotalMemoryBytes int64 `json:"total_memory_bytes,omitempty"`
|
||||
}
|
||||
|
||||
type Action struct {
|
||||
Kind string `json:"kind"`
|
||||
X int `json:"x,omitempty"`
|
||||
Y int `json:"y,omitempty"`
|
||||
FromX int `json:"from_x,omitempty"`
|
||||
FromY int `json:"from_y,omitempty"`
|
||||
ToX int `json:"to_x,omitempty"`
|
||||
ToY int `json:"to_y,omitempty"`
|
||||
Key string `json:"key,omitempty"`
|
||||
Text string `json:"text,omitempty"`
|
||||
DurationMillis int `json:"duration_millis,omitempty"`
|
||||
Kind string `json:"kind"`
|
||||
X int `json:"x,omitempty"`
|
||||
Y int `json:"y,omitempty"`
|
||||
FromX int `json:"from_x,omitempty"`
|
||||
FromY int `json:"from_y,omitempty"`
|
||||
ToX int `json:"to_x,omitempty"`
|
||||
ToY int `json:"to_y,omitempty"`
|
||||
Key string `json:"key,omitempty"`
|
||||
Text string `json:"text,omitempty"`
|
||||
DurationMillis int `json:"duration_millis,omitempty"`
|
||||
Selector string `json:"selector,omitempty"`
|
||||
ResolvedBounds *BoundsRecord `json:"resolved_bounds,omitempty"`
|
||||
TapPoint *PointRecord `json:"tap_point,omitempty"`
|
||||
}
|
||||
|
||||
type BoundsRecord struct {
|
||||
X int `json:"x"`
|
||||
Y int `json:"y"`
|
||||
Width int `json:"width"`
|
||||
Height int `json:"height"`
|
||||
}
|
||||
|
||||
type PointRecord struct {
|
||||
X int `json:"x"`
|
||||
Y int `json:"y"`
|
||||
}
|
||||
|
||||
type Exception struct {
|
||||
@@ -41,13 +67,14 @@ type Exception struct {
|
||||
}
|
||||
|
||||
type Meta struct {
|
||||
Seed int64 `json:"seed"`
|
||||
SpecPath string `json:"spec_path"`
|
||||
BundleSHA256 string `json:"bundle_sha256"`
|
||||
Platform string `json:"platform"`
|
||||
BundleID string `json:"bundle_id"`
|
||||
StartedAt time.Time `json:"started_at"`
|
||||
UatuVersion string `json:"uatu_version"`
|
||||
Seed int64 `json:"seed"`
|
||||
SpecPath string `json:"spec_path"`
|
||||
BundleSHA256 string `json:"bundle_sha256"`
|
||||
Platform string `json:"platform"`
|
||||
BundleID string `json:"bundle_id"`
|
||||
StartedAt time.Time `json:"started_at"`
|
||||
EndedAt *time.Time `json:"ended_at,omitempty"`
|
||||
UatuVersion string `json:"uatu_version"`
|
||||
}
|
||||
|
||||
type Writer struct {
|
||||
@@ -95,6 +122,17 @@ func (w *Writer) WriteStep(step Step) error {
|
||||
}
|
||||
|
||||
func (w *Writer) WriteScreenshot(stepIndex int, png []byte) error {
|
||||
return w.writePNG(fmt.Sprintf("step-%05d.png", stepIndex), png)
|
||||
}
|
||||
|
||||
// WriteScreenshotAfter writes the post-action screenshot for a step.
|
||||
// Callers use this after applyAction + waitForIdle so the UI can show a
|
||||
// before/after pair.
|
||||
func (w *Writer) WriteScreenshotAfter(stepIndex int, png []byte) error {
|
||||
return w.writePNG(fmt.Sprintf("step-%05d-after.png", stepIndex), png)
|
||||
}
|
||||
|
||||
func (w *Writer) writePNG(name string, png []byte) error {
|
||||
if len(png) == 0 {
|
||||
return nil
|
||||
}
|
||||
@@ -102,8 +140,7 @@ func (w *Writer) WriteScreenshot(stepIndex int, png []byte) error {
|
||||
if err := os.MkdirAll(directory, 0o755); err != nil {
|
||||
return fmt.Errorf("mkdir screenshots: %w", err)
|
||||
}
|
||||
path := filepath.Join(directory, fmt.Sprintf("step-%05d.png", stepIndex))
|
||||
return os.WriteFile(path, png, 0o644)
|
||||
return os.WriteFile(filepath.Join(directory, name), png, 0o644)
|
||||
}
|
||||
|
||||
func (w *Writer) Close() error {
|
||||
|
||||
@@ -44,6 +44,109 @@ func TestWriteMeta_RoundTrip(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteMeta_EndedAtRoundTrip(t *testing.T) {
|
||||
directory := t.TempDir()
|
||||
writer, err := NewWriter(directory)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer writer.Close()
|
||||
|
||||
endedAt := time.Date(2026, 4, 17, 22, 31, 0, 0, time.UTC)
|
||||
meta := Meta{
|
||||
Seed: 7,
|
||||
SpecPath: "spec.ts",
|
||||
Platform: "android",
|
||||
BundleID: "in.test",
|
||||
StartedAt: time.Date(2026, 4, 17, 22, 30, 0, 0, time.UTC),
|
||||
EndedAt: &endedAt,
|
||||
UatuVersion: "0.0.1",
|
||||
}
|
||||
if err := writer.WriteMeta(meta); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
body, err := os.ReadFile(filepath.Join(directory, "meta.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(body), `"ended_at": "2026-04-17T22:31:00Z"`) {
|
||||
t.Errorf("ended_at not in meta.json: %s", body)
|
||||
}
|
||||
var got Meta
|
||||
if err := json.Unmarshal(body, &got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.EndedAt == nil || !got.EndedAt.Equal(endedAt) {
|
||||
t.Errorf("EndedAt round-trip wrong: %v", got.EndedAt)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteMeta_OmitsEndedAtWhenNil(t *testing.T) {
|
||||
directory := t.TempDir()
|
||||
writer, _ := NewWriter(directory)
|
||||
defer writer.Close()
|
||||
if err := writer.WriteMeta(Meta{StartedAt: time.Now().UTC()}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
body, _ := os.ReadFile(filepath.Join(directory, "meta.json"))
|
||||
if strings.Contains(string(body), "ended_at") {
|
||||
t.Errorf("ended_at should be omitted when nil: %s", body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteStep_HierarchyAndResidualsRoundTrip(t *testing.T) {
|
||||
directory := t.TempDir()
|
||||
writer, _ := NewWriter(directory)
|
||||
defer writer.Close()
|
||||
|
||||
step := Step{
|
||||
Index: 1,
|
||||
Timestamp: time.Now().UTC(),
|
||||
Action: &Action{
|
||||
Kind: "tap",
|
||||
Selector: "id:next",
|
||||
ResolvedBounds: &BoundsRecord{X: 10, Y: 20, Width: 100, Height: 50},
|
||||
TapPoint: &PointRecord{X: 60, Y: 45},
|
||||
},
|
||||
Residuals: map[string]json.RawMessage{
|
||||
"prop1": json.RawMessage(`{"op":"true"}`),
|
||||
},
|
||||
}
|
||||
if err := writer.WriteStep(step); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
body, _ := os.ReadFile(filepath.Join(directory, "trace.jsonl"))
|
||||
var got Step
|
||||
if err := json.Unmarshal(body, &got); err != nil {
|
||||
t.Fatalf("bad jsonl: %v\n%s", err, body)
|
||||
}
|
||||
if got.Action.Selector != "id:next" {
|
||||
t.Errorf("selector = %q", got.Action.Selector)
|
||||
}
|
||||
if got.Action.ResolvedBounds == nil || got.Action.ResolvedBounds.Width != 100 {
|
||||
t.Errorf("resolved_bounds round-trip wrong: %+v", got.Action.ResolvedBounds)
|
||||
}
|
||||
if got.Action.TapPoint == nil || got.Action.TapPoint.X != 60 {
|
||||
t.Errorf("tap_point round-trip wrong: %+v", got.Action.TapPoint)
|
||||
}
|
||||
if string(got.Residuals["prop1"]) != `{"op":"true"}` {
|
||||
t.Errorf("residuals round-trip wrong: %s", got.Residuals["prop1"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteStep_OmitsEmptyHierarchyAndResiduals(t *testing.T) {
|
||||
directory := t.TempDir()
|
||||
writer, _ := NewWriter(directory)
|
||||
defer writer.Close()
|
||||
if err := writer.WriteStep(Step{Index: 1}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
body, _ := os.ReadFile(filepath.Join(directory, "trace.jsonl"))
|
||||
if strings.Contains(string(body), "hierarchy") || strings.Contains(string(body), "residuals") {
|
||||
t.Errorf("empty hierarchy/residuals must omit: %s", body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteStep_AppendsOneJsonLine(t *testing.T) {
|
||||
directory := t.TempDir()
|
||||
writer, err := NewWriter(directory)
|
||||
|
||||
@@ -273,6 +273,22 @@ func (v *Verifier) EvaluateProperties() map[string]ltl.Verdict {
|
||||
return verdicts
|
||||
}
|
||||
|
||||
// Residuals returns the residual formula for each registered property after
|
||||
// the most recent EvaluateProperties call. Properties that errored during
|
||||
// predicate evaluation surface as ErrorFormula so the inspect UI can render
|
||||
// "predicate threw" inline.
|
||||
func (v *Verifier) Residuals() map[string]ltl.Formula {
|
||||
residuals := map[string]ltl.Formula{}
|
||||
for name, evaluator := range v.evaluators {
|
||||
if predicateErr := v.PredicateError(name); predicateErr != nil {
|
||||
residuals[name] = ltl.ErrorFormula{Message: predicateErr.Error()}
|
||||
continue
|
||||
}
|
||||
residuals[name] = evaluator.Residual()
|
||||
}
|
||||
return residuals
|
||||
}
|
||||
|
||||
// NextAction resolves the root action generator into a single Action.
|
||||
// Returns ErrNoAction when no branch of the generator produces one after a
|
||||
// small number of retries. Retrying avoids wedging when most branches of a
|
||||
|
||||
Reference in new issue
Block a user