feat: LTL operators, sampling, and default generators (#17)

* feat(ltl): add Now/Next/Eventually/Implies/Or/And/Not formulas

Replace the fold-with-latch evaluator with a residual-formula reducer.
Each Observe() instantiates a fresh obligation from the root (stripping
an outer Always), reduces each pending obligation against current state,
latches Violated on first failure, and surfaces Pending verdicts for
deferred obligations. Existing Always/Pure/Thunk tests continue to pass.

* feat(ltl): support relative duration for eventually().within()

* feat(proto): add Swipe, PressKey, RecentLogs RPCs

* feat(verifier,runner): formula handles, new action kinds, rich state

- verifier: add formula-spec registry; bindNow/bindNext/bindEventually with
  chainable .implies/.or/.and/.not and .within(n,unit) on eventually; bindFrom
  for uniform sampling. bindAlways keeps accepting plain predicates.
- verifier: store lastTree, lastAction, step time, logs, exceptions on the
  Verifier; SnapshotInput replaces the (snapshots, tree) pair. stateObject now
  produces state.lastAction/time/logs/exceptions matching the TS State type.
- verifier: make taps/swipes/waitOnce/pressKey built-in generators actually
  fire; taps picks a clickable, enabled element from the last hierarchy.
- agent: add exceptions field to Message wire format.
- driver: add Swipe/PressKey/RecentLogs to Driver interface; wire maestro
  client and mock driver. LogEntry exposed for runner consumption.
- runner: apply Swipe/PressKey/Wait actions; collect logcat and exceptions;
  pass lastAction and step time into PushSnapshot.

* feat(spec-api): LTL operators, new actions, richer State

- ltl.ts exports now/next/eventually; always overload accepts a Formula
- types.ts: Formula gains implies/or/and/not; EventuallyFormula adds .within;
  State gains lastAction/time/logs/exceptions; Swipe/PressKey/Wait action types
- actions.ts: Swipe/PressKey/Wait/from constructors; waitOnce + pressKey
  default generators
- tests exercise the chaining, sampling, and new actions through a recorded
  fake runtime

* feat(sidecar): add swipe, pressKey, recentLogs RPC handlers

* feat(sdk-android): capture uncaught exceptions

Install a default uncaught handler on Uatu.start, chained with any
existing handler so Android's crash reporter still runs. Expose
Uatu.reportError for callers to forward caught throwables. A bounded
circular buffer (default 50) drains into each STATE message's new
exceptions field. Protocol.kt serializes/deserializes the field,
matching the Go wire format added to internal/agent/protocol.go.

* feat(spec-api): add @uatu/spec/defaults/properties bundle

* feat(sample-app): exercise new LTL operators + defaults

spec.ts now imports eventually/next/now/from from @uatu/spec and
noUncaughtExceptions from @uatu/spec/defaults/properties. It declares
three properties that exercise the new surface:

- accountCountNonNegative: plain always() safety
- addAccountAdvances: always(now(x).implies(next(y)))
- eventuallyLoggedIn: eventually(p).within(30, "seconds")
- noUncaughtExceptions: imported default

The weighted actions root uses from() for random phone/name sampling
and entries for taps/swipes/waitOnce/pressKey built-ins.

SampleApplication gains a debug hook gated on the system property
uatu.inject_error so the e2e run can synthesize an Uatu.reportError and
verify noUncaughtExceptions violates.

cmd/uatu/test_run.go adds a subpath alias so specs importing
"@uatu/spec/defaults/properties" resolve against the in-tree source
when running from the uatu checkout. The spec-integration tests swap
the old click-counter fixtures for the new login hierarchy.

* feat(trace): record swipe/key/wait details + exceptions

trace.Step gains an Exceptions array so the trace captures the
class/message/stackTrace for each SDK-reported throwable in a step.
trace.Action gains FromX/FromY/ToX/ToY/Key/DurationMillis so the full
payload of Swipe/PressKey/Wait actions is visible in trace.jsonl.

sample-app's debug error hook now gates on ApplicationInfo.DEBUGGABLE
instead of a system property (adb setprop fails on non-rooted
emulators).
This commit is contained in:
pj authored and GitHub committed 2026-04-20 02:19:39 +07:00
1 parent 5b5594ee05
commit 7493945251
40 files changed
+2980 -262

No files matched your search

@@ -0,0 +1,62 @@
package dev.uatu.sdk
import java.io.PrintWriter
import java.io.StringWriter
internal class ExceptionRecorder(private val capacity: Int = DEFAULT_CAPACITY) {
data class Entry(
val className: String,
val message: String,
val stackTrace: String,
val unixMillis: Long,
)
private val buffer: ArrayDeque<Entry> = ArrayDeque()
private var chainedHandler: Thread.UncaughtExceptionHandler? = null
@Volatile private var installed: Boolean = false
@Synchronized
fun install() {
if (installed) return
chainedHandler = Thread.getDefaultUncaughtExceptionHandler()
Thread.setDefaultUncaughtExceptionHandler { thread, throwable ->
record(throwable)
chainedHandler?.uncaughtException(thread, throwable)
}
installed = true
}
@Synchronized
fun uninstall() {
if (!installed) return
Thread.setDefaultUncaughtExceptionHandler(chainedHandler)
chainedHandler = null
installed = false
}
@Synchronized
fun record(throwable: Throwable, now: Long = System.currentTimeMillis()) {
val stackTrace = StringWriter().also { throwable.printStackTrace(PrintWriter(it)) }.toString()
val entry = Entry(
className = throwable.javaClass.name,
message = throwable.message ?: "",
stackTrace = stackTrace,
unixMillis = now,
)
if (buffer.size >= capacity) {
buffer.removeFirst()
}
buffer.addLast(entry)
}
@Synchronized
fun drain(): List<Entry> {
val snapshot = buffer.toList()
buffer.clear()
return snapshot
}
companion object {
const val DEFAULT_CAPACITY: Int = 50
}
}
@@ -31,6 +31,7 @@ data class Message(
val platform: String? = null,
val appPackage: String? = null,
val snapshots: Map<String, Any?>? = null,
val exceptions: List<Map<String, Any?>>? = null,
val extractor: String? = null,
val result: Any? = null,
val error: String? = null,
@@ -50,8 +51,16 @@ data class Message(
fun resume(id: Long): Message = Message(MessageType.RESUME, id = id)
fun state(id: Long, snapshots: Map<String, Any?>): Message =
Message(MessageType.STATE, id = id, snapshots = snapshots)
fun state(
id: Long,
snapshots: Map<String, Any?>,
exceptions: List<Map<String, Any?>>? = null,
): Message = Message(
MessageType.STATE,
id = id,
snapshots = snapshots,
exceptions = exceptions,
)
fun extractResult(id: Long, extractor: String, result: Any?, error: String? = null): Message =
Message(MessageType.EXTRACT_RESULT, id = id, extractor = extractor, result = result, error = error)
@@ -106,6 +115,15 @@ object Protocol {
}
json.put("snapshots", snapshotsJson)
}
message.exceptions?.let { exceptions ->
val array = JSONArray()
for (entry in exceptions) {
val entryJson = JSONObject()
for ((key, value) in entry) entryJson.put(key, wrap(value))
array.put(entryJson)
}
json.put("exceptions", array)
}
message.extractor?.let { json.put("extractor", it) }
message.result?.let { json.put("result", wrap(it)) }
message.error?.let { json.put("error", it) }
@@ -126,6 +144,14 @@ object Protocol {
snapshots = json.optJSONObject("snapshots")?.let { snapshotsJson ->
snapshotsJson.keys().asSequence().associateWith { unwrap(snapshotsJson.get(it)) }
},
exceptions = json.optJSONArray("exceptions")?.let { array ->
buildList {
for (index in 0 until array.length()) {
val item = array.optJSONObject(index) ?: continue
add(item.keys().asSequence().associateWith { unwrap(item.get(it)) })
}
}
},
extractor = json.optStringOrNull("extractor"),
result = if (json.has("result") && !json.isNull("result")) unwrap(json.get("result")) else null,
error = json.optStringOrNull("error"),
@@ -36,6 +36,16 @@ object Uatu {
activeRuntime.register(name, function)
}
/**
* Records a caught [Throwable] so it surfaces in the next STATE message's
* exceptions field. Useful for coroutine CoroutineExceptionHandler,
* OkHttp interceptors, or anywhere else the host app catches errors it
* still wants verified against properties like noUncaughtExceptions.
*/
fun reportError(throwable: Throwable) {
runtime?.reportError(throwable)
}
@Synchronized
internal fun stopForTest() {
runtime?.stop()
@@ -8,23 +8,30 @@ internal class UatuRuntime(
private val version: String,
private val platform: String,
private val appPackage: String,
private val exceptionRecorder: ExceptionRecorder = ExceptionRecorder(),
) {
private val extractors = LinkedHashMap<String, () -> Any?>()
@Volatile private var sender: SocketClient.MessageSender? = null
private val socketClient = SocketClient(transport, AgentHandler())
fun start() {
exceptionRecorder.install()
socketClient.start()
}
fun stop() {
socketClient.stop()
exceptionRecorder.uninstall()
}
fun register(name: String, extractor: () -> Any?) {
synchronized(extractors) { extractors[name] = extractor }
}
fun reportError(throwable: Throwable) {
exceptionRecorder.record(throwable)
}
internal fun snapshot(): Map<String, Any?> {
val drained = synchronized(extractors) { LinkedHashMap(extractors) }
val result = LinkedHashMap<String, Any?>(drained.size)
@@ -67,9 +74,17 @@ internal class UatuRuntime(
Log.w(LOG_TAG, "snapshot failed: $cause")
emptyMap()
}
val exceptions = exceptionRecorder.drain().map { entry ->
mapOf(
"class" to entry.className,
"message" to entry.message,
"stack_trace" to entry.stackTrace,
"unix_millis" to entry.unixMillis,
)
}.takeIf { it.isNotEmpty() }
val activeSender = sender ?: return
try {
activeSender.send(Message.state(id, snapshots))
activeSender.send(Message.state(id, snapshots, exceptions))
} catch (cause: Exception) {
Log.w(LOG_TAG, "failed to send STATE: $cause")
}
@@ -0,0 +1,64 @@
package dev.uatu.sdk
import org.junit.Assert.assertEquals
import org.junit.Assert.assertTrue
import org.junit.Test
class ExceptionRecorderTest {
@Test fun recordsClassMessageAndStackTrace() {
val recorder = ExceptionRecorder()
recorder.record(RuntimeException("boom"))
val drained = recorder.drain()
assertEquals(1, drained.size)
val entry = drained[0]
assertEquals("java.lang.RuntimeException", entry.className)
assertEquals("boom", entry.message)
assertTrue(
"stackTrace should include the class name, got: ${entry.stackTrace}",
entry.stackTrace.contains("RuntimeException"),
)
}
@Test fun drainClearsBuffer() {
val recorder = ExceptionRecorder()
recorder.record(RuntimeException("first"))
recorder.record(RuntimeException("second"))
assertEquals(2, recorder.drain().size)
assertEquals(0, recorder.drain().size)
}
@Test fun dropsOldestWhenOverCapacity() {
val recorder = ExceptionRecorder(capacity = 2)
recorder.record(RuntimeException("a"))
recorder.record(RuntimeException("b"))
recorder.record(RuntimeException("c"))
val drained = recorder.drain()
assertEquals(2, drained.size)
assertEquals("b", drained[0].message)
assertEquals("c", drained[1].message)
}
@Test fun installChainsExistingHandler() {
val recorder = ExceptionRecorder()
val original = Thread.getDefaultUncaughtExceptionHandler()
var chainedInvoked = false
Thread.setDefaultUncaughtExceptionHandler { _, _ -> chainedInvoked = true }
try {
recorder.install()
// Simulate an uncaught exception by invoking the installed handler
// directly — we don't need to actually terminate a thread.
Thread.getDefaultUncaughtExceptionHandler()!!.uncaughtException(
Thread.currentThread(),
IllegalStateException("chain me"),
)
assertTrue("chained handler should have fired", chainedInvoked)
assertEquals(1, recorder.drain().size)
} finally {
recorder.uninstall()
Thread.setDefaultUncaughtExceptionHandler(original)
}
}
}
@@ -116,6 +116,22 @@ class ProtocolTest {
assertTrue("expected missing-type error, got: ${error.message}", error.message!!.contains("missing type"))
}
@Test fun roundTripStateWithExceptions() {
val exceptions = listOf(
mapOf<String, Any?>(
"class" to "java.lang.RuntimeException",
"message" to "boom",
"stack_trace" to "at Foo.bar(Foo.kt:42)",
"unix_millis" to 1_700_000_000_000L,
),
)
val got = roundTrip(Message.state(3, mapOf("screen" to "home"), exceptions))
assertNotNull(got.exceptions)
assertEquals(1, got.exceptions!!.size)
assertEquals("java.lang.RuntimeException", got.exceptions[0]["class"])
assertEquals("boom", got.exceptions[0]["message"])
}
@Test fun streamsMultipleFrames() {
val messages = listOf(
Message.hello("v", "android", "com.x"),