mirror of
https://github.com/priyanshujain/sanderling.git
synced 2026-10-02 19:17:10 +00:00
fix(trace): an action names the generator that produced it
The setup exclusion landed for the model arm only, because only a model pick stamped a source. A seeded run returned setup's action through the same entry with no marker, so its denominator still counted the login while the model arm's did not, and the two are compared. serializeAction names setup and seeded on the wire, so both arms are counted by one rule. An already-recorded trace names nothing and keeps exactly the count it was reported with; unattributed_actions counts those steps so the old denominator cannot pass as the new one. TraceVersion is deliberately unbumped: oracle-reduction refuses a differing version, and a bump would make all 169 recorded runs unreplayable.
This commit is contained in:
1 parent
66fd5bce5d
commit
454988fbc8
19 files changed
+377
-82
No files matched your search
@@ -109,6 +109,16 @@ type Metrics struct {
|
||||
TotalMemoryBytes int64 `json:"total_memory_bytes,omitempty"`
|
||||
}
|
||||
|
||||
// The three producers an action can come from. The spec's setup drives the app
|
||||
// into position and explores nothing, so a per-action rate divides by the other
|
||||
// two; an action recorded before this distinction existed carries none of them
|
||||
// and cannot be attributed after the fact.
|
||||
const (
|
||||
ActionSourceSetup = "setup"
|
||||
ActionSourceSeeded = "seeded"
|
||||
ActionSourceModel = "llm"
|
||||
)
|
||||
|
||||
type Action struct {
|
||||
Kind string `json:"kind"`
|
||||
X int `json:"x,omitempty"`
|
||||
@@ -123,8 +133,8 @@ type Action struct {
|
||||
Selector string `json:"selector,omitempty"`
|
||||
ResolvedBounds *BoundsRecord `json:"resolved_bounds,omitempty"`
|
||||
TapPoint *PointRecord `json:"tap_point,omitempty"`
|
||||
// Source names the backend that chose this action: "llm" when the LLM
|
||||
// action backend selected it, empty for the seeded picker. LLMReasoning is
|
||||
// Source names which of the three producers chose this action, and is empty
|
||||
// only on a trace recorded before actions named themselves. LLMReasoning is
|
||||
// the model's short rationale, shown by the replay UI to explain the pick.
|
||||
Source string `json:"source,omitempty"`
|
||||
LLMReasoning string `json:"llm_reasoning,omitempty"`
|
||||
|
||||
Reference in new issue
Block a user