mirror of
https://github.com/priyanshujain/sanderling.git
synced 2026-10-02 19:17:10 +00:00
fix(verifier): a secure field's typed value never reaches the record
A folio login run wrote the account email and password in cleartext into llm-calls.jsonl, 166 times in one run, beside screenshots of the same screens. Three sites rendered it: the recent-action memory, the candidate list, and the trace. One helper now covers all three so a fourth cannot bypass it, and the driver still receives the real text. Android redacts every typed value because it cannot tell a secure field from any other. That asymmetry is deliberate and documented: safe by default on the target that cannot tell.
This commit is contained in:
1 parent
b1e95739ad
commit
38d328df90
12 files changed
+418
-16
No files matched your search
@@ -29,6 +29,12 @@ sanderling test --spec spec.ts --bundle-id com.example.app --duration 30m
|
||||
|
||||
The trace is written incrementally. An interrupted run is complete up to the step where it stopped.
|
||||
|
||||
## Typed values in the record
|
||||
|
||||
A run types into whatever the app puts on screen, login forms included, and the trace and the model call record are both shared. So a typed value is written down as `[redacted]` whenever the target may be a credential entry: the trace action, the recent-action memory the prompt carries, and the numbered candidate list all render it that way. The app still receives the real keystrokes; only the record is redacted, and the record still names the field that was typed into.
|
||||
|
||||
Which values that covers differs by platform, because the platforms differ in what they report. iOS and web state on every editable field whether it masks its input, so only the fields that do are redacted and the rest of the memory keeps its values. Android reports nothing: uiautomator's password attribute is dropped by the native tree mapper before the driver sees it, a password field is indistinguishable from a search box there, and so every typed value on Android is redacted.
|
||||
|
||||
## App state across runs
|
||||
|
||||
By default each run wipes app data before launch and starts cold. Pass `--clear-data=false` to resume whatever the previous run left behind (an account, cached responses, completed onboarding). See the [CLI reference](../cli/#sanderling-test).
|
||||
|
||||
@@ -78,7 +78,7 @@ s.ax.find({ testTag: "AccountCard", clickable: true })
|
||||
|
||||
Every key-value pair must match. A key means the same thing here as in the string form: `id`, `desc`, `idPrefix`, `descPrefix` and `tag` keep their matching rules, and every other key is an attribute name, with substring and boolean rules per attribute.
|
||||
|
||||
Known attribute names are typed; you get autocomplete on `testTag`, `text`, `content-desc`, the boolean states (`clickable`, `enabled`, `focused`, `checked`, `selected`), and the cross-platform aliases (`identifier`, `accessibilityIdentifier`, `accessibilityText`, `accessibilityLabel`, `label`, `resource-id`, `class`, `elementType`, `package`, `placeholderValue`, `hintText`). Boolean state attributes accept a native `true` / `false`. Other attribute keys still type-check as a string-valued fallback so raw driver attributes remain reachable.
|
||||
Known attribute names are typed; you get autocomplete on `testTag`, `text`, `content-desc`, the boolean states (`clickable`, `enabled`, `focused`, `checked`, `selected`, `secure`), and the cross-platform aliases (`identifier`, `accessibilityIdentifier`, `accessibilityText`, `accessibilityLabel`, `label`, `resource-id`, `class`, `elementType`, `package`, `placeholderValue`, `hintText`). Boolean state attributes accept a native `true` / `false`. Other attribute keys still type-check as a string-valued fallback so raw driver attributes remain reachable.
|
||||
|
||||
A key that names neither an accepted selector key nor an attribute some element on screen carries fails the run, naming the key and the accepted list. Such a key can never match, and an empty result is indistinguishable from a screen with no matching element: the generator declines to act, the runner waits out the step, and the run ends clean having explored nothing. The string form keeps its open kind space, since `<attr>:<value>` is the documented way to reach a raw driver attribute.
|
||||
|
||||
@@ -126,6 +126,7 @@ Fields available on every element returned by `find` / `findAll`:
|
||||
| `checked` | `boolean` | Checkbox or toggle state |
|
||||
| `focused` | `boolean` | Element has input focus |
|
||||
| `selected` | `boolean` | Selection state |
|
||||
| `secure` | `boolean \| null` | Field masks what is typed into it; `null` where the platform does not report it (Android never does) |
|
||||
| `bounds` | `{ left, top, right, bottom }` | Bounding box in device pixels |
|
||||
| `x` | `number` | Center X (derived from bounds) |
|
||||
| `y` | `number` | Center Y (derived from bounds) |
|
||||
|
||||
@@ -15,6 +15,7 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/priyanshujain/sanderling/internal/hierarchy"
|
||||
"github.com/priyanshujain/sanderling/internal/llmclient"
|
||||
"github.com/priyanshujain/sanderling/internal/trace"
|
||||
"github.com/priyanshujain/sanderling/internal/verifier"
|
||||
@@ -107,7 +108,7 @@ func (s *llmSource) NextAction(ctx context.Context, stepIndex int) (verifier.Act
|
||||
if err == nil {
|
||||
s.lastFromSetup = true
|
||||
s.record(stepIndex, trace.LLMCall{Outcome: trace.LLMOutcomeSetupAction})
|
||||
s.history.add(describeAction(action))
|
||||
s.history.add(describeAction(action, s.verifier.Tree()))
|
||||
return action, nil
|
||||
}
|
||||
if !errors.Is(err, verifier.ErrNoAction) {
|
||||
@@ -129,7 +130,7 @@ func (s *llmSource) NextAction(ctx context.Context, stepIndex int) (verifier.Act
|
||||
s.lastReasoning = selection.reasoning
|
||||
s.lastChoice = selection.choice
|
||||
s.lastChosenAction = selection.chosenAction
|
||||
s.history.add(describeAction(selection.action))
|
||||
s.history.add(describeAction(selection.action, s.verifier.Tree()))
|
||||
return selection.action, nil
|
||||
}
|
||||
|
||||
@@ -427,10 +428,13 @@ func parseChoice(content string) (choiceOutput, error) {
|
||||
}
|
||||
|
||||
// describeAction renders a short action summary for the recent-action memory.
|
||||
func describeAction(action verifier.Action) string {
|
||||
// The tree is the one the action was chosen against, which is what says whether
|
||||
// a typed value may be written into the memory at all.
|
||||
func describeAction(action verifier.Action, tree *hierarchy.Tree) string {
|
||||
switch action.Kind {
|
||||
case verifier.ActionKindInputText:
|
||||
return fmt.Sprintf("InputText %s = %q", actionTarget(action), action.Text)
|
||||
return fmt.Sprintf("InputText %s = %q",
|
||||
actionTarget(action), verifier.RecordedActionText(action, tree))
|
||||
case verifier.ActionKindScroll:
|
||||
// A builtin gesture carries endpoints rather than a selector, so name the
|
||||
// container by where the drag starts; that is what tells two scrollable
|
||||
|
||||
@@ -75,11 +75,11 @@ func TestDescribeActionNamesGesturesByOrigin(t *testing.T) {
|
||||
Kind: verifier.ActionKindScroll, Direction: "down",
|
||||
FromX: 200, FromY: 500, ToX: 200, ToY: 340,
|
||||
}
|
||||
if got := describeAction(builtin); got != "Scroll down (200,500)" {
|
||||
if got := describeAction(builtin, nil); got != "Scroll down (200,500)" {
|
||||
t.Errorf("builtin gesture described as %q, want the drag origin", got)
|
||||
}
|
||||
authored := verifier.Action{Kind: verifier.ActionKindScroll, Direction: "up", On: "id:List"}
|
||||
if got := describeAction(authored); got != "Scroll up id:List" {
|
||||
if got := describeAction(authored, nil); got != "Scroll up id:List" {
|
||||
t.Errorf("authored scroll described as %q, want its selector", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,180 @@
|
||||
package runner
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
mockdriver "github.com/priyanshujain/sanderling/internal/driver/mock"
|
||||
|
||||
"github.com/priyanshujain/sanderling/internal/verifier"
|
||||
)
|
||||
|
||||
// typedCredential stands in for what a login setup types. Nothing the run
|
||||
// records or sends may carry it.
|
||||
const typedCredential = "fixture-passphrase-9f21"
|
||||
|
||||
// iosLoginTreeJSON is the shape internal/driver/ioscompanion produces for a
|
||||
// login form: every editable field reports `secure`, so a field carrying
|
||||
// `secure:false` is positively known not to be a credential entry.
|
||||
const iosLoginTreeJSON = `{
|
||||
"attributes": {"bounds": "[0,0,390,844]", "class": "Window"},
|
||||
"children": [
|
||||
{"attributes": {"identifier": "LoginEmail", "class": "TextField", "hintText": "Email", "bounds": "[20,200,370,244]"},
|
||||
"editable": true, "enabled": true, "secure": false, "children": []},
|
||||
{"attributes": {"identifier": "LoginPassword", "class": "SecureTextField", "hintText": "Password", "bounds": "[20,260,370,304]"},
|
||||
"editable": true, "enabled": true, "secure": true, "children": []}
|
||||
]
|
||||
}`
|
||||
|
||||
// webLoginTreeJSON is the same form as the chrome dump reports it.
|
||||
const webLoginTreeJSON = `{
|
||||
"attributes": {"bounds": "[0,0,1280,800]", "tag": "html"},
|
||||
"children": [
|
||||
{"attributes": {"resource-id": "login-email", "tag": "input", "hintText": "Email", "bounds": "[20,200,400,240]"},
|
||||
"editable": true, "enabled": true, "secure": false, "children": []},
|
||||
{"attributes": {"resource-id": "login-password", "tag": "input", "hintText": "Password", "bounds": "[20,260,400,300]"},
|
||||
"editable": true, "enabled": true, "secure": true, "children": []}
|
||||
]
|
||||
}`
|
||||
|
||||
// androidLoginTreeJSON is the same form on Android, where the native tree
|
||||
// mapper drops uiautomator's password attribute and neither field carries the
|
||||
// fact.
|
||||
const androidLoginTreeJSON = `{
|
||||
"attributes": {"bounds": "[0,0,1080,2340]"},
|
||||
"children": [
|
||||
{"attributes": {"resource-id": "login_email", "class": "android.widget.EditText", "hintText": "Email", "bounds": "[20,200,1060,300]"},
|
||||
"enabled": true, "children": []},
|
||||
{"attributes": {"resource-id": "login_password", "class": "android.widget.EditText", "hintText": "Password", "bounds": "[20,320,1060,420]"},
|
||||
"enabled": true, "children": []}
|
||||
]
|
||||
}`
|
||||
|
||||
func typeInto(selector string) verifier.Action {
|
||||
return verifier.Action{Kind: verifier.ActionKindInputText, On: selector, Text: typedCredential}
|
||||
}
|
||||
|
||||
func TestTraceActionForRedactsTypedValuesTheTargetCannotClear(t *testing.T) {
|
||||
for _, testCase := range []struct {
|
||||
name string
|
||||
treeJSON string
|
||||
selector string
|
||||
}{
|
||||
{"ios secure field", iosLoginTreeJSON, "id:LoginPassword"},
|
||||
{"web secure field", webLoginTreeJSON, "id:login-password"},
|
||||
{"android field reported as neither", androidLoginTreeJSON, "id:login_email"},
|
||||
{"android password field", androidLoginTreeJSON, "id:login_password"},
|
||||
} {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
traceAction := traceActionFor(typeInto(testCase.selector), mustParseTree(t, testCase.treeJSON))
|
||||
if traceAction.Text != verifier.RedactedInputText {
|
||||
t.Errorf("trace action text = %q, want it redacted", traceAction.Text)
|
||||
}
|
||||
if traceAction.Selector != testCase.selector {
|
||||
t.Errorf("trace selector = %q, want %q so the record still shows which field was typed into",
|
||||
traceAction.Selector, testCase.selector)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestTraceActionForKeepsTypedValuesForAFieldReportedNotSecure(t *testing.T) {
|
||||
for _, testCase := range []struct {
|
||||
name string
|
||||
treeJSON string
|
||||
selector string
|
||||
}{
|
||||
{"ios", iosLoginTreeJSON, "id:LoginEmail"},
|
||||
{"web", webLoginTreeJSON, "id:login-email"},
|
||||
} {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
const address = "[email protected]"
|
||||
action := verifier.Action{Kind: verifier.ActionKindInputText, On: testCase.selector, Text: address}
|
||||
traceAction := traceActionFor(action, mustParseTree(t, testCase.treeJSON))
|
||||
if traceAction.Text != address {
|
||||
t.Errorf("trace action text = %q, want the typed value on a field reported not secure", traceAction.Text)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The redaction is a rendering rule, never a change to what is dispatched: the
|
||||
// app has to receive the keystrokes a user would have produced.
|
||||
func TestApplyActionTypesTheRealValueIntoEveryField(t *testing.T) {
|
||||
for _, testCase := range []struct {
|
||||
name string
|
||||
treeJSON string
|
||||
selector string
|
||||
}{
|
||||
{"ios secure field", iosLoginTreeJSON, "id:LoginPassword"},
|
||||
{"web secure field", webLoginTreeJSON, "id:login-password"},
|
||||
{"android field", androidLoginTreeJSON, "id:login_password"},
|
||||
} {
|
||||
t.Run(testCase.name, func(t *testing.T) {
|
||||
fastFocusSettle(t)
|
||||
driverMock := mockdriver.New()
|
||||
mustDispatch(t, driverMock, typeInto(testCase.selector), mustParseTree(t, testCase.treeJSON))
|
||||
|
||||
typed := ""
|
||||
for _, recorded := range driverMock.Actions() {
|
||||
if recorded.Kind == mockdriver.ActionInputText {
|
||||
typed = recorded.Text
|
||||
}
|
||||
}
|
||||
if typed != typedCredential {
|
||||
t.Errorf("driver received %q, want the real typed value", typed)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
const secureLoginSpec = `
|
||||
import { llm, always, actions, InputText, taps, typing, weighted } from "@sanderling/spec";
|
||||
globalThis.properties = { ok: always(() => true) };
|
||||
globalThis.setup = actions(() => {
|
||||
if (globalThis.__typed) return [];
|
||||
globalThis.__typed = true;
|
||||
return [InputText({ into: "id:LoginPassword", text: "` + typedCredential + `" })];
|
||||
});
|
||||
globalThis.actions = weighted([1, taps], [1, typing]);
|
||||
globalThis.generator = llm({ model: "test/model" });
|
||||
`
|
||||
|
||||
// The recorded call is llm-calls.jsonl: its user prompt carries the
|
||||
// recent-action memory and its candidates carry the numbered list, which is
|
||||
// where a login run put the account password in cleartext beside a screenshot
|
||||
// of the same screen.
|
||||
func TestLLMCallRecordKeepsTypedSecretsOutOfThePromptAndCandidates(t *testing.T) {
|
||||
fake := newFakeOpenRouter(t)
|
||||
source, verifierInstance := newLLMSourceWithSpec(t, fake, secureLoginSpec)
|
||||
|
||||
pushSnapshotTree(t, verifierInstance, iosLoginTreeJSON)
|
||||
action, err := source.NextAction(context.Background(), 0)
|
||||
if err != nil {
|
||||
t.Fatalf("setup NextAction: %v", err)
|
||||
}
|
||||
if action.Text != typedCredential {
|
||||
t.Fatalf("setup action text = %q, want the real value dispatched to the app", action.Text)
|
||||
}
|
||||
|
||||
pushSnapshotTree(t, verifierInstance, iosLoginTreeJSON)
|
||||
fake.choice = 1
|
||||
fake.chosenAction = mustCandidates(t, verifierInstance, verifier.LabelSourceVisibleText)[0].Description
|
||||
if _, err := source.NextAction(context.Background(), 1); err != nil {
|
||||
t.Fatalf("llm NextAction: %v", err)
|
||||
}
|
||||
|
||||
call := lastCall(t, source)
|
||||
if strings.Contains(call.UserPrompt, typedCredential) {
|
||||
t.Error("the recorded user prompt carries the typed value")
|
||||
}
|
||||
if !strings.Contains(call.UserPrompt, "InputText") {
|
||||
t.Errorf("the recorded user prompt lost the recent-action memory entirely: %q", call.UserPrompt)
|
||||
}
|
||||
for _, candidate := range call.Candidates {
|
||||
if strings.Contains(candidate.Description, typedCredential) {
|
||||
t.Errorf("recorded candidate %d carries the typed value: %q", candidate.Index, candidate.Description)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1586,7 +1586,7 @@ func traceActionFor(action verifier.Action, tree *hierarchy.Tree) *trace.Action
|
||||
traceAction.Selector = action.On
|
||||
stampSelectorTarget(traceAction, action, tree)
|
||||
case verifier.ActionKindInputText:
|
||||
traceAction.Text = action.Text
|
||||
traceAction.Text = verifier.RecordedActionText(action, tree)
|
||||
traceAction.Selector = action.On
|
||||
stampSelectorTarget(traceAction, action, tree)
|
||||
case verifier.ActionKindSwipe:
|
||||
|
||||
@@ -43,6 +43,7 @@ const canonicalElement = `{
|
||||
"enabled": true,
|
||||
"focused": false,
|
||||
"id": "TxnAmountField",
|
||||
"secure": null,
|
||||
"selected": false,
|
||||
"text": "199",
|
||||
"x": 200,
|
||||
|
||||
@@ -82,6 +82,12 @@ func (v *Verifier) CurrentScreen() string {
|
||||
return v.lastTree.Elements[0].Screen
|
||||
}
|
||||
|
||||
// Tree returns the hierarchy of the most recent snapshot, nil when none was
|
||||
// pushed. It is what a caller resolves an action's target against.
|
||||
func (v *Verifier) Tree() *hierarchy.Tree {
|
||||
return v.lastTree
|
||||
}
|
||||
|
||||
// SampleInput draws one InputText value from the shared corpus via the bundled
|
||||
// __sanderlingSampleInput__. It errors when the bundle did not install the
|
||||
// callable (a raw-JS fixture) so the caller can skip typing rather than send an
|
||||
@@ -139,6 +145,10 @@ type ActionCandidate struct {
|
||||
// prob is the internal accumulated selection probability, summed across
|
||||
// dedup, then rounded into Weight. Not exposed in the prompt directly.
|
||||
prob float64
|
||||
// secure is what the target reports about being a secure text entry, which
|
||||
// is what Description is rendered under. It is not part of what the model
|
||||
// sees.
|
||||
secure secureFact
|
||||
}
|
||||
|
||||
// maxLabelRunes caps a visible-text label so joined descendant text stays short
|
||||
@@ -402,6 +412,7 @@ func (v *Verifier) candidateFromDescriptor(value goja.Value, labels labelContext
|
||||
Kind: kind,
|
||||
Label: target.label,
|
||||
InputType: target.inputType,
|
||||
secure: target.secure,
|
||||
Action: Action{Kind: kind, On: target.selector, X: target.x, Y: target.y, Text: text},
|
||||
}, true
|
||||
case ActionKindScroll:
|
||||
@@ -464,6 +475,7 @@ type resolvedTarget struct {
|
||||
selector string
|
||||
label string
|
||||
inputType string
|
||||
secure secureFact
|
||||
}
|
||||
|
||||
// resolveTarget reads an authored action's target. Ax element handles carry
|
||||
@@ -498,10 +510,13 @@ func (v *Verifier) resolveTarget(value goja.Value, labels labelContext) (resolve
|
||||
if selector == "" {
|
||||
selector = stringField(object, "selector")
|
||||
}
|
||||
target := resolvedTarget{x: x, y: y, selector: selector}
|
||||
target := resolvedTarget{x: x, y: y, selector: selector, secure: secureFactFromHandle(object)}
|
||||
if element := v.findBySelector(selector); element != nil {
|
||||
target.label = labels.label(element)
|
||||
target.inputType = inputTypeHint(element)
|
||||
if !target.secure.reported {
|
||||
target.secure = secureFactOf(element)
|
||||
}
|
||||
}
|
||||
if target.label == "" {
|
||||
target.label = truncateLabel(v.handleLabel(object, labels))
|
||||
@@ -519,6 +534,7 @@ func (v *Verifier) targetFromSelector(selector string, labels labelContext) reso
|
||||
target.x, target.y = element.Bounds.Center()
|
||||
target.label = labels.label(element)
|
||||
target.inputType = inputTypeHint(element)
|
||||
target.secure = secureFactOf(element)
|
||||
return target
|
||||
}
|
||||
|
||||
@@ -556,6 +572,7 @@ func (v *Verifier) collectBuiltin(verb string, prob float64, weighted bool, labe
|
||||
element := targets[entry.targetIndex].element
|
||||
candidate.Label = labels.label(element)
|
||||
candidate.InputType = inputTypeHint(element)
|
||||
candidate.secure = secureFactOf(element)
|
||||
}
|
||||
*out = append(*out, candidate)
|
||||
}
|
||||
@@ -664,7 +681,8 @@ func describeCandidate(candidate ActionCandidate) string {
|
||||
}
|
||||
return fmt.Sprintf("Type into %q", candidate.Label)
|
||||
}
|
||||
return fmt.Sprintf("Type %q into %q", candidate.Action.Text, candidate.Label)
|
||||
return fmt.Sprintf("Type %q into %q",
|
||||
recordedInputText(candidate.Action.Text, candidate.secure), candidate.Label)
|
||||
case ActionKindScroll:
|
||||
return "Scroll " + candidate.Direction
|
||||
case ActionKindSwipe:
|
||||
|
||||
@@ -425,7 +425,9 @@ func TestCandidatesCallsAuthoredLeafOnce(t *testing.T) {
|
||||
t.Errorf("authored action on a disabled control was dropped: %v", descriptions(candidates))
|
||||
}
|
||||
// Authored InputText replays its own sampled value (LLM does not supply it).
|
||||
authored, ok := findCandidate(candidates, `Type "42" into "Amount"`)
|
||||
// The fixture is an android tree, which reports no secure fact, so the
|
||||
// rendered value is redacted while the action still carries it.
|
||||
authored, ok := findCandidate(candidates, `Type "[redacted]" into "Amount"`)
|
||||
if !ok {
|
||||
t.Fatalf("authored typing missing: %v", descriptions(candidates))
|
||||
}
|
||||
@@ -747,8 +749,8 @@ func TestCandidatesAcceptASingleItemAuthoredSampler(t *testing.T) {
|
||||
const webFieldTreeJSON = `{
|
||||
"attributes": {"tag": "html", "bounds": "[0,0,400,800]"},
|
||||
"children": [
|
||||
{"attributes": {"resource-id": "txn-amount", "tag": "input", "class": "input amount-input", "bounds": "[0,100,400,160]"}, "editable": true, "enabled": true, "children": []},
|
||||
{"attributes": {"resource-id": "txn-note", "tag": "input", "class": "input", "bounds": "[0,200,400,260]"}, "editable": true, "enabled": true, "children": []}
|
||||
{"attributes": {"resource-id": "txn-amount", "tag": "input", "class": "input amount-input", "bounds": "[0,100,400,160]"}, "editable": true, "enabled": true, "secure": false, "children": []},
|
||||
{"attributes": {"resource-id": "txn-note", "tag": "input", "class": "input", "bounds": "[0,200,400,260]"}, "editable": true, "enabled": true, "secure": false, "children": []}
|
||||
]
|
||||
}`
|
||||
|
||||
@@ -761,14 +763,14 @@ const webFieldTreeJSON = `{
|
||||
func TestCandidatesNameWebAuthoredFieldsByTheirHint(t *testing.T) {
|
||||
const amountHandle = `{
|
||||
"id": "txn-amount", "text": "", "desc": "", "class": "input amount-input",
|
||||
"clickable": true, "enabled": true, "editable": true, "focused": false,
|
||||
"clickable": true, "enabled": true, "editable": true, "focused": false, "secure": false,
|
||||
"x": 200, "y": 130, "bounds": {"left": 0, "top": 100, "right": 400, "bottom": 160},
|
||||
"attrs": {"tag": "input", "aria-label": "", "id": "txn-amount",
|
||||
"class": "input amount-input", "placeholder": "0.00", "hintText": "Amount"}
|
||||
}`
|
||||
const noteHandle = `{
|
||||
"id": "txn-note", "text": "", "desc": "", "class": "input",
|
||||
"clickable": true, "enabled": true, "editable": true, "focused": false,
|
||||
"clickable": true, "enabled": true, "editable": true, "focused": false, "secure": false,
|
||||
"x": 200, "y": 230, "bounds": {"left": 0, "top": 200, "right": 400, "bottom": 260},
|
||||
"attrs": {"tag": "input", "aria-label": "", "id": "txn-note", "class": "input",
|
||||
"placeholder": "What's this for?", "hintText": "Note (optional)"}
|
||||
@@ -807,7 +809,7 @@ func TestCandidatesNameWebAuthoredFieldsByTheirHint(t *testing.T) {
|
||||
// The identifier arm must stay blind to anything a user reads, hint included.
|
||||
func TestCandidatesNameWebAuthoredFieldsByIdentifierOnThatArm(t *testing.T) {
|
||||
const amountHandle = `{
|
||||
"id": "txn-amount", "text": "", "editable": true, "enabled": true,
|
||||
"id": "txn-amount", "text": "", "editable": true, "enabled": true, "secure": false,
|
||||
"x": 200, "y": 130,
|
||||
"attrs": {"tag": "input", "hintText": "Amount"}
|
||||
}`
|
||||
|
||||
@@ -124,6 +124,14 @@ func nodeObject(runtime *goja.Runtime, tree *hierarchy.Tree, node *hierarchy.Nod
|
||||
_ = object.Set("focused", element.Focused)
|
||||
_ = object.Set("selected", element.Selected)
|
||||
_ = object.Set("editable", element.Editable)
|
||||
// Three-valued, unlike the other state flags: null where the platform
|
||||
// reported nothing at all, which is what separates an ordinary field from a
|
||||
// password field on a platform that cannot tell them apart.
|
||||
var secure any
|
||||
if element.SecureReported() {
|
||||
secure = element.Secure
|
||||
}
|
||||
_ = object.Set("secure", secure)
|
||||
_ = object.Set("x", centerX)
|
||||
_ = object.Set("y", centerY)
|
||||
_ = object.Set(tagSelector, unambiguousSelector(tree, node, selector))
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
package verifier
|
||||
|
||||
import (
|
||||
"github.com/dop251/goja"
|
||||
|
||||
"github.com/priyanshujain/sanderling/internal/hierarchy"
|
||||
)
|
||||
|
||||
// RedactedInputText stands in for a typed value in every record. It is fixed,
|
||||
// so it gives away neither the value nor its length.
|
||||
const RedactedInputText = "[redacted]"
|
||||
|
||||
// secureFact is what a target says about being a secure text entry. `reported`
|
||||
// separates "the platform says this is not one" from "the platform says
|
||||
// nothing", which are the two cases the redaction rule has to tell apart.
|
||||
type secureFact struct {
|
||||
reported bool
|
||||
secure bool
|
||||
}
|
||||
|
||||
// secureFactFromHandle reads an element handle's own report. The web host
|
||||
// injects handles built in the page, which carry no selector to resolve against
|
||||
// the goja-side tree, so the handle is the only thing that knows.
|
||||
func secureFactFromHandle(object *goja.Object) secureFact {
|
||||
value := object.Get("secure")
|
||||
if value == nil || goja.IsUndefined(value) || goja.IsNull(value) {
|
||||
return secureFact{}
|
||||
}
|
||||
return secureFact{reported: true, secure: value.ToBoolean()}
|
||||
}
|
||||
|
||||
func secureFactOf(element *hierarchy.Element) secureFact {
|
||||
if element == nil {
|
||||
return secureFact{}
|
||||
}
|
||||
return secureFact{reported: element.SecureReported(), secure: element.Secure}
|
||||
}
|
||||
|
||||
// RecordedActionText renders the typed value of an action for anything that is
|
||||
// persisted or sent, resolving the action's target in the tree it was chosen
|
||||
// against.
|
||||
func RecordedActionText(action Action, tree *hierarchy.Tree) string {
|
||||
if action.Kind != ActionKindInputText {
|
||||
return action.Text
|
||||
}
|
||||
var target *hierarchy.Element
|
||||
if tree != nil && action.On != "" {
|
||||
target = tree.Find(action.On)
|
||||
}
|
||||
return recordedInputText(action.Text, secureFactOf(target))
|
||||
}
|
||||
|
||||
// recordedInputText is the one place a typed value is rendered for a record.
|
||||
// The prompt's recent-action memory, the numbered candidate list and the trace
|
||||
// all go through it, so a fourth record added later cannot publish a value the
|
||||
// other three withhold. The driver dispatch reads Action.Text directly and is
|
||||
// the only reader of the real value, which is what keeps the app receiving the
|
||||
// keystrokes a user would have produced.
|
||||
//
|
||||
// A target the platform reports as a secure entry is redacted, and so is a
|
||||
// target carrying no report at all. iOS and web state the fact on every
|
||||
// editable element, so a missing one means Android, whose native tree mapper
|
||||
// drops uiautomator's password attribute before the sidecar sees it. There a
|
||||
// password field cannot be told from a search box, and the target that cannot
|
||||
// be told apart is treated as the credential.
|
||||
func recordedInputText(text string, target secureFact) string {
|
||||
if target.reported && !target.secure {
|
||||
return text
|
||||
}
|
||||
return RedactedInputText
|
||||
}
|
||||
@@ -0,0 +1,111 @@
|
||||
package verifier
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// typedCredential stands in for what a login setup types. Nothing rendered for
|
||||
// a record may carry it.
|
||||
const typedCredential = "fixture-passphrase-9f21"
|
||||
|
||||
// secureLoginTreeJSON is the shape iOS and web produce for a login form: both
|
||||
// report `secure` on every editable field, so a field carrying `secure:false`
|
||||
// is positively known not to be a credential entry.
|
||||
const secureLoginTreeJSON = `{
|
||||
"attributes": {"bounds": "[0,0,390,844]", "class": "Window"},
|
||||
"children": [
|
||||
{"attributes": {"identifier": "LoginEmail", "class": "TextField", "hintText": "Email", "bounds": "[20,200,370,244]"},
|
||||
"editable": true, "enabled": true, "secure": false, "children": []},
|
||||
{"attributes": {"identifier": "LoginPassword", "class": "SecureTextField", "hintText": "Password", "bounds": "[20,260,370,304]"},
|
||||
"editable": true, "enabled": true, "secure": true, "children": []}
|
||||
]
|
||||
}`
|
||||
|
||||
// androidLoginTreeJSON is the same form as Android reports it. The native tree
|
||||
// mapper drops uiautomator's password attribute, so neither field carries the
|
||||
// fact and the two are indistinguishable.
|
||||
const androidLoginTreeJSON = `{
|
||||
"attributes": {"bounds": "[0,0,1080,2340]"},
|
||||
"children": [
|
||||
{"attributes": {"resource-id": "login_email", "class": "android.widget.EditText", "hintText": "Email", "bounds": "[20,200,1060,300]"},
|
||||
"enabled": true, "children": []},
|
||||
{"attributes": {"resource-id": "login_password", "class": "android.widget.EditText", "hintText": "Password", "bounds": "[20,320,1060,420]"},
|
||||
"enabled": true, "children": []}
|
||||
]
|
||||
}`
|
||||
|
||||
func authoredTypingCandidates(t *testing.T, selector, text, treeJSON string) []ActionCandidate {
|
||||
t.Helper()
|
||||
actions := `{kind:'actions', generate: () => [{kind:'InputText', into:'` + selector +
|
||||
`', text:'` + text + `'}]}`
|
||||
return mustCandidates(t, enumVerifier(t, actions, treeJSON), LabelSourceVisibleText)
|
||||
}
|
||||
|
||||
func TestCandidatesRedactTextTypedIntoASecureField(t *testing.T) {
|
||||
candidates := authoredTypingCandidates(t, "id:LoginPassword", typedCredential, secureLoginTreeJSON)
|
||||
if len(candidates) != 1 {
|
||||
t.Fatalf("candidates = %v, want the one authored typing action", descriptions(candidates))
|
||||
}
|
||||
candidate := candidates[0]
|
||||
if !strings.Contains(candidate.Description, RedactedInputText) {
|
||||
t.Errorf("candidate description = %q, want the typed value redacted", candidate.Description)
|
||||
}
|
||||
if !strings.Contains(candidate.Description, "Password") {
|
||||
t.Errorf("candidate description = %q, want it to still name the field typed into", candidate.Description)
|
||||
}
|
||||
if candidate.Action.Text != typedCredential {
|
||||
t.Errorf("candidate action text = %q, want the real value so the driver still types it", candidate.Action.Text)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCandidatesRedactEveryTypedValueWhereTheTargetCannotReportSecure(t *testing.T) {
|
||||
for _, selector := range []string{"id:login_email", "id:login_password"} {
|
||||
candidates := authoredTypingCandidates(t, selector, typedCredential, androidLoginTreeJSON)
|
||||
if len(candidates) != 1 {
|
||||
t.Fatalf("%s: candidates = %v, want the one authored typing action", selector, descriptions(candidates))
|
||||
}
|
||||
if strings.Contains(candidates[0].Description, typedCredential) {
|
||||
t.Errorf("%s: candidate description carries the typed value: %q", selector, candidates[0].Description)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCandidatesKeepTextTypedIntoAFieldReportedNotSecure(t *testing.T) {
|
||||
const address = "[email protected]"
|
||||
candidates := authoredTypingCandidates(t, "id:LoginEmail", address, secureLoginTreeJSON)
|
||||
if len(candidates) != 1 {
|
||||
t.Fatalf("candidates = %v, want the one authored typing action", descriptions(candidates))
|
||||
}
|
||||
if !strings.Contains(candidates[0].Description, address) {
|
||||
t.Errorf("candidate description = %q, want the typed value on a field reported not secure", candidates[0].Description)
|
||||
}
|
||||
}
|
||||
|
||||
// A spec that types into an element HANDLE is the shape every login setup has
|
||||
// (examples/folio/sanderling/spec.ts). The handle carries the goja host's own
|
||||
// `secure` field, which is a plain boolean and reads false on a platform that
|
||||
// reports nothing, so a rule trusting it would leave every Android value in the
|
||||
// clear. The tree the handle names is the only carrier of the three-way fact.
|
||||
func TestCandidatesRedactTextTypedIntoAnAndroidElementHandle(t *testing.T) {
|
||||
v := newVerifier(t)
|
||||
loadActionSpec(t, v, `
|
||||
import { InputText, actions, extract } from "@sanderling/spec";
|
||||
const field = extract((s) => s.ax.find({ "resource-id": "login_password" })).named("field");
|
||||
globalThis.actions = actions(() =>
|
||||
field.current ? [InputText({ into: field.current, text: "`+typedCredential+`" })] : []);
|
||||
`)
|
||||
pushTree(t, v, androidLoginTreeJSON)
|
||||
|
||||
candidates := mustCandidates(t, v, LabelSourceVisibleText)
|
||||
if len(candidates) != 1 {
|
||||
t.Fatalf("candidates = %v, want the one authored typing action", descriptions(candidates))
|
||||
}
|
||||
if strings.Contains(candidates[0].Description, typedCredential) {
|
||||
t.Errorf("candidate description carries the typed value: %q", candidates[0].Description)
|
||||
}
|
||||
if candidates[0].Action.Text != typedCredential {
|
||||
t.Errorf("candidate action text = %q, want the real value so the driver still types it",
|
||||
candidates[0].Action.Text)
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user