SQUAT User ManualHow testers work

Working with your squad

How testers work

A squad member isn't a script or a screen recording — it's an AI performing a specific persona, using your product through the capabilities the current client provides. The persona may represent a human, be synthetic, or combine permitted sources.

The actor and the evaluator

Every persona/scenario pair in a round runs as two separate passes — two AI contexts with different jobs, on purpose:

  • The actor loads the persona's compiled briefing and becomes that person attempting the scenario — narrating what they see, think, and do, in character, including hesitation, confusion, and giving up (abandoning is real signal, not a failed test).
  • The evaluator is a separate pass that never sees the actor's private reasoning — only the transcript and screenshots the actor produced. It scores impartially: pass, fail, or blocked, with a severity for anything found.

The separation is what keeps scoring impartial — a tester never grades its own homework. An AI that both plays the user and grades the run knows what it meant to do, so it forgives what actually happened. Two passes can mean two different models, and that's the ideal casting: an inexpensive model as the actor (a too-clever actor reads past exactly the confusing labels your real users trip on) and your strongest model as the evaluator, where judgment pays (see Setup for model routing). But one model running both passes as strictly separate conversations works too — the evaluator still never sees the actor's reasoning, and the round records which way it ran.

Provenance-excluding performance bundles

Source governance is visible to authorized operators, not to the performance roles. The actor receives a compiled runtime persona plus the current task or spoken question. A concept-interview moderator receives the participant reference, instrument, and dialogue history. The evaluator receives scoring material without provenance until scoring is complete.

None of those SQUAT-generated bundles contains source identity, basis, permission evidence, raw writing samples, operator commentary, acquisition origin, real/synthetic labels, or another role's private fields. SQUAT records which bundle it returned, but it does not attest that an external model host isolated contexts. Clients with subagents should use separate contexts; other clients should use separate model calls or system contexts.

During a new round, the coordinator records every spoken question, probe, answer, hesitation, interjection, and disclosure in the canonical dialogue ledger. SQUAT reconstructs the ordered exchange from those protected events, including who spoke and what each response addressed.

Transcript origin

The canonical dialogue record notes whether its turns were captured live in the SQUAT round, recovered after an interrupted append, or imported from an archive. That provenance is recorded once on the transcript, not repeated on every line or summary, and never injected into the actor or moderator context.

How a tester sees, and how it clicks

When a browser tool is connected, the actor perceives your product the way agent platforms let AI perceive the web: it reads the page's structure (the accessibility tree and DOM — the same information a screen reader gets: what's a button, what it's labeled, what's focused) and it takes screenshots when appearance matters. It acts by issuing real browser commands — click this element, type into that field, scroll, navigate — against your live product. Real requests hit your real server; the ledger records what actually happened, not a simulation of it.

Two plain consequences of that perception model:

  • Structure is read fluently; pixels need vision. A mislabeled button or a confusing flow surfaces naturally from the structural view. A misaligned layout or a broken style only surfaces through screenshots — which is why the actor captures them at first impression, at every friction point, and anywhere the evaluator will need to judge visually, and why visually-focused scenarios prefer a vision-capable evaluator model. If the evaluator genuinely can't see images, it says so rather than guessing.
  • Personas shape behavior, not eyesight. Maya's phone-first impatience changes what she tries, how long she persists, and what she complains about — it can't literally shrink the viewport. When mobile matters, say so in the scenario ("using the mobile-width view") so the actor sets the viewport accordingly; the round records what actually ran.

With and without a browser

Rounds run in one of two modes, and the ledger records which one actually happened:

Live browser roundNarrative round
What happensThe actor drives your real product — real clicks, real requests, screenshots as evidence.No browser tool is connected, so the actor reasons through the scenario against the product as described (your project context, scenario steps) and says so plainly in the transcript.
Good forEverything — this is the standard. Required for visual findings.Early flow critique, copy review, quick sanity passes — closer to a design panel than a full test.
Watch outNeeds the one-time setup below.Can't find what it can't see: no layout bugs, no real error states, weaker evidence. Never presented as if clicks happened.

The standing rule in every client: use what's actually connected, never invent an interaction that didn't happen. If a scenario needs a browser and none is available, the tester records that plainly instead of pretending.

Setting up browser control (one time, per machine)

Pick the row matching your client. In every case, once the tool is connected the actor discovers and uses it automatically — there's nothing SQUAT-specific to configure.

ClientWhat to set up
Claude app / Claude CodeEasiest: install the Claude in Chrome extension and connect it — the actor gets full, DOM-aware control of a real Chrome. Alternative: the Chrome DevTools route below, if you prefer a separate, dedicated browser instance.
Any MCP client (LM Studio especially)Add a browser MCP server alongside your squat entry — a Chrome DevTools MCP (drives a Chrome you launch with debugging enabled, below) or a Playwright MCP (launches and manages its own browser — no debugger flags needed). LM Studio has none built in; without one of these, its rounds are narrative.
ChatGPT / CodexBrowsing is governed by the session/sandbox configuration, independent of SQUAT — whatever the session may reach, the actor can use.

The Chrome debugger route (technical)

Chrome DevTools–based MCP servers control a Chrome that's been started with its remote debugging port open. Two steps:

1. Launch Chrome with debugging enabled (macOS shown; use a dedicated profile so your personal browsing stays out of test sessions):

/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome \
  --remote-debugging-port=9222 \
  --user-data-dir="$HOME/.squat/chrome-test-profile"

2. Register the DevTools MCP server in your client's MCP config (same file where your squat server is registered), pointed at http://localhost:9222 per that server's documentation.

Warning

An open debugging port means anything on your machine can drive that Chrome instance. Use the dedicated profile, keep the port on localhost (never expose it on a network), and quit that Chrome when you're not testing. Sign the profile into test accounts only — which is also how logins work in rounds: the session is prepared before the tester starts, because no tester ever sees, types, or asks for a password.

What gets recorded

Every round stamps what actually ran: which model played the actor, which judged, what browser control was available, and the product build under test — so a finding is never mistaken for an artifact of the harness, and a quiet round is never mistaken for a healthy product. Connected clients upload screenshots and other files through the hosted artifact library, where SQUAT protects them automatically and keeps them available for later rounds and reports. During a service interruption, a launcher-capable client may preserve an already-authorized turn whose append failed and recover it into the same open panel later — see Running rounds.