Walkthroughs — Asserted Screenshot Flows

The Short Version

A walkthrough drives a UI flow, asserts on every page, screenshots each step, and publishes an immutable page showing what shipped. There are two ways to run one and they use the same spec:

# One-off: capture a feature you just shipped, against the live service. No deploy.
tawa walkthrough my-feature.yaml --prod -s my-svc --label pr-123

# Standing tour: declare it in catalog-info.yaml (catalog 0.7.0+) and it runs on every deploy.

Both print a link: Walkthrough: https://iec-test.tawa.pro/walkthroughs/{service}/{env}/{buildId}/index.html

Which one to use

A feature-specific step does not belong in the catalog unless that screen is permanently part of the product tour. If it is in the catalog it runs on every deploy, forever.

The spec

title: "Collections & agency assignment"
auth: portal-editor                  # optional — a NAMED profile on iec-test, never a token
viewport: { width: 1440, height: 900, scale: 2 }   # optional, this is the default
steps:
  - path: /
    expect: { h1: "Dashboard" }
    note: "The console lands here. There is no /dashboard route — the dashboard IS the root."

  - path: /collections/assign
    expect:
      h1: "Assign Collections"           # structural: right page
      "table tbody tr": "ACME"           # substantive: the page has the data
    note: "Assignment now writes the collector's agency (#46)."
    act: [{ click: "button:has-text('Scheduled reports')" }]
    shot: { selector: "[data-testid='panel']" }
    annotate:
      - at: "table thead th:last-child"
        text: "New agency column"
    redact: ["[data-pii]"]

In a catalog the same object goes under spec.walkthrough, with insureco.io/catalog-version: "0.7.0".

The rules that make it trustworthy

1. Every step needs expect. A step without one is rejected — by the CLI's builder call, by the catalog parser (the build fails before an image is built, naming steps[N]), and by iec-test. A screenshot without an assertion is not evidence.
2. A failed step is never published as a clean screenshot. It renders as a failure card with expected-vs-actual, and no image is written.
3. Never write a route or a heading from memory. Read it from the source: the route file, the page component, the actual <h1>. Three of the first four paths written for this system from memory were wrong, and every one returned HTTP 200 with a plausible page (/orders/failed matched /orders/[id]). The assertion is what caught them.
4. Assert on data, not just chrome. h1 proves you are on the right page. It does not prove the page has content: a session can authenticate and still render an empty table. For any step whose value depends on records, add a record-bearing selector (a row, a cell, a total).
5. expect is a substring match on the element's text. Pick text that is stable — a heading or a seeded record — not a timestamp or a count that changes.
   • Several matches: any one may contain it. "table tbody tr": "ACME" means "some row says ACME". To assert on one particular element, say which: "table tbody tr >> nth=0".
   • Only what a person can see is read. A hidden element never makes a step pass, and head > title is always hidden. A portal page shows its title in an input, not an h1: assert on a heading in its body.
   • Case counts. "Orders" does not match orders.
6. Redact before you publish. Anyone with the URL can read a bundle. Blur names, emails, account numbers with redact; leave amounts, dates and statuses legible. Prefer seeded data.

What a failed expect is telling you

The message names which of five things happened. They send you to five different places.

Before 2026-09-29 a selector that matched several elements, or one that was hidden, failed as "…" does not contain "…" with your text plainly inside the actual (insureco/iec-test#50). The workaround was >> nth=0 on every selector. It still works and is no longer needed.

Gating

Which host a run drives

This is not a detail. Both hostnames serve the same deployment, so every status check stays green either way — but an authenticated app's cookie host, OAuth callback and login redirect are pinned to one of them. auth: injects the session cookie scoped to the host being driven, so driving the other renders signed out and fails every assertion with nothing saying why.

On-demand runs resolve the custom domain as of iec-builder#154. The resolved URL is in the CLI's output and in the run's audit record — read it first when a run that should be signed in is not.

Deploy-time runs still use the internal URL, which is right for smoke and e2e (it stops a new service's gate from waiting on DNS) and wrong for an authenticated walkthrough. A run carries one target for all its suites, so separating them needs iec-test (insureco/iec-test#39). Until then the build log names both URLs; a deploy-time walkthrough of an app whose session is bound to its public host will capture signed-out pages, so run that one on demand.

Choosing the host: --host

A service that serves several sites from one deployment has one default and many sites. hypesites, for example, serves insureco.ai, policyos.io, chaac.ai, askjefferson.com and hypesites.tawa.pro/<slug>/, and its default is whichever verified domain was registered first. Name the one you want:

tawa walkthrough site.yaml -s hypesites --prod --host policyos.io
tawa walkthrough site.yaml -s hypesites --prod --host hypesites.tawa.pro   # paths start /<slug>/

• --host takes a bare hostname. A scheme, path or port is refused by the CLI before anything is sent.
• The builder accepts only the service's platform hostname or one of its verified custom domains for that environment. Anything else is 400 HOST_NOT_ALLOWED, and the message lists the hosts that are allowed. An unverified domain, a domain registered for another environment, and another service's host are all refused. No arbitrary URL can be driven.
• Without --host, the default above applies unchanged.
• The API field is host on POST /services/:name/walkthrough. Requires tawa 0.30.0+ (iec-builder#216, tawa-cli#147).

Before this, an on-demand run of hypesites always landed on insureco.ai, and every step of a walkthrough meant for another site failed against its "Not found" page.

Auth

auth: names a profile configured on iec-test (WALKTHROUGH_AUTH_PROFILES). It is always a name — a token in a spec is rejected at parse time, and manifest.auth records the name only. A spec with no auth runs signed out; a spec naming a profile that is not configured fails the run.

Two profile kinds matter on the platform, and they are not equivalent:

kind: "persona" — drive the app as a real user

"portal-editor": { "kind": "persona", "org": "insureco", "persona": "portal-editor" }

auth: portal-editor       # still a profile NAME

iec-test mints in two hops. It gets its own client_credentials token requesting the Bio-ID scope personas:mint, then calls POST {BIO_ID_URL}/api/personas/{org}/{persona}/token with { audience }. Bio-ID returns an ordinary RS256 user token — same claim builder as a real login — which iec-test injects as the app's session cookie, scoped to the host being driven, path=/, httpOnly, secure on https.

The scope is granted once, by an administrator, on iec-test's OAuth client (allowedScopes). Without the grant every persona run fails with MISSING_SCOPE and says so. Every other Bio-ID refusal — PERSONA_NOT_FOUND, PERSONA_NOT_ACTIVE, UNKNOWN_AUDIENCE, ORG_NOT_FOUND, NOT_A_MEMBER, RATE_LIMITED, PERSONA_CANNOT_MINT — fails the run with a message saying what to do, rather than photographing signed-out pages.

The token lives 3 hours with no refresh. iec-test caches one per (client, org, persona, audience) until five minutes before expiry and never writes one to disk.

The builder supplies the audience and the cookie name

A persona token is only accepted if its aud is the target service's own OAuth client, and the cookie is only read if it has the name that service looks for. Neither is guessable, and both live in the builder. So the builder sends them in the run payload as walkthrough.session = { clientId, cookieName }:

The builder attaches the block on both paths — the deploy gate and tawa walkthrough — and omits it when the service has no OAuth client. The same two facts are readable directly at GET /services/:name/session-info?env=… (viewer role; a client id and a public hostname, never a secret).

Declare the cookie in the target service's catalog so nothing has to guess:

spec:
  auth:
    mode: sso
    sessionCookie: portal_access_token   # default when omitted: bio_access_token

A persona can write

A persona is a real org member with real roles, and everything targets prod. An act: step that clicks a destructive control changes real data. Keep act steps read-only unless the flow is meant to change something, give a persona in a real org view-level roles, and put a persona that does things in a dedicated demo org with seeded records.

Provenance. The manifest and the page header record the persona as org/slug — names only. A bundle is readable by anyone with its URL, so it says who the screens were seen as and nothing that could be used to be them again. The token reaches no log line, manifest, page or error string; a test scans all of them for eyJ after every run.

kind: "client-credentials" — the role-less fallback

auth: staff mints a short-lived service token from iec-test's own OAuth client and injects it as bio_access_token. It carries an orgSlug and no roles, so anything behind a super-admin check 403s — assert on screens an ordinary org member can see. It still works for services that have not moved to a persona.

Two sharp edges:

• cookieName is per-app. It defaults to bio_access_token; an app reading its own name renders signed out with nothing logged. WALKTHROUGH_AUTH_PROFILES is one blob shared by every service this runner drives, so give a new app a sibling profile (portal-staff) rather than editing staff, which other teams' specs depend on.
• The app must also accept the token. It is minted for iec-test's client, so a service that pins the aud of its session cookie must allow-list that client id — one exception per service, forever, for a session that can only ever be a role-less nobody. A persona removes the problem instead of working around it.

The orgSlug rule, both kinds. A token with no orgSlug authenticates and then renders empty pages (the OAuth client predates the field). iec-test decodes the token and fails the run rather than capturing blank screens as documentation.

Locally, log in yourself once and reuse the session:

# in the iec-test repo
npx tsx src/cli.ts capture https://my-svc.tawa.pro --out ./data/profiles/staff.json
npx tsx src/cli.ts walkthrough --catalog ../my-svc/catalog-info.yaml \
  --target http://localhost:3000 --local --watch --storage-state ./data/profiles/staff.json

Reading the result

• The page: provenance header (service, environment, build, commit, time), each step with its assertion rendered verbatim, overlays, redaction notes, and a ← Previous build link.
• manifest.json beside it: the machine-readable record (steps[].status, failure.expected, failure.actual, warnings, previousBuildId).
• Build log (deploy runs): Test result: walkthrough: passed (N/N) and Walkthrough: <url>.
• Bundles are immutable and retained: last 10 builds in full, trimmed after 10 days, manifest only after 90, failed captures 7 days.

Finding a walkthrough later

You do not need the link from the build log. iec-test lists every walkthrough its storage still holds at https://iec-test.tawa.pro/ui/walkthroughs.

• Sign in with Bio-ID. Signed out, you are redirected to the login. Access is for people whose token carries org insureco, or super_admin. Anyone else gets a 403, and a service token never passes.
• It reads each bundle's manifest.json only. A bundle with no manifest (an incomplete capture) is skipped, and retention applies: what has been trimmed or deleted is not listed. A run whose page retention has removed shows as expired.
• Bundle URLs are unchanged. .../walkthroughs/{service}/{env}/{buildId}/index.html still opens by URL without signing in, so a link in a PR body keeps working for anyone who has it. Only the index is gated.
• There is no pagination and no logout yet.

What NOT to do

# ❌ no assertion — rejected everywhere
- path: /reports
  note: "The new reports page"

# ❌ chrome only, on a data page — goes green on an empty table
- path: /collections
  expect: { h1: "Collections" }

# ❌ a path from memory
- path: /dashboard          # there is no such route; the dashboard is "/"

# ❌ a token in the spec
auth: eyJhbGciOi...          # rejected — auth is a profile NAME

# ✅
- path: /collections
  expect: { h1: "Collections", "table tbody tr": "ACME" }
  redact: ["td.insured-name", "td.email"]

• Don't add a one-off feature step to the catalog — it will run on every deploy from then on.
• Don't annotate with a selector you haven't checked; it fails soft, so a typo just silently drops the marker (look for it under warnings).
• Don't use shot: full by default — a full-page 2× PNG is ~1 MB versus ~190 KB for a viewport.

See also: Tawa Test — the product walkthroughs are one feature of, and where the other suites and the gating live; Test Personas for the session a persona profile uses; spec.walkthrough in the catalog reference; and the /tawa-walkthrough command for the authoring procedure.