Webmaster Tools — iec-webmaster

The Short Version

iec-webmaster is the Tawa service that knows every domain InsurEco owns and what Google thinks of it: the property registry seeded from the Marketing Domain Map, Google Search Console verification through Cloudflare DNS, and (later milestones) search performance, index health, page sources, Topics, Pieces, Links and agency Listings. Backend only, Hono, Mongo, in-process jobs. The portal's Webmaster tile and PolicyOS's Listings module are the front doors; nothing here has a UI.

• Spec, stories and mockups live in the portal, not in the repo: Webmaster Tools — spec v0.3 · user stories · mockups.
• Code and milestones: insureco/iec-webmaster on Forgejo. M1 Registry and verification is what this page describes; M3 Page sources is where a Tawa service joins (below).
• Calling it: declare it in your catalog with the scope you need and present your Bio-ID service token. No key to ask for.

# your catalog-info.yaml
spec:
  dependencies:
    - service: iec-webmaster
      scopes: [webmaster:read]          # → IEC_WEBMASTER_URL, and the scope on your Bio-ID client

curl -H "Authorization: Bearer $SERVICE_TOKEN" "$IEC_WEBMASTER_URL/api/properties?status=verified"

Who may call it (the allowlist)

There is no caller list by name and no shared key. The gate is two claims on a verified Bio-ID token:

The service publishes three scopes in its catalog-info.yaml (spec.modules):

A caller gets a scope by declaring iec-webmaster under its own spec.dependencies with scopes: and redeploying; same-org grants auto-approve (Console → Permissions → API Access). A client created with the CLI rather than a catalog (jci-brain's jci-plane-prod) is granted once by hand:

tawa oauth update jci-plane-prod --scopes "openid,webmaster:read,webmaster:sync"

Per route, requireScope() checks the one scope it needs; a platform admin passes every scope. Tokens are verified against Bio-ID's JWKS in-process, the same way @insureco/bio does it.

Routes (M1)

Base: IEC_WEBMASTER_URL (through Janus). Every response is { success, data | error }; every refusal is { success: false, error: { code, message } }. The OpenAPI document is served at /openapi.json.

All routes are gas: 0. Build clients against /openapi.json, not against this table.

How verification runs

unverified → token-requested → txt-written → verifying → verified
                    │                             └──────→ error   (lastError: source, code, message)
                    └─ nameservers not on Cloudflare: stops here with the TXT value stored → manual path

Google hands out the google-site-verification=… value; the service writes it as a TXT record at the apex through Cloudflare (idempotent, owned by the comment iec-webmaster:google-site-verification, foreign records such as SPF are never touched); polls public DNS until the value shows; asks Google to verify; adds sc-domain:<domain> to Search Console; adds the configured humans (WEBMASTER_OWNER_EMAILS, default [email protected]) as owners. Propagation retries with backoff inside a 60-minute budget. A verified domain is re-checked and DNS is never rewritten. A run stranded by a pod restart is picked up by the verify-resume job (every five minutes) once it has been quiet for longer than the budget; domains parked on the manual path are not resumed. Every transition is stamped on the Property (verification.*At); the coarse status follows at the ends.

Search Console pull (POST /api/search/pull, iec-webmaster#88)

• Poll, do not wait. A back-fill takes minutes, longer than any request should live. Follow poll (GET /api/jobs/:id, kind: "search-pull") until status leaves running. Every run, cron or manual, is a record in the search_pull_runs collection, so any replica answers and the record outlives the run.
• One pull at a time. Asking while one is running is refused with 409 PULL_IN_PROGRESS, carrying error.details.jobId when known. Treat it as "already happening": poll that id.
• Failures live on the job. The old 502 PULL_FAILED is gone; a failed pull is status: "failed" with error on its record.
• A killed pull resumes. A deploy or restart mid-pull leaves its record running. The search-pull-resume job (every 5 minutes, and 45 s after boot) marks a run whose lease has lapsed starts a new pull with trigger resume, pulling only the remainder. The resume job does not mark the stale record failed itself: the new pull abandons it (status: failed, abandoned: true) once that pull holds the lease, so a start that loses the lease abandons nothing and the next tick tries again. It does nothing while the lease is live.

Routes (M3: page sources)

Legacy is the default, and a to-do

A verified Property that no HypeSite or Tawa service has claimed defaults to source.kind: 'legacy' (source.setBy: 'auto', plus setAt), so the weekly legacy crawl covers it. It happens in two places: the verify flow, the moment a Property becomes verified, and a sweep at the start of every legacy-crawl run, which catches rows that were already verified. Septor: webmaster.source.defaulted.

• A person's none is never overridden. POST …/source { kind: "none" } records setBy: 'person'; neither the sweep nor the verify flow nor the on-demand crawl touches it. A row with no setBy (from before this rule) counts as auto.
• A legacy claim cannot overwrite it, even in a race. claimSource writes a legacy claim with a filter ("no source, an auto none, or already legacy"), so a person's none that lands between the read and the write is not overwritten. The claim is refused the same way as the precheck: POST …/crawl answers 409 SOURCE_SET_BY_PERSON.
• Precedence is unchanged. A HypeSite or Tawa service always takes a legacy Property over.
• Legacy is a to-do, not a resting state. Every legacy Property (ignored ones excepted) carries one open WebIssue of kind needs-hypesite: "No HypeSite or Tawa service serves this domain yet; build one". It resolves itself when a HypeSite or service claims the Property, when a person sets none, or when the Property is ignored. The sweep also backfills and reconciles them.
• A person can keep it legacy. See Kept legacy.
• The sweep is lease-aware. It extends the legacy-crawl lease every 20 rows and stops, reporting interrupted, if the lease is lost or the service is stopping. A sweep that fails is logged and recorded on the run (summary.sweepError); the crawl still goes ahead.
• On demand. POST /api/legacy/sweep runs the sweep alone and answers its counts; POST /api/legacy/crawl starts the whole job in the background (202 with a run id, poll GET /api/legacy/runs/:id, 409 while one runs).
• Where to see it. GET /api/overview has needsHypeSite, overall and by family (beside the buckets: it does not change total). GET /api/issues?kind=needs-hypesite lists the sites.

Kept legacy: an old system that needs no HypeSite

Some domains (raterspot.com, policypay.io) are old systems that stay as they are, so the needs-hypesite to-do is wrong for them. A person marks one with POST /api/properties/:domain/source { kind: "legacy", keep: true, reason? } (webmaster:sync). It sets source.keep: true with setBy: person, setAt and setByActor (and keepReason), and it needs a named person: a Bio-ID user token, or a service token acting for one (actingUser); otherwise 403 PERSON_REQUIRED. keep only applies to kind: legacy.

A kept Property:

• is still monitored. The weekly legacy crawl and its issues (sitemap, canonical, schema) run as for any legacy Property;
• never carries needs-hypesite. Marking it resolves an open one; the sweep resolves any stale one and never raises one;
• is never re-defaulted or overwritten by the sweep, the verify flow or a legacy claim: those writes filter on source.keep, so it holds under a race, not just on a read;
• is still taken over by a HypeSite or Tawa service claim, as the precedence says. The keep goes with the old source, and Septor records it: webmaster.source.registered with overrodeKept: true, plus webmaster.source.unkept with via: claimed.

Undo with { kind: "legacy", keep: false }: a normal legacy Property with its to-do again. A person's none also drops the keep (unkept, via: none). Omitting keep on a kept row leaves it kept. GET /api/overview excludes kept rows from needsHypeSite and counts them as keptLegacy, overall and by family, so needsHypeSite + keptLegacy is every legacy Property that is not ignored. Septor: webmaster.source.kept, webmaster.source.unkept.

Placeholders: served today, still needs a HypeSite

Most of the estate's custom domains are served by one Tawa service (policyeco-web serves 42 of them, all but insureco.io) as a stand-in: each is to get its own site (a HypeSite) eventually. They belong on the "needs a HypeSite" list although a service serves them. A person marks one with POST /api/properties/:domain/source { placeholder: true } (webmaster:sync; or POST /api/properties/placeholder-bulk { domains[], placeholder } for many), which sets source.placeholder: true with placeholderSetAt and placeholderSetByActor. It needs a named person (a user token, or a service token acting for one), as keep does; it applies only to a tawa-service or hypesite source (409 NOT_A_SERVED_SOURCE for legacy, none and SiteMaker); and it is sent alone, with no kind.

A placeholder:

• does not change the source. source.kind stays tawa-service: the service keeps serving the domain, its sitemap stays submitted, its Pieces stay;
• carries one open needs-hypesite WebIssue, the same kind legacy uses;
• survives the same service re-registering. POST /properties/upsert-from-service and the deploy hook run on every deploy of the service, and neither clears the flag nor resolves the to-do. The flag is carried inside the write that records the claim, so a person marking the domain while a deploy lands is not lost either;
• is cleared by a different source. A HypeSite, or another service (a different name), claiming the domain drops placeholder and resolves the to-do: the domain got its own site. A person setting none or legacy also drops it;
• is reconciled by the weekly sweep like legacy: it raises a missing to-do and resolves a stale one (flag cleared, or the Property ignored).

needsHypeSite is non-kept legacy plus placeholders (a Property is only ever one of them), placeholders is the placeholder part, both overall and by family: marking 42 placeholders raises needsHypeSite by 42. Undo with { placeholder: false }. Septor: webmaster.source.placeholder, and webmaster.source.unplaceholder with via: person, none or claimed.

The weekly legacy crawl

The legacy-crawl job (Sundays 05:30 UTC) reads every verified Property whose source is legacy, after the sweep above. POST /api/properties/:domain/crawl runs the same crawl for one Property.

1. Read the sitemap: /sitemap.xml, else the first Sitemap: line of robots.txt.
2. No sitemap (a 404, or a 200 that is not one): a shallow crawl from / (depth 2, one host, 200 pages, one request a second or the robots Crawl-delay; robots.txt obeyed, also for the final URL after a redirect).
3. Register the pages as legacy-page Pieces. A partial read (page cap hit, a page or child sitemap that timed out, answered 5xx or 429, or a source that flipped from sitemap to crawl this run) adds and updates Pieces but marks none removed.
4. Check the home page and up to ten sampled pages (every crawled page, for a crawl) for a canonical link and schema.org markup.
5. Raise or resolve WebIssues, one open per (domain, kind): sitemap-error, no-canonical, no-schema. A site that is down leaves the Pieces untouched; a timeout, network failure, 5xx or 429 raises sitemap-error only on the second failing run in a row, a clean 404 at once. platform-copy-indexed is reserved and not raised yet (it needs M4's URL inspection).

Every fetch (pages, robots.txt, sitemap files) resolves the host first and is refused if it is loopback, private, link-local, CGNAT or multicast, and must be https. Septor: webmaster.legacy.crawled. Run records live in legacy_crawl_runs.

What a hook does with each domain

• Every hook answers 202 straight away. A failure never fails the deploy or the publish that triggered it.
• It is idempotent on (domain, version). A non-prod deploy is acknowledged and ignored. A platform hostname (*.tawa.pro) is never processed.
• A HypeSite or Tawa service takes over the Property. Once it does, the legacy crawl stops for that Property.
• For a verified domain, the hook:
  1. reads the sitemap (the reader only fetches from the Property's own domain);
  2. registers Pieces;
  3. submits the sitemap to Search Console;
  4. pings IndexNow with the URLs whose lastmod moved.
• For a domain that is not verified yet, the hook starts the verify flow and the delivery waits.
• Problems become WebIssues (sitemap-error, deploy-hook-failed) in GET /api/issues. The next success resolves them.
• The hourly service-sync job retries pending deliveries. It also reads the builder's GET /seo/services to catch a deploy the hook missed.

The Blog as a page source (iec-webmaster#18)

The Blog Items table is retired. Marketing posts live in the portal's Blog store (kind: 'marketing'), and the hourly piece-sync job (:41) reads every published one from GET {PORTAL_URL}/api/blog/public/marketing?all=1 (iec-portal#943), through Janus with iec-webmaster's own Bio-ID service bearer.

• One blog Piece per post, at its canonicalUrl, with title, publishedAt and blog: { postId: 'brand/slug', brand, author? }.
• The Property is the one that serves the URL's host (the longest Property domain the host is on: blog.policyos.io → policyos.io). A post whose host is no Property, or that has no canonicalUrl, is skipped and counted. No WebIssue is raised.
• No duplicates with the sitemap. A sitemap Piece at the post's URL is upgraded to blog in place: still one row per (propertyId, url). Later sitemap reads, by hook or by crawl, record lastmod and the serving source on it, but never change its kind, title or removal.
• Never deleted. A post that is no longer published gets removedAt. A list that hits the portal's cap of 2000 marks nothing removed.
• A portal outage changes nothing. The run logs it and ends { ok: false }.
• Topic and Clicks 28d live on the Piece. Every Piece carries topicId and clicks28d (number | null), both null until set. clicks28d is filled from M6 and is never 0 by default. GET /api/pieces?kind=blog is the Blog view the portal's Pieces screen shows.
• Septor: webmaster.pieces.synced (source: 'blog', with counts), only on a run that changed something.

Looking up one Piece: GET /api/pieces/lookup (iec-webmaster#111)

How the portal shows, on a published marketing piece, what Webmaster knows about its page. Marketing owns writing and publishing; Webmaster owns the live page and how it performs.

• One of two keys. ?post=<brand>/<slug> matches blog.postId; ?url=<absolute url> is canonicalised like every Piece URL (lowercase host, fragment dropped, path as given, so a trailing slash is a different page) and matched against the stored URL. A live Piece wins over a removed one with the same key.
• Answer: 200 { success: true, data: { piece, search } }. piece is the same view as GET /api/pieces/:id. search is { clicks, impressions, ctr, position, from, to }, or null.
• search is the page's totals over the last 28 days of stored data, summed from the stored Search Console page rows for the Piece's URL. The window ends at the Property's last pulled Google date and is clamped to its first, so from / to are the dates actually used and the window can be shorter than 28 days. ctr is clicks / impressions; position is weighted by impressions.
• null, never 0. search is null when no page row is stored for that URL in the window or the Property has never been pulled. A page can read null although it exists: the page breakdown holds only what the daily pull kept (the pages Google returned, by clicks, up to the pull's row cap), Google omits page-days with no impressions, and the URL is matched exactly as Google reports it.

Entities: the public read, the approval step and JSON-LD (iec-webmaster#15, #100)

An Entity is the organisation, brand, product or person a Property speaks for (spec §9). Sites render it as schema.org: the @insureco/seo SDK's <SeoHead>, and HypeSites. Its id is readable: ent_<slug> (ent_raterspot).

Nothing is emitted until a named person approves the Entity. An Entity has two copies:

Public routes (no auth, Cache-Control: public, max-age=300)

// PublicEntity
{ "id": "ent_raterspot", "slug": "raterspot", "kind": "organization", "name": "RaterSpot",
  "alternateNames": [], "description": "Rate every program in one place.",
  "url": "https://raterspot.com", "logoUrl": null, "sameAs": [], "parentId": null,
  "address": null, "phone": null, "email": null, "categories": [], "approved": false }

GET /jsonld finds the Entity from entity if given, else the host's Property entityId, else the Entity whose url is on that host. The graph is @graph: [Organization (or Product / Person), WebSite, WebPage, BreadcrumbList] (BreadcrumbList only below /). The Entity node's @id is <entity url>#organization, the same id the SDK uses. address, phone, email and categories (as knowsAbout) are in this graph; the SDK 0.1.0 emits name, url, sameAs, description and logo only.

Parents. With a parentId, a Product gets brand and manufacturer, and an Organization or brand gets parentOrganization, all { "@id": "<parent url>#organization" }; the parent's node joins the graph and is a product site's WebSite publisher. Only the parent's approved copy is used: a draft parent means no link and no node, and the preview answers parent: { id, approved: false } so the editor can say so. @insureco/seo 0.1.1 does the same in <SeoHead> (it fetches the parent by parentId; buildParentLink in /core).

HypeSites inject this graph into every 200 page on a custom domain, before </head> (hypesites SEO_API_URL, default https://iec-webmaster.tawa.pro). A page whose author wrote its own JSON-LD keeps it; the platform host never gets the block.

The Entity screen API (gated)

• version is required on edit, approve and unapprove. A stale one is 409 VERSION_CONFLICT with error.current, so a newer edit is never approved unseen.
• A named person is a user token, or a service sending x-acting-user (the portal tile). A bare service token gets 403 PERSON_REQUIRED: agents may edit and plan, people approve.
• Editable fields: kind, name, alternateNames, description, url (https), logoUrl, sameAs (https), parentId, address (streetAddress, addressLocality, addressRegion, postalCode, addressCountry ISO-2), phone, email, categories.
• A write through this API forgets the public cache at once.
• Septor: webmaster.entity.edited (with changed), webmaster.entity.approved, webmaster.entity.unapproved, plus webmaster.entity.seeded from the seed.
• Portal: Webmaster → Entities (/webmaster/entities) lists, edits, previews the JSON-LD and approves.

SEO_ENTITY_ID and the seed

• SEO_ENTITY_ID is either form. POST /properties/upsert-from-service answers entityId: ent_<slug> when the spec.seo.entity slug (or the service name) names an Entity, and the slug unchanged when none does. GET /entities/:idOrSlug accepts both, so pass the env value straight through.
• The seed (src/entities/seed.json in iec-webmaster) holds one Organization per family flagship in the Domain Map: name, url, and the Message column as description. It is applied at boot, idempotent on slug, and never touches an Entity a person edited or approved. Each write emits webmaster.entity.seeded. Correct a brand on the Entity screen, not in the seed.

Move to Cloudflare + DNS editor (Registrars Phase 2, iec-webmaster#52)

For domains whose DNS is not on Cloudflare. One domain in flight at a time; every write is a named person's typed confirmation, pressed as themselves in the portal's property DNS tab; nothing writes unless DNS_MOVE_ENABLED=true (default false), and a nameserver change through a registrar's API also needs REGISTRAR_DNS_WRITES=true.

A move that never happened is flagged, not acted on (#114). After an API switch the service reads the registrar's own nameservers back (GoDaddy GET /v3/domains/domain-names/{d}, 101domains GET /v1/dns/{d}/nameservers, falling back to GET /v1/domains/{d}). A synchronous answer that still lists the old pair sets lastError { code: REGISTRAR_NOT_CHANGED } and falls back to manual; a queued change (202 / operation id) is only a warning. The dns-move-check job flags a move that has waited in awaiting-nameservers for more than 24 hours with public DNS and the registrar still on previousNameservers: stalledAt (first time) and lastError { code: MOVE_STALLED }. Nothing rolls back by itself; a person does that in the portal. Both flags clear when Cloudflare's nameservers are seen. The job appends a history event only when the state, zone status, live nameservers or lastError change, so the move's real events are not pushed out of the 50-event history by 10-minute noise.

A stalled move stops blocking the estate, and can be retried (#117). Only one move runs at a time (dnsMove.inFlight, a unique partial index). When the check flags a move stalled it also removes inFlight: plan on any other domain stops reporting it as otherInFlight, and that domain can start its own move. A stalled move still finishes by itself if its nameservers appear (it takes the lock back if free, and finishes without it when another move holds it). retry-switch takes the lock back first and is refused with MOVE_IN_FLIGHT naming the other domain if one started meanwhile. Marking a domain "Not ours" is refused by the move's state, not by the lock. A person presses Retry switch in the portal (the same own-session gate as switch and rollback); agents never apply.

101domains nameserver changes are read by change_status (#117). PUT /v1/dns/{d}/nameservers answers 200 (already configured) or 202 (pending), both with data.change_status: completed is done (200 means it already had them), pending is accepted (queued), failed is an error; the HTTP status alone is not trusted (rainbows.lv's switch on 2026-10-02 returned success and changed nothing). The documented 400 codes (POLICY_VIOLATION, DOMAIN_STATE_PROHIBITS, UNKNOWN_HOST_OBJECT, INVALID_INPUT, OPERATION_FAILED, BAD_REQUEST) are reported in plain words followed by the registrar's own message. The fixtures test/fixtures/101domains/nameservers*.json are transcribed from the OpenAPI document (api.101domain.com/api/docs?api-docs-v1.json), not recorded live. | GET …/dns/records, POST …/dns/plan, POST …/dns/apply | read / read / webmaster:dns + portal session, or the allowlisted JCI client for dns/apply only | Record editor: plan a change set → hash; apply only that hash |

GET /api/properties?dns=not-cloudflare lists the candidates. The Cloudflare calls are limited in code to zone create, activation check and dns_records.

Who may apply a DNS change (iec-webmaster#106)

Applying is zone create, nameserver switch, rollback and record apply. After webmaster:dns, each passes requireDnsApplier(), which accepts only a Bio-ID user token that is

1. minted for the portal's OAuth client: client_id and every aud entry in DNS_APPLY_CLIENT_IDS (the portal's prod client, portal-prod), with no act, azp or persona claim; and
2. for a person whose Bio-ID (sub, equal to bioId when both are present) is in DNS_APPLY_BIO_IDS.

Either variable unset or empty: nobody can apply. Everything else is 403 DNS_APPLY_REQUIRES_PORTAL_SESSION (with error.details.reason, never a token): the tawa CLI's token (client_id: tawa-cli, which every agent on a developer's machine holds), a token exchanged from it (it carries azp/act), a service token with x-acting-user, another service forwarding a session, a persona, an unlisted person, an expired or forged token. x-acting-user is never read for an apply.

The token is the bearer (a user caller) or, from the portal, x-forward-user: <jwt> (raw; it must carry an orgSlug, as Janus requires) beside its service bearer (Janus requires a service bearer on /i/ calls and forwards the header). The portal's Webmaster tile sends the signed-in person's own portal_access_token on the four applies only, from the server, never to the browser. Plans, checks and reads are unchanged, so agents and JCI may plan, diff and check. Each apply is audited under the verified person: actor.id = the token's sub, actor.clientId = its client_id.

Setup: tawa config set DNS_APPLY_CLIENT_IDS=portal-prod DNS_APPLY_BIO_IDS=<bioId> on iec-webmaster, then deploy iec-webmaster before the portal.

Record edits from JCI (iec-webmaster#109)

POST /api/properties/:domain/dns/apply, and only that route, also accepts a service token when all hold:

1. token_type: client_credentials and client_id in DNS_RECORD_APPLY_SERVICE_CLIENT_IDS (jci-brain's client; unset = this path is off);
2. the exact scope webmaster:dns (* does not satisfy it);
3. x-acting-user in DNS_APPLY_BIO_IDS, the same allowlist as above.

planHash, the active-zone check and DNS_MOVE_ENABLED apply as before. jci-brain mints a separate token carrying webmaster:dns alone for this one call and only for a platform admin; its shared token still carries read/sync. The same token on dns-move/zone, /switch, /retry-switch and /rollback is 403 DNS_APPLY_REQUIRES_PORTAL_SESSION: nameservers, zone create and rollback stay in the portal session. Each record change is audited with actor.clientId, actor.id, planHash, before/after and actor.via (jci or portal). Agents may edit records for the allowlisted person; they never press the nameserver switch.

Unattended switch for new HypeSite domains (iec-webmaster#120)

The one narrow exception to "people press the nameserver switch". When a HypeSite is attached to a domain that is new to our Cloudflare account, the portal's service login calls:

POST /api/properties/:domain/dns-move/auto-switch
{ "nameServers": ["ada.ns.cloudflare.com", "bob.ns.cloudflare.com"], "reason": "hypesite-attach", "site": "my-slug" }
→ { "success": true, "data": { "outcome": "switched" | "already" | "needs-approval",
                               "reasons": ["<code>: <words>"], "approveUrl"?: "…", "operationId"?: "…" } }

Gate: a service token with webmaster:dns whose client_id is in DNS_AUTO_SWITCH_CLIENT_IDS (unset: DNS_APPLY_CLIENT_IDS, the portal's prod client). A person's token, the tawa CLI, jci-brain and any other service get 403 DNS_AUTO_SWITCH_CLIENT_NOT_ALLOWED. It is off unless DNS_AUTO_SWITCH_ENABLED (default false) and DNS_MOVE_ENABLED and REGISTRAR_DNS_WRITES are true; with any off the answer is needs-approval and nothing is written.

The switch is made only if every condition holds, each checked server-side (the caller's nameservers are only ever compared with Cloudflare's):

1. the domain is held at GoDaddy or 101domains, looked up live (an unknown domain becomes a Property), and that API can set nameservers;
2. nameServers equal exactly the pair our Cloudflare account assigned to the domain's zone (no zone: needs-approval; this route never creates one from nothing);
3. nothing is lost: the move plan's records-at-the-current-host-missing-from-the-zone set, ignoring registrar parking defaults, is empty (any MX, or any non-parking A / AAAA / CNAME / TXT, counts as something to lose);
4. the domain is new (registered or first seen within 30 days) or unused (parked, dark, or a lame delegation);
5. no other move is in flight, no person rolled this domain back, and no unattended switch was tried in the last 24 hours.

It goes through the ordinary DnsMoveFlow, so history, previousNameservers, the one-at-a-time lock and rollback are the manual path's. The audit event is the manual webmaster.nameservers.switched with actor.auto: true and the checks in autoChecks. Anything else is needs-approval with every failing reason and an approveUrl (the portal's page for the domain), for a person to press as before. already means the domain is delegated to the zone's nameservers, or a switch to them is under way or done.

Credentials: vault aliases by name

Every outside key comes from the Locker by handle, bound once with tawa config set --secret <ENV> --from-locker lk://insureco/<vault>/<item>; the service reads the env var at boot and refuses to start naming any alias that is missing. A value never passes through a terminal, a repo, a log line, an error message, a Septor payload or a test fixture.

Alias names are hyphenated: the builder refuses a slash (iec-webmaster #28, #29).

Rotation is a vault action, re-run the bind, redeploy. The Cloudflare token is never the builder's (tawa domain add --cloudflare uses the builder's own).

Ownership. The Google Cloud project (insureco-webmaster), its service account and the Search Console properties live under the company Google Workspace organisation — the one behind insureco.io mail (per #8 the org is thepolicyspot.com, with insureco.io as a secondary domain on it) — never a personal Google account. Derek created it as Workspace admin; [email protected] is the second project owner so it survives one person, and both are added as human owners on every verified property.

Only Janus moves money

This service never calls iec-wallet, never meters gas, never holds a wallet credential and has no billing code. Its own routes are gas: 0 in M1. The agency Listings tiers (spec §6: Base and Plus, per office per month) are priced routes in PolicyOS's catalog that Janus charges; iec-webmaster only reads tier and office count from PolicyOS. See Service-to-Service Communication → Only Janus moves money and Gas & Wallet.

What it writes to Septor

Every outward-facing write is a webmaster.* event with the caller's actor: webmaster.map.synced, webmaster.property.ignored, webmaster.dns.txt.created / .updated / .removed, webmaster.property.verified, webmaster.property.verification.failed, webmaster.owner.added, webmaster.verify.bulk.started / .finished, webmaster.source.set (a person) / .registered (a source claimed it) / .defaulted (auto: none → legacy) / .kept / .unkept (a person keeps a legacy Property, or it stops being kept), .placeholder / .unplaceholder (a person marks a served source as a placeholder, or it stops being one: a person, none, or a different source claimed it), webmaster.legacy.crawled, webmaster.pieces.synced, webmaster.search.pulled, webmaster.search.pull.started / .failed / .abandoned, webmaster.service.started, and for DNS moves webmaster.zone.created, webmaster.nameservers.switched, webmaster.nameservers.retried (#117: outcome is pending, completed, already-set, refused or not-applied; Septor event names cannot carry a hyphen), webmaster.nameservers.rolled_back, webmaster.dns.record.changed (before/after). Payloads carry domains, codes, messages and the public TXT value; never a credential. Convention: Septor.

How a Tawa service joins (M3): spec.seo

From catalog-info.yaml 0.8.0 a nextjs or static service opts into the SEO machine with one block, the same shape as spec.analytics (changelog: catalog-info.yaml 0.8.0, spec §12):

metadata:
  annotations:
    insureco.io/catalog-version: "0.8.0"   # required, or spec.seo is ignored with one warning
    insureco.io/framework: nextjs          # nextjs or static only
spec:
  seo:
    enabled: true
    entity: raterspot                  # Brand Entity slug; defaults to the service's product
    pages: sitemap                     # 'sitemap' (default) or an explicit list of paths
    topic: insurance rating software   # default Topic for pages that do not name one

On tawa deploy --prod the builder resolves the service's verified custom domains (a platform hostname is never submitted for indexing and carries noindex), upserts a Property per domain in iec-webmaster with source.kind = 'tawa-service' — verifying it through the DNS path above if it is not verified yet — injects SEO_PROPERTY_ID, SEO_ENTITY_ID and INDEXNOW_KEY as runtime env (never NEXT_PUBLIC_*), and once the deploy is healthy calls POST /hooks/deploy with its own Bio-ID service login. The service then reads https://<domain>/sitemap.xml, registers Pieces, submits the sitemap to Search Console and Bing and pings IndexNow. Sandbox and UAT register and stop. Nothing in this path can fail a deploy; the hourly service-sync job is the safety net.

The builder side shipped with 0.8.0 behind SEO_INTEGRATION_ENABLED. It defaults to off, and while it is off every call is logged as a dry run.

On the service side, these are built: the hooks, the builder contract routes, the Piece model, the sitemap reader and service-sync (iec-webmaster #10, #11).

Still to come in M3: IndexNow (#12) and the @insureco/seo helpers (#14: <SeoHead>, sitemap(), robots(), indexNowKeyRoute()).

Switching it on needs two things:

1. The builder's Bio-ID client holds webmaster:hooks.
2. SEO_INTEGRATION_ENABLED=true is set on the builder.

Until then, declaring spec.seo is safe, and a service can check its payloads in the build log.

Where it runs

One Tawa service, hono, auth.mode: service-only (its own Bio-ID login, no user redirect), databases: [mongodb], and one dependency, portal. policyos and insurebio arrive with M9.

The builder is not a dependency (iec-webmaster#30). It runs under PM2 on the builder host, is not in Koko and cannot be reached through Janus. The service calls it at its public URL, BUILDER_URL (plain config, default https://builder.tawa.pro), with its own Bio-ID service login. On the builder side, GET /seo/services accepts that login (iec-builder#206). Jobs run in-process behind a Mongo lease (map-sync hourly at :07, noop-heartbeat every 15 min) because iec-queue's enqueue is broken platform-wide (iec-queue#1); they move to spec.schedules when it is fixed. The Domain Map sync reads the portal's table data plane with the service's own bearer; the portal accepting that bearer is iec-portal #853.

Working on it

Read the repo's CLAUDE.md first. Truth is the spec in the portal; if code and spec disagree, the spec wins. npm run typecheck and npm test (unit, recorded upstream fixtures under test/fixtures/, and an HTTP-level e2e scenario under test/e2e/) must be green; none of it needs a credential. Agents open PRs and never merge their own, never deploy, never edit the portal pages or this Bible page directly (this page changes through a PR to tawa-docs, conventions/webmaster.md), one ticket per branch.

Common mistakes

• Applying zone create, nameserver switch or rollback from an agent, the CLI, or a service with x-acting-user. It is refused by design (403 DNS_APPLY_REQUIRES_PORTAL_SESSION); press confirm in the portal tile as yourself. (Record edits from the allowlisted JCI client are the one exception, #109.) Do not try to mint a token that would pass.
• Asking for an API key. There is none; declare the dependency with a scope.
• Calling from a service outside org insureco, or with a user token that is not a platform admin: 403 by design.
• Pasting a credential into tawa config set. Bind the Locker item with --from-locker lk://insureco/<vault>/<item>.
• Writing the verification TXT with the builder's Cloudflare token, or by hand: the service owns it (iec-webmaster: comment) and re-checks it.
• Expecting POST /api/search/pull (no body) to answer with the summary. It answers 202 with a job; poll GET /api/jobs/:id, and treat 409 PULL_IN_PROGRESS as "already running" (error.details.jobId), not a failure.
• Expecting POST …/verify to answer with the outcome. It answers 202 with a job; poll GET /api/jobs/:id or read the Property's verification.
• Expecting a legacy Property to be a finished state. It is a to-do (needs-hypesite); publish a HypeSite or deploy a Tawa service for the domain and it resolves itself.
• Expecting a service redeploy to clear a placeholder, or a placeholder to stop the service serving. It does neither: only a different source claiming the domain, or a person, clears it, and source.kind never changes.
• Setting none to "park" a domain by editing the row. Use POST …/source { kind: "none" }: only that records setBy: person, which keeps it from being defaulted back to legacy.
• Adding billing to this service. Janus charges; PolicyOS prices; iec-webmaster reads.