Our first finding was our own defect
We pointed this product at a month of real agent output before we pointed it at anyone else’s. The first number it gave us was a headline. The headline was wrong, and the error was ours, not the agents’.
The gap is not agents improving. It is our probe aimed at the wrong repository
— the exact failure this product is named after. An agent’s cwd is
where it was standing, not where it committed.
- 73 — real commits in a sibling repository on the same disk.
cae8c30is real inrekt-capital.7b3256dis real inhelicon.da23d89is real inmountain-of-helicon. The agent was right and the probe looked in the wrong place. - 11 — machinery, not claims: a SHA inside a shell command the agent was
running, or inside fenced git output it was reading. Across the
hand-labelled sample of 40 extractions, six were
deadbee, this repository’s own test fixture, surfacing in transcripts about building this gate. The tool for catching false claims found its own test data and would have counted it as an agent claim. - 19 — what is left after both corrections. Every exclusion is counted with its reason rather than silently deleted; a filter that quietly shrinks a finding list is the flattering version, and the refusal is the product.
The scan, as it printed
$ python3 gate/corpus_scan.py --db ~/.trace/trace.db --code-root ~/CODE
78,618 messages examined, of 144,306 in the corpus · 83 repos on disk
filter: role='assistant' and is_human=0 and text is not null and length(text) > 20
52,878 of those were written in a directory that is still a git repo today
RAW 247 sha claims · 103 disagree · 41.7%
CORRECTED 236 sha claims · 19 disagree · 8.1%
11 dropped — shell commands, fenced output, and this repo's own test fixtures
73 resolved in a SIBLING repo — the agent was right, the probe was aimed at the wrong repo
5 path claims dropped — a code identifier, not a file
1 path claims dropped — a hostname, not a repository path
1 path claims dropped — an absolute path outside the repository
Corpus: ~/.trace/trace.db — 2,672
indexed transcripts of real agent sessions across 74 repositories. Not a fixture, not written
for this. Ships as gate/corpus_scan.py; run it on yours with
witness-corpus --db <your.db> --code-root <dir>.
What this number does not tell you
- 8.1% is not an incidence rate, and neither was 41.7%. Neither is your fleet’s error rate and neither is presented as one.
- n = 13. Hand-labelling a seeded random sample of 40 extractions put the
extractor’s precision on conversational prose at 13/40 — the rest
were citations (“Commit 7b3256d landed on…”), shell commands and
fixtures. Of those 13 real claims, 6 disagreed with the repository.
A direction, not a measurement. The sample and its labels ship in
fixtures/corpus-sample-40.jsonso you can re-label them and disagree. - One operator, one machine. Nothing here generalises to a population. The hand labels were assigned by the same agent that wrote the report — the weakest link on this page, and the cheapest one to fix.
- The denominator was fixed before anyone looked at a claim.
docs/CORPUS-PREREGISTRATION-2026-08-27.mdnames what counts as a claim, which claim types may carry a headline, and five confounds — including the one that turned out to matter. That document is the method, and the method is the product. A vendor who shows you a number without one is showing you a negotiation. - The raw number is kept, not deleted. 41.7% is printed beside the
corrected figure by the scan itself and in
docs/ENTERPRISE-CASE-2026-08-27.md; the earlier, cruder pass that produced 42.0% before these confounds were tested survives separately indocs/CORPUS-MEASUREMENT-2026-08-27.md. Two runs, both kept, neither mixed into the other: deleting either would hide the one thing worth learning. - The corpus is alive, so this page quotes a frozen run. The same scan
re-run on 2026-08-29, against the same database now grown to 154,970 messages, reports
84,560 examined · 50.6% raw → 11.5% corrected, with 82 of 161 again resolved in a sibling repository. The totals move because the corpus moves; the shape does not. A number on a page needs the date it was measured, so the number above carries 2026-08-27 and is not silently refreshed.
The record this finding came from
One real hold, written by the GitHub Action on a real
pull request. Not seeded — demo_seed_enabled is false in
production.
- Record
- H-a6151a95ac
- Decision
- HOLD · gate BLOCK · policy
enforce - Repository
- Morkeeth/agent-work-record-witness-ata
- Pull request
- #1 · verify-claims (red by design)
- Written by
- source=github-action · actor=Morkeeth (the Action passes the PR author)
- Head commit
- c99589111f82
- Stored
- 2026-08-29T12:15:09+00:00 · Firestore
- Session join
- 01Lzbh4XPYTAgCKg1dciFS3Q
| Verdict | Claim | Probe | Evidence |
|---|---|---|---|
| BLOCK | committed as deadbee | git cat-file -t deadbee |
NOT a commit in this repo (no sibling repos were checked) |
| BLOCK | wrote docs/auth-migration-2026.md | stat docs/auth-migration-2026.md |
NO SUCH PATH in the repo |
Note the evidence string on the first row. After the corpus finding, the probe says “no sibling repos were checked” in its own output, because that is precisely the assumption that made 73 correct agents look wrong.
Where this runs on Google
Every row below is a service on the request path, with the probe that shows it. Nothing here is an architecture diagram of an intention.
| Layer | What runs there | How you check it |
|---|---|---|
| Cloud Run | The witness gateway. Serves this console at /hold/ and the
/clearance API. Service fleet-wedge, region
us-central1. |
GET /health → payload below |
| Firestore | The record. Every clearance, hold and exception is a document and the API
never deletes one. Keyed store, not an append-only log — closing a
hold rewrites that clearance to open: false in place, so the prior
version is gone. Live: H-1d8344f3dc. Said here rather than found. |
"store": "firestore" on /health;
GET /audit/export |
| GitHub Actions | The verify-claims check. The probe runs inside your checkout
where the repository already is; only the verdict crosses the network. The
product never needs read access to your code. |
PR #1 on this repo — red on purpose |
| Application token gate (not IAM) | Cloud Run is public at the IAM layer by design so you can click this
console — the only binding on fleet-wedge is
allUsers → roles/run.invoker. The 401 is application-level:
_require_token() in cloud/service.py refuses every mutating
route without a bearer token. Honest limit: one shared bearer token, not
per-agent identity, and not IAM. |
curl -X POST .../clearance -d '{}' → 401;
gcloud run services get-iam-policy fleet-wedge → allUsers |
| Gemini via ADK | google.adk.runners.Runner drives an LlmAgent
(gemini-3.5-flash-lite) to explain a hold in the reviewer’s
words. It never decides one.
Gemini explains. Python decides. |
agent_explanation.invoked on record H-a6151a95ac |
Live /health, probed 2026-08-29
$ curl -s https://fleet-wedge-33kamss2jq-uc.a.run.app/health
{
"ok": true,
"service": "agent-work-record-witness-gateway",
"product": "THE AGENT WORK RECORD WITNESS",
"console": "/hold/",
"auth_required": true,
"demo_seed_enabled": false,
"store": "firestore",
"agent": {
"class": "google.adk.agents.llm_agent.LlmAgent",
"constructed": true,
"invoked": false,
"last_run": "never invoked in this process — POST /agent/run"
},
"policy": { "mode": "enforce", "agent_only": true,
"label": "agent", "break_glass_role": "break-glass" }
}
invoked: false is reported deliberately and is per-process, not per-product: a
class name proves an import, not a run, so /health reports the last real
invocation instead of a type() string. The run that did happen is recorded on
the hold itself, where it can be checked.
What is not on this path
Google Agent Builder / the managed GEAP surfaces — Agent Registry, Memory Bank, Model Armor — are not wired here, and naming a managed product we have not wired is the sharpest available lie in a submission that is about false claims.
Known posture, said first rather than found.
HOLD_API_TOKEN is mounted from Secret Manager
(hold-api-token:latest) on the live revision fleet-wedge-00014-q2g
— probed 2026-08-31, not asserted. It is never in the repository and never a plaintext
revision env var. This paragraph said the opposite until today: it described an older
revision and nobody re-probed it, which is this product's own failure mode pointed at its own
console. What is still open: the service
runs as the default compute service account
568004190078-compute@developer.gserviceaccount.com, which holds
roles/editor — the principal behind the record can delete the
collection. Both are open items, not design positions.
The gap is measured rather than glossed: docs/GEAP-GAP-2026-08-27.md
walks all seven surfaces at the object and marks each one present, partial or absent, including
the three defects that audit found in our own live service. What is real today is the gateway,
the audit record, and token-gated writes.
Hold queue
Calm default: empty. Humans only see releases the object disproves. Reading the queue and audit needs no token.
Operator token (writes only)
Queue and audit are read-only without a token. Paste X-HOLD-Token only for break-glass or policy changes. Stored in this browser only.
Install
Platform eng adds the clearance check on agent PRs. Five minutes. No dashboard login required for the gate itself.
1 · Policy URL
2 · GitHub Action (agent-scoped)
3 · Make it binding (when ready)
Run in report-only first. When branch protection requires verify-claims, promote to enforce. Until then the check is advisory — do not call it a required check while protection is off.
Policy
Progressive enforcement — report-only first, then fail-closed on agent paths.
Audit
Every clearance decision, and what the object said. The API never deletes a row — but this is a keyed store, not an append-only log: closing a hold rewrites that clearance in place. Click a clearance row to open the full release detail.
| When | Kind | Decision | PR / claim | Actor |
|---|
Registry
Surviving practice → literal org skill. Module, not the hero. Field of 2 stays UNMEASURED.
Not run yet.