Human Oversight of AI: A Complete Report Isn't Evidence

Human oversight of AI breaks down without a single error message when the person signing trusts completeness. An AI-generated report can carry every finding and every met criterion while describing a different revision than the one you’re shipping. Before you sign, check that each item still points at the work it claims to describe.

At a Glance A complete AI-generated report proves the report is complete. It doesn’t prove the report is about the thing you’re releasing, because every item reaches its subject through a name the report itself can’t re-check. In the When Structure Pays case study (September 2026), a purpose-built five-check completeness gate passed two checks over input it never read, and the repair to one of them created a third. For a GRC lead evidencing human oversight, the question before signing is whether the report’s links still hold.

Key Takeaways

• Completeness belongs to the report. Whether the report describes your release depends on links it can’t check from the inside: finding IDs, cache keys, revision names.
• In September 2026 I built a five-check completeness gate for the documents my pipeline produced. Two of its checks reported a pass over input they never read, my repair to one created a third, and a second model found all three.
• The EU AI Act, ISO/IEC 42001 and the NIST AI RMF ask for an assigned overseer, a documented process and logs. None requires a record of what the overseer checked.
• Before you sign, ask where each claim’s identifier points today. Then log every catch, because each one names a check your pipeline is missing, and watch whether catches rise with volume while reported defects stay flat.

What does a complete report prove?

A complete report shows that every section and field is filled in. Whether each field still describes your release is a question about links, and links decay at a rate researchers have measured. A 2014 PLOS ONE study found that one in five science, technology and medicine articles suffers from reference rot, links that break or quietly drift.

Picture a GRC lead like Priya Shah at 10pm on a Tuesday. She runs governance, risk and compliance at a 1,200-person SaaS company, and her engineers have started attaching AI-generated review reports to every release. The one in front of her is clean. Every acceptance criterion is met, every finding carries a file and a line number that resolve, and the revision sits in the header.

Each of those items reaches its subject the same way. The report stores a name (a finding ID, a cache key, a revision string) and the reader assumes the name still points where it pointed when it was written. The first hop, from report to name, is checkable. The second hop, from name to the actual work, is where the evidence lives. Checking it means going back to the work itself, a step most pipelines never schedule.

Unchecked second hops are already turning up in public issue trackers. On August 27, 2026, the open-source Agenta project logged an issue in which its agent builder reported building a new agent after it had committed revision v7 onto an existing one (Agenta issue #6340). The issue’s author called the misrepresentation itself the bug.

When that hop breaks, nothing throws. A dead web link at least returns a 404. A broken evidential link behaves like a drifted one: it resolves fine and hands back the wrong thing.

My own completeness gate passed work it never read

In September 2026 I wrote a 118-line completeness gate with five checks, reported in When Structure Pays (§4.1). Two of those checks reported a pass over input they’d never read, and my repair to one introduced a third false pass in the same check. A second model, working under a fixed review contract, found all three.

I’d written and tested all five checks and found none of the three. I’d just converted a working review pipeline into a governed task graph and spent the day chasing the silent failures that conversion produced. The obvious next move was to stop fixing instances and build a gate that checks every document the pipeline produces. Its failures were a different kind from the broken links described above, and they landed in the same place: a passing report over work it hadn’t done. The gate checked five things: no em dashes, no forbidden phrases, every required section present, no duplicate sources, citations numbered in the order they appear.

The duplicate-source check only parsed entries that began with a number and a period. That’s one citation style out of four. For the other three it walked zero entries, found zero duplicates, and passed. The section check compared names by prefix in both directions, so a heading called “Funding mechanisms in prior research” satisfied a required Funding declaration.

Then I fixed the first one. My fix split the reference list on blank lines, which merged tightly packed numbered entries into one block, and everything after the first entry went unread. It passed my testing again.

I’d studied exactly this failure class before I wrote any of it. Closing two false passes took the gate from 118 lines to 162.

That’s the lesson I’d hand any auditor. A missing check leaves the suspicion in place, and a false pass retires it. Whoever runs that gate and sees five green lines has been given a reason to stop looking, and in two of the five the reason was false.

Why doesn’t a second reviewer close the gap?

A second AI reviewer widens coverage, but it can’t tell you whether its findings are about the revision you’re shipping. Its independence is limited too. The second model that caught the false passes in my own document-completeness gate worked under a review contract I wrote, so what it looked for was still mine.

And NeurIPS 2024 research found LLM evaluators favor their own output, which matters whenever reviewer and author are the same model.

The one time I watched a wrong revision reach sign-off in my own work, no reviewer caught it. I’m telling this from memory, since no commit records it. I was holding a release with a passing report in front of me when I noticed that one review inside it had examined an earlier revision than the release candidate. We redid the review and shipped. By any ordinary account the control worked.

But the control was me, remembering which revision had moved. No record shows that I did it. Commit 0783b35, covered below, now does part of that check so a person doesn’t have to.

What do the new oversight rules collect?

Read together, the EU AI Act, ISO/IEC 42001 and the NIST AI RMF give an auditor three things to collect for oversight: who was appointed, what procedure they followed, and when the decision was logged. The Act’s Article 14 requires high-risk systems to be designed so people can effectively oversee them.

Article 12 requires automatic event logs over a system’s lifetime. In July 2026 the Digital Omnibus on AI moved those obligations for Annex III systems to December 2, 2027.

Article 26 adds that deployers must hand oversight to people with the competence, training and authority to do it. ISO/IEC 42001 asks for event logs and for processes that govern responsible use of AI systems. And the NIST AI Risk Management Framework, published in January 2023, asks organizations to define roles for human-AI configurations and to document their oversight processes (NIST AI 100-1).

Hugh Deura, who wrote the paper’s reading of these regimes, explains why a record of what the overseer examined isn’t on that list: a checkbox for it would only capture the overseer’s own statement.

So Priya can be fully conformant on this control and still sign a report about the wrong revision. The questions in the next section belong in her sign-off step, where that gap sits.

What should you ask before you sign?

Start with the identifiers. For each claim in an AI-generated report, find the identifier it rests on and confirm it still points at the thing you’re releasing. In the When Structure Pays case study, that produced four questions, one for each capability the conversion to a task graph added: parallel reviews, caching, resumable runs and declared outputs.

• Which finding does this disposition rule on, and has that finding been renumbered since?
• Which inputs does this summary cover?
• Which revision did this review examine, and is it the commit you’re about to ship, or one that was current when an interrupted run started?
• Does each step write only where it said it would?

Those four came out of our change, so treat them as an example. Yours come from asking what your own pipeline added (a cache, retries, parallel reviewers) and which names each addition keeps.

Then start a catch log. Whenever you stop a release because something didn’t line up, write down the date, what you caught and which link it concerned. Those entries are plain facts. Over a quarter, they add up to a rate you can watch.

Our revision catch became commit 0783b35, which now refuses to resume a review whose first round never completed when the repository head has moved. It’s a partial fix, and every catch can become a check like it. If your catch rate climbs with volume while your reported defect rate stays flat, that’s the signature of a person doing a job the pipeline claims to do. We keep ours in the shared decision log we started when we put our own governance manifesto through its gate.

Where this argument stops

The When Structure Pays case study covers one conversion of one system, by the person who built it. The 82-minute conversion turned up three link defects, but the search stopped when the symptom did, so nobody knows how many existed.

The false passes in my completeness gate were found by a model working under a contract I wrote. That limits how independent the finding can be.

What would raise confidence is the same links examined by someone with no stake in the answer. Kapardhi Kannekanti, who read the manuscript closely, has offered to do exactly that against the named commits. The offer has been taken up. His findings aren’t in the paper, and they won’t pass through me first.

The general claim doesn’t lean on my case, though. Any report that stores a name for its subject, rather than re-checking the subject, can be complete on its own terms and wrong about the work, the way Agenta’s v7 report was.

This week, pull the last AI-generated report you signed, check where each identifier points today, and start the catch log. You’ll know it’s working when your catches start naming checks you can build.

The rest of our work on governed autonomy lives in the AI Governance series. The full paper, When Structure Pays, is in the Frontier Founder library and free with an account. It’s also citable at doi.org/10.13140/RG.2.2.31151.29608.

Frequently asked questions

Does the EU AI Act require proof that a human reviewed AI output?

Not proof of what they reviewed. Article 14 requires high-risk systems to be built for effective human oversight, Article 26 requires deployers to assign competent people to that oversight, and Article 12 requires event logs. After the 2026 Digital Omnibus, those obligations apply to Annex III systems from December 2, 2027. None of them requires a record of what the overseer checked.

Is a completeness check worthless for AI governance?

No. It’s fast, and it catches missing pieces. Run against an earlier paper, my gate flagged sixteen missing required sections in about a second (When Structure Pays, 2026), matching what a manual critique had already found. What it can’t tell you is whether a report describes your release. A check that passes input it never read is worse than having no check.

How do I know which revision an AI code review examined?

Find the revision the review itself recorded when it ran, then compare it to the commit you’re releasing. The report header isn’t enough on its own. If your pipeline resumes interrupted reviews, confirm the review didn’t start against an earlier head. If the two don’t match, rerun the review before you sign.

What is a catch log?

It’s a running record of every time a person stopped a release because the evidence didn’t line up: the date, what was caught, and which link it concerned. It turns a hidden dependency on one person’s attention into a rate you can watch.

Where can I read the full When Structure Pays paper?

It’s in the Frontier Founder essay library, free with an account, and it carries the DOI 10.13140/RG.2.2.31151.29608. Written by Dorian Cougias with Hugh Deura of DISC InfoSec, it reports the whole case, including the commits a reader can check.

Part of the AI Governance series. Read the full AI Governance series →

Published September 16, 2026. Last updated September 16, 2026. Sources retrieved September 16, 2026.