If your reviewers approve 99.8 per cent of AI outputs, that number is a risk signal, not proof of control. Singapore's agentic AI framework now suggests that override rates be tracked. The clinical evidence shows why.
A reviewer in your team signs off AI-drafted work. The dashboard says she approved 99.8 per cent of it last quarter. Read one way, that is a tool performing well. Read the other way, it is a person who has stopped checking. The dashboard cannot tell you which. Singapore's Model AI Governance Framework for Agentic AI, version 1.5, published in May 2026, now suggests that organisations track human override rates and response times, and regularly audit whether human oversight is still effective. It says a low override rate may signal rubber-stamping. The concern behind that guidance shows up in the clinical literature, where the question has been studied with real clinicians.
The cleanest demonstration used radiologists. In a 2023 study in Radiology, 27 radiologists read mammograms with a suggestion from a purported AI system attached. Some suggestions were deliberately wrong. When the suggestion was wrong, the proportion of correct readings fell sharply. Inexperienced readers went from 79.7 per cent correct to 19.8 per cent. Very experienced readers went from 82.3 per cent to 45.5 per cent. The suggestions were set by the researchers rather than produced by a real system, only five of the 27 readers were very experienced, the sample was small and there was no without-AI arm in the same design. Within those limits the finding is stark: on average, the most experienced readers in the study rated more than half of the mammograms wrongly when the suggestion was wrong.
The second finding is about what happens to the skill over time. A 2025 observational study in The Lancet Gastroenterology & Hepatology compared colonoscopies performed without AI at four Polish endoscopy centres in the three months before and the three months after AI-assisted polyp detection was introduced. On those colonoscopies, the adenoma detection rate fell from 28.4 per cent to 22.4 per cent, a six-point absolute drop. The study is retrospective and observational, so it cannot prove the AI caused the decline. What it suggests is that a professional's unassisted performance may fall after a period of assisted work, which is one way a reviewer could lose the ability to tell when the system is wrong.
The two findings together describe the approval-rate problem. The first says a person shown a wrong answer will often accept it. The second says the ability to notice may erode with use. A reviewer whose approval rate is climbing towards 100 per cent may be reviewing a better tool, or may be a person in whom both effects are now operating. The rate alone cannot distinguish them.
For trainers and facilitators: both studies make a session opener that lands harder than any slide about automation bias, because the people in them were radiologists and endoscopists. The point to draw out is not that experts are careless. It is that expertise did not fully protect them.
AIRS-J/1 scores verification as dimension T1, Evidence Verification. Level 0 is acting on an unverified claim where verification was available and the claim was load-bearing, or treating system output as established fact. The reviewer at 99.8 per cent who has stopped reading is at Level 0 on every approval that mattered. Level 2 checks the load-bearing claim against an available source before acting, unprompted, and distinguishes generated content from retrieved content.
The anchor's note closes the escape route that careful people reach for. Rejecting AI output wholesale without examining it is a failure to verify, not caution. A reviewer who overrides everything has an approval rate of zero and has checked nothing. The Standard scores that at Level 0 too, which is why the override rate on its own is as uninformative as the approval rate on its own. Both extremes are the absence of the behaviour.
Level 3 is where the measurement becomes meaningful. It requires verification effort proportionate to consequence: heavy where the decision is irreversible or high-consequence, light where it is neither. And it requires the work product to state which claims were verified and which were not. That second clause is what makes verification auditable. A piece of work that says "the delivery figures were confirmed with the vendor's references; the market estimate was not independently checked" has recorded its own verification. A piece of work that says nothing has left the reviewer's behaviour invisible.
This is close to what Article 14(4) of the EU AI Act describes for the people who oversee high-risk systems, in its own vocabulary. The system must be provided so that the people assigned to oversee are enabled to remain aware of the possible tendency to over-rely on the output, and to decide, in any particular situation, to disregard, override or reverse it. A reviewer who cannot be shown to have done either has not shown that the oversight the Article describes is taking place.
An approval rate of 99.8 per cent and an approval rate of zero describe the same reviewer. Neither one checked.
The crowded answer to overreliance is explainability. Make the model show its reasoning and the reviewer will catch its errors. The evidence for that is mixed. A 2023 study found explanations reduced overreliance on hard tasks but not easy ones, and that written explanations produced more overreliance than simple highlights. People verify when verifying feels worth the effort, and an explanation helps only when it makes checking cheaper. An earlier study found that designs forcing a person to think before accepting the AI's suggestion, such as making their own decision first, reduced overreliance more than simple explanations did, and that people rated the designs that worked best the least favourably.
The alternative is to measure the reviewer, which the Singapore framework points towards. Three measures together say more than any approval rate, and the first two are indicators the framework names. The override rate: how often did the person change or reject the output? The time to decision: did the review take long enough for a check to have happened? And whether the override was right: when the person changed the output, was the change an improvement? The third measure is not one the framework names. It is the one that needs a retained trace, because it cannot be answered from a dashboard. It needs the work product, the source the reviewer consulted, and the reasoning they stated, kept together.
The trace is what the Standard requires for any observation to count. Transcripts under AIRS-J/1 retain the full decision trace: decisions, sources consulted, stated rationale, revisions and timings. Every observation must be traceable to identified evidence in that trace. Keystroke capture, affect inference and biometric signals are never scoring inputs. The Standard reads what a person did, which can be shown to them and explained. That is also the only form of evidence that could tell you whether your 99.8 per cent reviewer is reading.
For HR and L&D: the three measures are not a performance metric for the person. They are a diagnostic for the workflow. An override rate near zero across a whole team says something about the task design, the time allowed or the tool, before it says anything about any individual.
You do not need a programme to start. One workflow, one quarter, three numbers. Pick the AI-assisted workflow in your area with the highest consequence if wrong. For a quarter, record the override rate, the median time from receipt to sign-off, and, for a sample of overrides, whether a second person judged the override correct. At the end of the quarter you will know something you do not know now: whether the review step in that workflow is a check or a signature.
If the override rate is near zero and the median time is under a minute, the step is a signature. If the override rate is near zero and the time is reasonable and the sampled overrides were right, you may have a good tool and a good reviewer. The numbers will not tell you which person is at which level. They will tell you whether the question is worth asking, and a team view of ten or more people against the Standard is how it is answered.
A high approval rate is a question, not an answer. The Standard scores the behaviour behind it, and the only way to see the behaviour is to keep the trace.
Measure one workflow, then ask for a team view
AIRS-J/1 assesses how a person decides, not what they know. A result is not a measure of intelligence, personality, AI technical skill or domain expertise, and it is not a substitute for supervision, professional licensing or regulatory authorisation. The clinical studies cited measure diagnostic performance in specific populations and were not scored against the Standard; the endoscopy study is observational and cannot establish cause. The three workflow measures are a diagnostic suggestion, not an assessment, and produce no level for anyone. The Singapore framework names override rate and response time as indicators an organisation can track and gives no threshold for either; the third measure here, whether an override was right, is not in the framework.
AIR APAC publishes these articles in the institution's voice. Every figure carries its source and its limit. All articles.
Singapore's agentic AI framework asks for human approval checkpoints. Korea's AI Basic Act asks for documented oversight. Both assume you hold a list of decisions a machine, or a junior, may not take. Most departments have the policy and not the list.
"Keep a human in the loop" is the most repeated line in AI governance. The loop breaks when the person rubber-stamps, has lost the skill, has no authority to stop, or has too much to check. Only one of those is fixed by training.
When a decision involves AI, many managers choose the most cautious option and call it prudence. The Standard scores that at Level 1. Maximal caution where the cost is low is as much a failure as minimal care where the harm is severe.