Attendance, tool certificates and self-rated confidence are the three records most organisations hold on AI capability. None of them shows whether a person decides well with AI in the room.
Picture the capability report that reaches you at the end of this quarter. Eighty-two per cent of your people completed the AI fundamentals course. Forty have a vendor certificate for the tool the organisation licensed. The post-course survey says confidence in using AI at work rose from 3.1 to 4.2 out of 5. Every number is true. None of them tells you whether the person who signed off last week's AI-drafted tender could tell that one of its figures was wrong.
That is not a complaint about the training. It is a statement about what the records measure. They measure exposure and feeling. The thing you need to know about is behaviour under consequence, and the evidence says the two can run in opposite directions.
The clearest result comes from a small randomised trial of real work. In 2025 the research group METR gave sixteen experienced open-source developers 246 tasks in their own repositories, averaging about two hours each. With AI tools allowed, the developers took 19 per cent longer. Before starting, they had forecast a 24 per cent speed-up. After finishing, with the slowdown measured, they still believed the tools had made them about 20 per cent faster.
The sample is small and the tools were early-2025 versions. METR says the result does not show that AI fails to help developers in general, and its page now marks these results as out of date. The study shows something narrower and more useful here. Sixteen skilled people, asked to rate their own performance with AI, got the direction wrong. A self-rated confidence survey administered to them would have reported a gain.
A related pattern appears in larger samples. A 2025 study by Microsoft Research and Carnegie Mellon surveyed 319 knowledge workers who used generative AI at least once a week, collecting 936 first-hand examples of work. The more confident a person was that the AI could do the task, the less critical thinking they reported applying to it. The data is self-reported, and it measures perceived effort rather than output quality, so it cannot show that ability fell. What it shows is that trust in the tool and reported scrutiny of the tool move in opposite directions.
Higher AI literacy went with worse self-assessment, not better. In an experiment published in Computers in Human Behavior in February 2026, 246 participants solved 20 logical reasoning problems with AI help, and a second study with 452 participants repeated the result. In the first study, scores rose by about three points against a norm population, and participants overestimated their scores by about four. The familiar pattern where weaker performers overrate themselves and stronger performers underrate themselves disappeared. Participants overrated their results whatever their skill level. Those with higher AI literacy were more confident and less accurate about their own results. The task was abstract and had no workplace stakes, which limits what can be read across to work. The direction of the finding is what matters: knowing more about AI did not make people better judges of their own judgment.
For trainers and facilitators: a post-course confidence survey is measuring the thing these studies show to be unreliable. A session that ends with a self-rating has recorded a feeling, not a change.
An attendance record proves a person was in the room. A tool certificate proves they can operate the tool. A confidence score proves how they feel. None of the three involves a decision with something riding on it, and judgment only appears in a decision with something riding on it.
This is why the records cannot be improved into evidence. A longer course is still attendance. A harder tool certificate is still tool skill. A better-designed survey is still self-report. The gap is in kind, not degree.
The AIR APAC Standard for Judgment in AI-Enabled Work, AIRS-J/1, puts the rule in a single clause. Self-assessment, survey response, declared experience, training attendance and manager endorsement generate no observations and contribute nothing to a score. That is a firm line, and the research above shows why it is worth drawing.
You should expect to be asked for capability evidence by someone who matters. A client's procurement team, an auditor working through the organisation's AI management system, a regulator in a market where oversight has to be documented. Every one of them can read an attendance rate. Not one of them can learn from it whether your people check what the tool tells them.
For HR and L&D: the learning management system is not wrong. It is answering a different question. Keep it for completion and reach. Stop presenting its output as a capability measure.
What replaces the three records is an observation. In AIRS-J/1 an observation is a decision a person made, recorded as they made it, with the sources they consulted, the reasoning they stated and the point at which they committed. It is scored against a public anchor that describes what the behaviour looks like when absent, partial, adequate, calibrated or transferable.
Three rules make an observation evidence rather than impression. First, the trace is retained in full, so a scorer reads what the person did and can show it to them. Second, a prompted response caps at Level 1: if the exercise nudged the person towards checking a claim, that observation scores no higher than Level 1. Third, each of the nine behaviours needs at least three observations from distinct decision points. Two observations from the same decision count as one.
Under those rules a result can be wrong in the ordinary way, by misreading a trace, and corrected by reading the trace again. A confidence score cannot be corrected, because there is nothing beneath it to re-read.
An attendance record proves a person was in the room. Judgment only appears in a decision with something riding on it.
The practical consequence for a capability report is a different unit. The unit is no longer the course or the person-hour. It is the level a person demonstrated on a behaviour, and the band a person reaches across all nine. For an assessment, AIRS-J/1 gives a commissioning organisation a team view: counts of levels and bands across the people assessed, never a ranking, and only where ten or more people were assessed. Where an employer commissions an assessment of its own people, it also receives each person's result by name, including the evidence behind every observation. There is no league table because a league table invites false precision, and AIR APAC publishes none.
None of this says training is wasted. AIR APAC runs its own Level 1 course, and the behaviours the Standard describes are learnable. The claim is narrower. A course can build a capability and cannot, by itself, show that the capability exists. The showing is a separate act, done on a separate occasion, under conditions designed to make the behaviour visible.
The same separation already exists elsewhere in your organisation. A driving lesson is not a driving test. A first-aid course is not a first-aid certificate until someone watched the candidate act. Nobody confuses the two in those fields, and the only reason they are confused for AI is that the capability reports arrived before the assessment did.
For HR and L&D: the two records belong in two columns. One column says who has been trained. The other says what levels they demonstrated and when. A board paper that merges them is reporting hours as competence.
For trainers and facilitators: the strongest thing a training provider can offer is a programme that builds towards a published rubric and sends participants to a separate assessment on it. The rubric is public. AIRS-J/1 certification is open, and no sitting has run yet. Practice never awards it, and that protects the trainer as much as the candidate.
The hours were spent and the confidence went up. Whether anyone can now tell when the answer is wrong is a separate question, and it needs a separate record.
AIRS-J/1 assesses how a person decides, not what they know. A result is not a measure of intelligence, personality, AI technical skill or domain expertise, and it is not a substitute for supervision, professional licensing or regulatory authorisation. The studies cited measure self-assessment, reported critical thinking and task performance in specific populations. None of them scored anyone against the Standard, and the argument that their findings apply to judgment in AI-enabled work is an inference. Nine dimensions is a reporting hypothesis; the Standard says it will reduce the number if the evidence requires it. A team view reports counts of levels and bands; it does not rank people and AIR APAC publishes no league tables.
AIR APAC publishes these articles in the institution's voice. Every figure carries its source and its limit. All articles.
Judgment sounds like a trait. In AIRS-J/1 it is nine observable behaviours, each with a level from 0 to 4. One ordinary scenario, walked through all nine, shows what Level 2 looks like and why Level 3 is different.
If your reviewers approve 99.8 per cent of AI outputs, that number is a risk signal, not proof of control. Singapore's agentic AI framework now suggests that override rates be tracked. The clinical evidence shows why.
Singapore's agentic AI framework asks for human approval checkpoints. Korea's AI Basic Act asks for documented oversight. Both assume you hold a list of decisions a machine, or a junior, may not take. Most departments have the policy and not the list.