Why does no model's opinion stand alone as a reading?
When an AI model helps to score an Assessment, AIRS-J/1 treats its output as a proposal, never as a reading on its own. This article sets out what Section 11.4 requires and what these requirements protect.
Judgment is how well a person decides once AI has given the answer. AIRS-J/1, the Standard that AIR APAC publishes for judgment in AI-enabled work, reads it from what a person did in a recorded Assessment. AIR APAC states the scoring rule in two sentences: "Every reading is scored by published rules and traces to what the person did. No model's opinion stands alone as a reading."
The second sentence rests on Section 11.4 of the Standard. In its words, a model-generated score "is not an observation and never stands alone".
Two kinds of scoring
Section 11.4 separates two ways a result can be scored.
The first is deterministic rule-based scoring by an approved instrument. The instrument applies published rules to the retained record of what the person did. In the Standard's words, it "generates observations directly: every level cites the events and the rule that produced it, and no human confirmation is needed". Where an approved instrument scores by rule, AIR APAC issues the certification decision under Section 12 (Section 11.6).
The second is model-generated scoring, for example by a large language model. A model may help a person to score. Its output is a proposal, and a person decides what the observation is.
What the Standard requires of a model's proposal
Section 11.4 sets six requirements when a model takes part in scoring:
- A model may propose a level, with cited evidence from the transcript.
- A qualified human assessor confirms or overrides the proposal, and the reasons are recorded.
- The proposal and the assessor's decision count as one observation, attributed to the human assessor.
- Where the model and the assessor differ by more than one level, the observation goes to a second human assessor, whose decision is final.
- The model-assisted proportion is recorded on the result. Section 11.4 also states that it is "disclosed in the register entry".
- The model version and the prompt version are recorded in the transcript.
Two further sections add to these. Section 11.5 requires model-assisted scoring to be checked again against a separate set of transcripts scored by people. This happens on each model or prompt change, and at least once every three months. If the scoring fails the agreement test of Section 11.2 against human scores, automated scoring is suspended for the dimensions affected. Section 11.6 adds that where a human assessor scores or confirms an observation, only an assessor authorised under AIR APAC's assessor standard may do so.
What these requirements protect
This is AIR APAC's plain reading of how the requirements connect to other provisions of the Standard.
Evidence that can be shown to the person. Section 8.8 requires every observation to trace to identified evidence in the transcript. In its words, "An observation whose evidence cannot be pointed at does not exist". Section 8.7 adds that the Standard "reads what a person did, which can be shown to them and explained". A model that proposes a level must cite transcript evidence (Section 11.4). Under Section 7.3, an assessor may not score on inferred motivation, attitude, confidence or personality.
A person to whom a model-assisted observation is attributed. The observation is attributed to the authorised human assessor, not to the model (Sections 11.4 and 11.6). Section 12.6 states that AIR APAC "is responsible for applying this Standard correctly and for the reading it gives". A person confirms each model-assisted observation, and AIR APAC is responsible for the reading.
A record that can be checked. Rule-based scoring "must reproduce": the same transcript always yields the same observations and levels (Section 11.2). Where people score, Section 11.2 sets a minimum level of agreement between independent assessors on each dimension. Section 9.6 requires every transcript to record the scenario version, the rubric version, the instrument version and the scaffolding level. A result that cannot state them is void. Section 11.4 adds the model version and the prompt version to that record.
A record that can be appealed. A candidate may appeal a certification-status decision or an employment-grade result released to an employer (Section 18.5). The grounds for appeal include "an observation not supported by transcript evidence" and "misapplication of an anchor" (Section 18.6). A procedural failure under Section 11 is also a ground. An appeal review "considers the transcript, published anchors, relevant access-support record and applicable procedures" (Section 18.5). Cited evidence and recorded reasons give the appeals reviewer, who was not involved in the original decision or delivery, something to examine.
What the candidate is told. Every candidate receives a report that includes the model-assisted proportion and the evidence cited for each observation, whether or not they certify (Section 12.5).
One fictional decision, worked through
A fictional example.
Shanti Devi is supplier quality lead at Tanjung Selasih Packaging Sdn Bhd, a packaging manufacturer in Penang. A new resin supplier must be rated before a first order is placed on Wednesday.
ChlorophoniaAI, the purchasing team's risk tool, proposes a rating and cites its source: "Supplier risk is low. The supplier's questionnaire states that every batch is tested before dispatch. I am 85 percent sure."
Shanti opens the cited source. It is the supplier's own answer, with no test certificate behind it. She asks the supplier for the certificates of the last three batches.
She does not accept the proposed rating on its own. She approves one trial order only, and she records why she overrode the tool. She tells the purchasing head and the plant manager.
Two certificates arrive. The third batch had not been tested. The team keeps one rule: a tool's proposed rating counts only once a person has opened what it cites and recorded the reason.
Section 8.2 of the Standard sets a parallel rule for an Assessment: self-report is not evidence. A proposal with cited evidence is useful, and a person still has to confirm it and take responsibility for it.
What this does not claim
- It does not claim that AI models score well or badly. It describes the rules the Standard sets for model assistance.
- It does not claim that no AI model is used in scoring. Section 11.4 allows a model to propose a level under the rules above.
- It gives no figure for the model-assisted proportion of any result. It does not describe how the register displays that proportion.
- It reports no validation finding. Every reading is scored by published rules and traces to what the person did. The validation report is published as the checks in the Standard are met (Section 11.7).
- The fictional example is a workplace decision. It is not an assessment scenario, and it does not show how an Assessment runs.
AIR APAC publishes the standard for judgment in AI-enabled work, certifies people against it through its official instrument, and stands behind every result.