Why is Level 2 the threshold?
Under AIRS-J/1, a person must have an overall mean of at least 2.3 and Level 2 on every dimension to be certified. This section explains what Level 2 means, how that requirement works with the floors, how it was set and how it is checked.
Judgment is how well a person decides once AI has given the answer. AIRS-J/1, the Standard that AIR APAC publishes, scores it on nine dimensions in four domains, on a scale from Level 0 to Level 4. A person is certified only when every dimension reaches Level 2, the overall mean reaches 2.3, and there is enough evidence to know both.
What Level 2 means
Every observation in an Assessment is scored on one scale (Section 7).
| Level | Label | General meaning, in the Standard's words |
|---|---|---|
| 0 | Not demonstrated | The behaviour was absent, or its opposite was displayed. |
| 1 | Emerging | The behaviour appeared partially, late, or only after prompting. |
| 2 | Competent | The behaviour appeared unprompted and was adequate to the situation. Threshold level. |
| 3 | Proficient | As Level 2, calibrated to consequence, and made inspectable: recorded or stated so another person could audit the reasoning. |
| 4 | Exemplary | As Level 3, and the judgment is made transferable: the candidate leaves a reusable rule, condition, indicator or check that would govern the next decision of this class. |
The Standard names Level 2 as the threshold in the scale itself. Level 2 is the first level at which the behaviour appears without being asked and is adequate to the situation. Levels are cumulative: each level requires everything in the level below it (Section 7.4).
Why "unprompted" matters
Level 2 needs two things: the behaviour appears unprompted, and it is adequate to the situation. Level 1 covers behaviour that appears partially, late, or only after prompting. Where an instrument prompts a behaviour, the level of prompting is recorded in the transcript, and "A prompted response caps at Level 1 for that observation" (Section 7.1).
An approved access support, such as a change of timing or format to remove a barrier unrelated to what is assessed, is not prompting (Section 7.1.1). The level is read from what appears in the transcript, never from motivation, attitude, confidence or personality (Section 7.3).
What the levels above add
Level 3 adds two things to Level 2. The response is calibrated to what is at stake, and the reasoning is recorded or stated so that another person could audit it. Level 4 adds one more: a reusable rule, condition, indicator or check for the next decision of the same class.
The bands above Certified start at higher composites and higher floors (minimum scores, set out below; Section 12.4):
- Certified: overall mean (the composite) 2.3 to 2.9, with all floors met.
- Certified with Merit: composite 3.0 to 3.4, with no dimension below 2.5.
- Certified with Distinction: composite 3.5 and above, with no dimension below 3.0.
Certified results are reported in bands, not raw composites, so that a result does not suggest false precision or serve as a ranking (Section 12.4).
Level 2 everywhere, and 2.3 overall
A dimension score is the mean of its valid observations. A domain score is the mean of its dimension scores, and the composite is the mean of all nine dimension scores (Section 12.2.1). So Level 2 on a dimension means a score of at least 2.0, not Level 2 on every observation. All six rules of Section 12.2 must be met for certification.
- Dimension floor: no dimension below 2.0.
- Domain floor: each of the four domains (Structure, Test, Engage, Perform) at or above 2.0.
- Composite: the mean of all nine dimensions at or above 2.3.
- Evidence: enough observations on all nine dimensions (Section 8.3).
- Stakes coverage: the scenarios cover the consequence families the Standard requires (Section 6.4).
- Confidence: Moderate or High.
The Standard explains why the rules are not redundant (Section 12.3). "A profile at exactly 2.0 across all nine yields a composite of 2.0 and fails rule 3". The floors stop a candidate reaching 2.3 on three high dimensions while one sits at 1.0. Strength in one domain does not offset weakness in another (Section 12.1).
Rules 1 to 3 govern the level. Rules 4 to 6 "govern whether the level is knowable" (Section 12.3).
How the cut was set, and how it is checked
Section 11.1 sets out how the thresholds, also called cut scores, were set and how they are reviewed.
- Initial cuts. AIR APAC set the thresholds on the review board's advice as initial cuts. They were set from the anchor meanings ("Level 2 is the threshold behaviour") and from the synthetic panels (simulated candidates, Section 11.7) in the Standard's checks V1 to V3.
- A yearly review. The thresholds are reviewed each year against the evidence programme (Annex A), including classification consistency near the cut score (Section 11.8).
- A study on real data. The Standard requires a standard-setting study with practitioners in a role family. The study must use the first real criterion data (ratings of how people perform in the role) for that family (Annex A19), and be published with its method.
- A change is a major version. A change to a threshold is a major version of the Standard. The review board reviews every major version before release (Section 19.1).
Below the threshold
A result that misses any one of the six rules does not certify (Section 12.2). Where the shortfall is in the evidence, Section 10 reports it as Provisional or Insufficient, a development reading with no band. The candidate still receives the report set out in Section 12.5, with every separately reportable dimension score, the four domain scores and the evidence cited for each observation. The Standard provides it "whether or not the candidate certifies". An Insufficient result shows the number of observations per dimension, not a level (Section 10.1).
A candidate may sit again no sooner than 30 days after a Not Certified outcome, and receives a different form (Section 9.4).
What this does not claim
- That the cut has been confirmed on real criterion data. The thresholds are initial cuts. This section reports no finding from the study that Section 11.1 requires.
- That a certified result forecasts performance in a role. Under Section 2.1, that claim for a role family is not made until check V8 is published for that family. Every reading is scored by published rules and traces to what the person did. The validation report is published as the checks in the Standard are met.
- That Level 2 is a junior level, or Level 4 a senior one (Section 7.2).
- That a person below the threshold cannot do their job. The Standard certifies judgment in AI-enabled work within an approved assessment; it does not certify occupational competence (Section 2.2).
- Any reason for the cut beyond the one the Standard gives.
- That this section prepares anyone for certifying scenarios. A scenario that appears in public preparation material is retired from certification (Section 9.3).
AIR APAC publishes the standard for judgment in AI-enabled work, certifies people against it through its official instrument, and stands behind every result.
Fees are confirmed at registration. Verification is free.