What does confidence mean in a result?
The confidence on an AIRS-J/1 result describes how much evidence stands behind the result, not how able the person is. Each entry names the section of the Standard that sets the rule, quotes the Standard's sentence where it has one, and gives a plain reading that adds no claim to it.
Glossary of AIRS-J/1 · Confidence
Eleven entries explain the confidence on an AIRS-J/1 result. Eight entries give the Standard's wording first, with its section, and a plain reading beside it. The other three name the section that sets the rule, then give a plain reading. The plain reading adds no claim to the Standard.
Judgment is how well a person decides once AI has given the answer. Confidence is a different thing. It states whether the Assessment collected enough evidence to support the result.
Confidence
The Standard's wording. "Confidence describes evidential sufficiency, not the candidate's ability." (AIRS-J/1, Section 10)
Plain reading. Confidence states whether the Assessment collected enough evidence, of the right kind, to support the result. Every result carries a confidence statement. It is printed on the credential and shown in the public register (Section 10). Section 14.1 requires each register entry to record it. An employer can tell a High-confidence Certified from a Moderate-confidence Certified. The register does not record the employer or the dimension scores. The candidate's own report carries the confidence level, whether or not the candidate certifies (Section 12.5).
Confidence level
Where the Standard sets it. Section 10.
Plain reading. There are four confidence levels: High, Moderate, Provisional and Insufficient. They are separate from the levels 0 to 4 that describe a person's decisions on each dimension.
High
The Standard's wording. "Five or more observations per dimension, from four or more scenarios, scored by rules that meet Section 11.2. Stakes coverage under 6.4 exceeded." (Section 10)
Plain reading. More observations, from more scenarios, scored by rules that pass the scoring agreement test, across more than the minimum range of consequences. Certification may be issued at Moderate or High confidence only (Section 12.2, rule 6). A High-confidence result does not mean the person decides better than a person with a Moderate-confidence result. It means more evidence stands behind the result.
Moderate
The Standard's wording. "Three to four observations per dimension, from three or more scenarios, stakes coverage under 6.4 met." (Section 10)
Plain reading. Three or four observations, from at least three scenarios, across the required range of consequences. Certification may be issued at Moderate or High confidence only (Section 12.2, rule 6).
Provisional
The Standard's wording. "Minimum count met, but observations cluster in a single scenario, or stakes coverage under 6.4 is not met. Several scenarios in one sitting are not clustering." (Section 10)
Plain reading. The count is met, but the observations come from one scenario, or miss the required range of consequences. A Provisional result is a development reading and carries no band. It does not mean "nearly certified": the evidence was too narrow to support certification.
Insufficient
The Standard's wording. "Minimum not met for one or more dimensions. Reported as a development reading, with the observation count per dimension; no band." (Section 10)
Plain reading. At least one dimension has too few observations.
Minimum count
The Standard's wording. "Each dimension requires a minimum of three independent observations arising from distinct decision points." (Section 8.3)
Plain reading. The minimum count is three independent observations on each dimension. The observations must come from distinct decision points. Two from the same decision point count as one (Section 8.3). Moderate or High confidence also requires observations from at least three scenarios (Section 10).
Stakes coverage
The Standard's wording. "A certifying assessment must span at least three distinct primary consequence families, of which at least one must involve irreversible or non-financial harm." (Section 6.4)
Plain reading. Stakes coverage means the scenarios cover at least three kinds of consequence, one of them irreversible or not financial. A certifying Assessment must span at least three distinct consequence families, at least one involving irreversible or non-financial harm (Section 6.4). Without the required stakes coverage, no certification is issued.
Precedence (order of reading)
The Standard's wording. "Conditions are evaluated in the order Insufficient, Provisional, Moderate, High." (Section 10.1)
Plain reading. The conditions are read in a fixed order: Insufficient, then Provisional, then Moderate, then High. Once the minimum observation count is met, any Provisional condition makes the result Provisional, however many observations there are.
Employment-grade result
Where the Standard sets it. Section 12.8.
Plain reading. An Assessment result from an approved instrument, at Moderate or High confidence, or a valid credential shown by its holder, is employment-grade (Section 12.8). Only an employment-grade result may be used as AIR APAC's reading in a hiring, promotion or appraisal decision.
Development reading
Where the Standard sets it. Sections 10, 10.1 and 12.6.
Plain reading. Provisional and Insufficient results are development readings. They carry no band. They are reported to the candidate. Where an employer commissioned an Assessment of its own people, the employer receives them for development only (Sections 10.1 and 12.6). For a Provisional or Insufficient result in a hiring Assessment, the employer is told only that no employment-grade result was obtained (Section 12.6).
Three things confidence does not mean here
- It is not how sure the assessor feels. It follows from the conditions in Section 10.
- It is not the person's self-confidence. An assessor may not score on inferred motivation, attitude, confidence or personality (Section 7.3).
- It is not the certainty an AI system states about its own answer. It describes the evidence collected about a person.
What this does not claim
- Confidence is not a measure of ability. It is not a band and it is not a ranking.
- A High-confidence result does not mean the person decides better than a person with a Moderate-confidence result. It means more evidence stands behind the result.
- Confidence says nothing about whether a person suits a particular job. A certified result describes the person's judgment in AI-enabled work at the level certified.
- A Provisional result does not mean "nearly certified": the evidence was too narrow to support certification.
AIR APAC publishes the standard for judgment in AI-enabled work, certifies people against it through its official instrument, and stands behind every result.
Verify a certificate. After a credential is issued, its code can be used to verify its register entry. Each entry records confidence (Section 14.1). Verification is free.