What does 'published rubric, no undisclosed criteria' mean?
Section 4.1 of AIRS-J/1 commits AIR APAC to score every result by published rules alone. This section explains what is published, what is held back and why, and how a reader can check the rules a result was scored against.
Judgment is how well a person decides once AI has given the answer. AIRS-J/1 is the Standard for judgment in AI-enabled work. The phrase "published rubric, no undisclosed criteria" describes how results under it are scored. It sums up Section 4.1, which reads in full:
"The complete rubric is public and versioned. AIR APAC applies no undisclosed criterion, weighting, adjustment or override. Any criterion not in this document, or in the instrument specification it incorporates (Annex F), does not exist."
In plain words: a result is scored only by rules that anyone can read.
What "published rubric" covers
The published rubric is the whole Standard, with the instrument specification (Annex F). Its main parts are:
- the nine dimensions, grouped in four domains: Structure, Test, Engage and Perform (Section 5);
- the 0 to 4 scale, and the anchors that describe each level on each dimension (Section 7);
- the decision rules, thresholds and bands that turn scores into a certification decision (Section 12);
- the instrument specification (Annex F), which sets out how the instrument's recorded acts become observations for each dimension;
- the other scoring rules, among them the minimum observations (Section 8.3) and the critical-review patterns (Section 12.2.2 and Annex D).
The Standard is published in full at airapac.org/standard/v2-3. Each released version keeps a permanent URL and is never overwritten (Section 19.2).
What "no undisclosed criteria" rules out
AIR APAC adds no private factor, gives no dimension extra weight in secret, and changes no result by hand outside the rules.
Section 4.5 adds a rule for people outside AIR APAC. It is headed "No customer-authored criteria". Under it, delivery partners and licensed trainers "may not vary, extend, reweight or supplement the dimensions, anchors, thresholds or decision rules in this Standard."
A scenario may be adapted to a local setting, sector or language (Section 8.5). The criteria may not. The decision points, the dimensions each decision point shows, the stakes profile and the anchors stay the same.
What stays unpublished, and why that is different
The rules are public. The test content is not.
Certifying scenarios are held back. Under Section 9.3, a scenario is retired from certification use in three cases. Its exposure passes the set threshold. It appears in public preparation material. Or scores shift in a way consistent with content leakage. A retired scenario may be released for practice, and the Standard requires AIR APAC to publish the list of released practice scenarios.
Holding back a scenario adds no hidden criterion. A scenario is the situation a candidate decides in. The criteria are the published rules that score what the candidate did there.
How the rubric changes
The rubric changes only through a new version (Section 19). Clarifications, errata and added guidance are released as v2.x. Material changes to the competency model, the anchors, the evidence requirements or the decision rules are released as v3.0 (Section 19.2). A change to a threshold is a major version (Section 11.1).
The owner of AIR APAC approves every version (Section 19.1). The review board reviews every major version before release, and the owner holds a veto over its recommendations (Section 4.2).
A credential issued under an earlier version stays valid to its expiry and is shown with its version. AIR APAC publishes what changed between versions (Section 18.3).
How a reader can check
- The register records the standard version of every credential (Section 14.1), which points to the text the result was scored against. A credential can be checked free, by its code, at airapac.org/standard/register/verify.
- Every candidate receives the anchors applied and the evidence cited for each observation, whether or not the candidate certifies (Section 12.5).
- Misapplication of an anchor is a ground to appeal a certification decision or an employment-grade result (Sections 18.5 and 18.6).
- Disagreement with an anchor itself is not a ground for appeal. Section 18.6 treats it as a comment on the Standard and routes it to the review board.
Built by Exeter Labs
Who builds the instrument
StepSim, an AIR APAC initiative, is the official instrument of AIRS-J/1 (Section 4.9). Its specification is the document that Annex F incorporates into the Standard, and it is published at airapac.org/standard/instruments/stepsim. A change to that specification that changes a rule is released with a new version of the Standard (Annex F).
Exeter Labs builds and operates StepSim, AIR APAC's official assessment instrument. The owner of AIR APAC also owns Exeter Labs.
What this does not claim
- It does not claim that the rubric is final, or correct in every case. Where monitoring shows a claim needs strengthening, AIR APAC acts on it in the next release (Section 4.8). Anyone may submit a correction or an objection (Section 19.3).
- It does not claim that certifying scenarios are published, or that a practice scenario is a certifying one.
- It does not claim that customers or partners shape the criteria (Section 4.5).
- It does not claim that any regulator or standards body has approved the Standard.
- It covers certification only. The AI Readiness Index research rules are separate, and neither is a route to the other.
AIR APAC publishes the standard for judgment in AI-enabled work, certifies people against it through its official instrument, and stands behind every result.