How are held-back scenarios kept apart from practice scenarios?
The scenarios that certify a person under AIRS-J/1 are held back from every demonstration and every practice run. This page sets out the rules that keep them apart, in the words of the Standard and of the official instrument's published specification, and says what a practice run may and may not be used for.
Built by Exeter Labs
What is a held-back scenario?
Judgment is how well a person decides once AI has given the answer. AIRS-J/1, the standard for judgment in AI-enabled work, reads it from the decisions a person makes in a scenario. A held-back scenario is a scenario used only for certification. It is kept out of every demonstration and every practice run.
StepSim is the official instrument of AIRS-J/1, and its specification is incorporated in the Standard by Annex F (Section 16.3).
Exeter Labs builds and operates StepSim, AIR APAC's official assessment instrument. The owner of AIR APAC also owns Exeter Labs.
The specification calls its scenarios "missions". Section 8.4 of the Standard sets the rule for a certifying result: "StepSim's rule is at least three held-back Assessment scenarios". The specification states that held-back missions are "never shown on any demo, the showcase or the film" (specification section 7.3). A certifying mission's package "is never served to a browser" (specification section 7.3): it stays on StepSim's server.
Why are practice and certifying content kept apart?
Section 4.3 of the Standard states: "Practice content and certifying content are kept apart (Section 9.3)". Section 9 gives the reason: "A simulation-based credential fails at the point where scenarios leak or identity is unverified". In plain words, a result earned on a scenario the person had already seen would show what the person remembered. It would not show how the person decides.
Can a scenario move from practice into certification?
The Standard describes no route from practice into certification. Section 9.3 retires a certifying scenario "when it appears in public preparation material". The specification states that retirement "is from Assessment only" and that "a retired case may be released for practice" (specification section 7.3). The movement the Standard describes runs one way: out of certification, into practice.
Can a retired scenario be used for practice?
It may be, once it is released. Section 9.3 states: "Retired scenarios may be released for practice. AIR APAC publishes the list of released practice scenarios, so that trainers and institutions know which content may be used in practice". A scenario on that list is no longer used for certification.
What stops a person meeting a certifying scenario they have seen?
The specification sets the rules (specification section 7.3). StepSim's server counts, for each version of a mission, the people who have seen it. "The published threshold is 200 people per case version". The specification calls one edition of a mission a case version. When 200 people have seen a version, it is retired from Assessment. The specification adds that the threshold "is a starting value" and that "the first validation report reviews it".
The specification also states: "A person never gets a mission they have seen, in any session the record links". On 24 October 2026, the record links a person's Assessment runs, which are the only runs that carry a person key. When every version of a mission is retired, or the person has seen it, the server refuses the start.
The Standard also requires parallel forms. Section 9.4 states: "Each track maintains at least two non-overlapping certifying forms of equivalent stakes coverage. A candidate reassessing after a Not Certified outcome receives a different form". On 24 October 2026, the published specification lists the second form among the items still to be built (specification section 7.3).
What happens if a scenario leaks?
A leak is scenario content that reaches people outside the assessment. Section 9.5 states: "AIR APAC monitors for published solutions and coaching content and treats confirmed leakage as a retirement trigger, not a marketing problem". On StepSim, a leak report retires the mission from Assessment at once, every version of it (specification section 7.3). The specification gives the reason: the versions of a mission "share most of their stimulus, so a leak of one exposes the others".
How is a practice run different from an Assessment run?
A practice run may guide the person. The specification lists the guides: marked passages, notices that point to a condition, worked examples, coaching and rewind (specification section 2.6). Every practice run records its scaffolding as "guided". Section 7.1 of the Standard limits what a guided response can count for: "A prompted response caps at Level 1 for that observation". The same section adds that "certification may not silently roll a scaffolded response into a comparable total". Levels per dimension come only from an Assessment (Section 20.3), so the cap does not give a practice run a level.
An Assessment run records its scaffolding as "none". The specification states: "Only Assessment runs yield certifying observations". A practice run may end with a debrief. An Assessment run ends on one fixed screen that gives no reading of the scenario, "because the case must stay unexposed" (specification section 7.3).
A practice run gives a development reading. Section 3 of the Standard defines it. A development reading "shows decisions and how the published rules read them, carries no level, band or confidence, and decides nothing about employment or certification". Section 20.3 adds: "Levels per dimension and bands come only from an Assessment".
Can a practice result be used in a hiring, promotion or appraisal decision?
No. A session gives a development reading; it must not be used in a hiring, promotion or appraisal decision. Section 12.8 of the Standard names practice, a training session and a public demonstration as development readings: "they inform development and decide nothing about a person's employment or certification". It adds: "AIR APAC releases no development reading to an employer in a hiring context". Section 12.11 sets a required term of every commissioning licence: the organisation uses no development reading in a hiring, promotion or appraisal decision.
Does practising on the instrument help someone certify?
This page makes no promise about outcomes. Section 4.3 states: "Certification outcomes are decided by the rules in Section 12 alone, never by what a candidate or employer bought". The same section states: "A candidate may present for assessment without prior training". Section 20.2 states that a record of completion "confers nothing toward certification" and that "Certification needs a separate Assessment on held-back scenarios". Practice is for development. It shows a person their own decisions and how the published rules read them.
Who checks that this is done?
StepSim's server counts exposure and refuses a start the rules do not allow. A reported leak is recorded by an operator's command (specification section 7.3), and AIR APAC monitors for leaked content (Section 9.5). Section 9.3 names a third trigger: a shift in performance distribution "consistent with content leakage". The specification states that this trigger is read in the validation report, not by the server. Every reading is scored by published rules and traces to what the person did. The validation report is published as the checks in the Standard are met. Section 4.8 states what follows a finding: AIR APAC "fixes the instrument, the rule or the anchor, and the validation report records what changed".
What this does not claim
- It does not claim that the two parallel forms Section 9.4 requires exist today.
- It does not say how many scenarios exist, which they are or what any of them contains.
- It does not claim that practice raises anyone's chance of certifying, or that practice is a step towards certification.
- It does not give a practice or demonstration reading a level, a band or a confidence.
- It does not present the threshold of 200 as final: the specification calls it a starting value.
- It makes no research claim for the instrument beyond the research line above.
AIR APAC publishes the standard for judgment in AI-enabled work, certifies people against it through its official instrument, and stands behind every result.