What is the difference between a session and an assessment?
A session and an Assessment run on the same official instrument and are read by the same published rules. They differ in the content used, the result the person receives, who sees it and what it may decide.
Judgment is how well a person decides once AI has given the answer. AIRS-J/1, the AIR APAC Standard for Judgment in AI-Enabled Work, describes how a person decides when working with, alongside or in supervision of AI systems. A training session and an Assessment both run on the Standard's official instrument. Both are read by the same published rules (Sections 7, 16.3, 20.3). The published rules belong to the Standard. In its words, "an instrument implements it, it does not define it" (Section 16.3). What changes between the two is the content used, the result, who sees it and what it may decide.
A session and an Assessment compared
| Session | Assessment | |
|---|---|---|
| What it is for | Practice and development. The person sees their own decisions and how the published rules read them (Section 20.3). | A reading against the Standard's decision rules, which can lead to certification (Section 12). |
| The scenarios used | Practice content. Scenarios retired from certification use may be released for practice, and the Standard says AIR APAC publishes the list of them (Section 9.3). | Certifying scenarios, kept apart from practice content (Sections 4.3, 9.3). The official instrument's rule is at least three held-back Assessment scenarios (Sections 8.4, 9.3; Annex F). |
| What the person receives | A development reading: the person's decisions and how the published rules read them. It carries no level, no band and no confidence (Sections 3, 20.3). | The report set out in Section 12.5, whether or not the person certifies. It includes every separately reportable dimension score, the four domain scores, the band, the confidence and the evidence cited for each observation. A Provisional or Insufficient result has no band (Section 10.1). |
| Who sees it | Participants are told who sees their reading before they start (Section 20.3). When an employer commissions a session, participants are told which roles receive their reading by name. It reaches the employer for development only (Section 12.6). | Candidates are told before they start who receives the result and what it may be used for. An employer that commissions it for its own people receives each result by name. An independent candidate's result goes only to the parties the candidate names (Section 12.6). |
| What it may decide | Nothing about a person's employment or certification (Sections 12.8, 20.3). | At Moderate or High confidence only, the result is employment-grade. An employer may rely on it as one input to a hiring, promotion or appraisal decision. The decision about the person stays with the employer (Sections 12.6, 12.8). Hiring use opens in a market only once the access-support procedure and applicant notice are published for it (Section 12.9). |
| The record | A completion record that states "This is not an AIRS-J/1 credential". It carries no level, band or confidence and confers nothing toward certification (Section 20.2). | If the person certifies, a credential recorded on the public register. It is verified by its code, and verification is free (Section 14). |
A session does not certify a person's judgment. Certification needs a separate Assessment on held-back scenarios (Sections 8.4, 9.3, 20.2).
When an Assessment gives a development reading
Confidence takes one of four values: High, Moderate, Provisional and Insufficient. It describes how much evidence a result is based on, not the person's ability (Section 10). An Assessment result at Provisional or Insufficient confidence is also a development reading. It carries no band and is never a basis for a hiring, promotion or appraisal decision (Sections 10.1, 12.8). Where an employer commissioned the Assessment for a hiring decision, the employer is told only that no employment-grade result was obtained (Section 10.1).
What an organisation that commissions a session must do
A session gives a development reading; it must not be used in a hiring, promotion or appraisal decision.
Before a session, participants are told that it gives a development reading, and that a hiring or certification decision uses a separate Assessment (Section 12.6). Section 12.8 requires every commissioning licence (Section 12.11) to bind the organisation not to use a development reading in a hiring, promotion or appraisal decision. AIR APAC releases no development reading to an employer in a hiring context (Section 12.8).
What this does not claim
- That a session certifies anyone, prepares a person for the Assessment's scenarios, or raises the chance of certifying (Sections 4.3, 20.2).
- That a session reading has a level or a band. Levels per dimension and bands come only from an Assessment (Section 20.3).
- That any result says a person should be given a particular job. A result describes the person's judgment in AI-enabled work at the level certified (Sections 2.2, 12.6).
- That a person's result is a percentage or a place in a ranking. The band is the reading, and AIR APAC provides no rank order of applicants (Section 12.6).
AIR APAC publishes the standard for judgment in AI-enabled work, certifies people against it through its official instrument, and stands behind every result.
Built by Exeter Labs
StepSim, the official instrument
The sessions and Assessments on this page run on StepSim. StepSim is an AIR APAC initiative and the official instrument of AIRS-J/1. Its specification is incorporated in the Standard by reference (Sections 4.9, 16.3; Annex F). Exeter Labs builds and operates StepSim, AIR APAC's official assessment instrument. The owner of AIR APAC also owns Exeter Labs.