What does the external academic reviewer do?
AIRS-J/1 names three review roles. These questions set out the one task the Standard gives the external academic reviewer. They also cover how the role differs from the review board and the appeals reviewer, and what it does not do.
Judgment is how well a person decides once AI has given the answer. AIRS-J/1 is the standard AIR APAC publishes for judgment in AI-enabled work.
How many review roles does the Standard name?
Three: the review board, the external academic reviewer and the appeals reviewer. Each role has a different job. The Standard lists the holder of the external academic reviewer role in its governance roster (Section 17). The page linked below publishes the roster.
What does the external academic reviewer do?
The Standard gives the external academic reviewer one task by name. The analysis of the V8 study is checked before publication by the external academic reviewer (Section 17), or by an independent specialist in the measurement of people through tests.
What is check V8?
Section 11.7 of the Standard sets out its validation programme as a list of checks. One of them, check V8, is a study of job performance. It asks whether Assessment results are related to later rated performance at work, for real people in a named role family. The margin and the criterion are declared before the study runs.
Is the report published whatever the result is?
Yes. The report is published whether the result is favourable or not (Section 11.7, V8).
Has the job-performance study run?
This page does not claim that the job-performance study (check V8) has run or has reported.
Does the role have any other task?
No. The Standard gives the role no other task. These answers add none.
What does the review board do?
The board is drawn from the standing councils of AIR APAC. It reviews every major version of the Standard before release and advises on the evidence programme. The owner of AIR APAC decides, and holds a veto over the recommendations of the board. The dissent of any board member from a released version is recorded in the version history (Section 4.2).
Who reviews appeals?
Appeals are reviewed by the appeals reviewer listed in Section 17. This is a review board member or an authorised assessor. The person was not involved in the original decision or delivery, and is not employed by or paid through Exeter Labs (Section 17; Section 18.5).
Does the external academic reviewer score anyone or decide any result?
No. Certification outcomes are decided by the rules in Section 12 alone. The owner of AIR APAC takes no part in any individual result (Section 4.3).
Does the external academic reviewer decide appeals?
No. Appeals go to the appeals reviewer (Section 18.5).
Does the external academic reviewer hold the veto?
No. The owner of AIR APAC holds the veto over the recommendations of the review board (Section 4.2).
Does the external academic reviewer endorse, approve or accredit?
No. The Standard gives the role no function of endorsing, approving or accrediting the Standard or any instrument. Annex E says its references inform design and external review and do not assert conformity, accreditation or legal compliance.
What else does the Standard require?
- A published rubric. The complete rubric is public and versioned. AIR APAC applies no undisclosed criterion, weighting, adjustment or override (Section 4.1).
- Findings act on the Standard. Where monitoring shows a claim needs strengthening, AIR APAC acts on it in the next release. The validation report records what changed (Section 4.8).
- Declared interests. The Standard declares the interests of the issuing body in Section 4.9. Exeter Labs builds and operates AIR APAC's official assessment instrument. The owner of AIR APAC also owns Exeter Labs.
- Interests on the board. A board member with an interest in a matter before the board declares it and does not advise on it (Section 17.1). The page linked below adds that council membership is not an endorsement of the Standard or of any instrument.
What this does not claim
- That the validation checks are complete. Every reading is scored by published rules and traces to what the person did. The validation report is published as the checks in the Standard are met.
- That the job-performance study (check V8) has run or has reported.
- That any university approves or endorses the Standard or any instrument.
- That the external academic reviewer reviews every result, every appeal or every version of the Standard.
- That the Standard conforms to any international standard or is approved by any regulator.
<!-- R66, R69, R73: on merge into the independence page, the build drops the declared-interest sentences, the closing action and any page eyebrow. -->