Data Review: check the information behind one use of AI.
A fixed-scope review of one planned use of AI. We list the information it would rely on, check each source with the people who own it, and give you a findings table, a set of test questions and a list of fixes in order of priority.
The problem this review solves
An AI tool is only as good as the records it reads. Before an organisation relies on AI for a task, it needs to know which information the AI will use, who owns it, whether it is complete and current, and whether it may be used this way at all. Most organisations find these answers only after the AI has given a wrong answer with confidence.
A fictional example. A customer-service assistant answers questions from the company's policy files. The files hold three versions of the refund policy. The assistant quotes the oldest one to customers for two months before anyone notices.
Who it is for
- Heads of operations, data, IT or a business function who plan to connect an AI tool to the organisation's own information.
- Teams choosing between AI suppliers who want to know first what their own records can support.
- One planned use of AI at a time, for example a search tool over policy documents, or supplier recommendations from purchase records.
What we do
One planned use of AI and the sources it would rely on, agreed in writing at the start.
Scope
Agree the use of AI, the question it must answer, and the sources it would read.
List the sources and their owners
For each source, record where it comes from, who is responsible for it, how often it is updated, and who may see it.
Check each source with its owner
Go through the seven checks with the people who own each source: source and owner, quality, access and permission, freshness, meaning, tracing an answer, and testing. Where a check needs records, your staff run it in your systems and show us the result.
Write the test questions
With your team, write a set of questions where you already know the right answer. Your team runs them against the AI tool and shows us the results, before you rely on it and again after each change. We do not run them.
Report and brief
A findings table, a list of fixes in order of priority, and a one-hour briefing for the owner of the use of AI.
What you receive
- A list of the sources, with the owner of each one.
- A findings table: each problem, the source it is in, who owns it, what it would do to the AI's answers, and the fix.
- Gaps in access and permission: information the AI should not use, or may not be allowed to use.
- A set of test questions with known answers.
- A list of fixes in order of priority, for your own team or any supplier.
Customer-service assistant: three findings
| Problem | Source and owner | Effect on the AI's answers | Fix |
|---|---|---|---|
| Three versions of the refund policy | Policy library, customer-service manager | The assistant may quote an old policy | Keep one current version; archive the rest outside the library |
| Product codes differ between two regions | Product records, regional operations heads | Answers about stock may mix two products | Agree one code list; map the old codes |
| Staff-only notes stored with customer replies | Ticket system, IT | The assistant could show internal notes to customers | Remove the notes from what the assistant may read |
Who does the work
You contract with AIR APAC Pte. Ltd., which is responsible for the review and its report. The review is led by Indhran Seghar, director of AIR APAC, with one lead for the whole engagement from the scoping call to the final briefing. The people behind AIR APAC
If the fixes need data cleaned, moved or connected, or software built, Exeter Labs, the product company of AIR APAC, can do that work under a separate agreement, or you can use your own team or any other supplier. Using Exeter Labs is never a condition of the review. Because Exeter Labs is part of AIR APAC, AIR APAC will not review its work for you as an independent party. Your contract names who does each task and who owns each deliverable.
Your information
We work with your staff, in your own systems, under your own access rules. AIR APAC does not take copies of your data, store it or move it, and does not put it into any AI tool. Where a check needs records, your staff run it and show us the result. We rely on the results your staff show us and do not verify them independently. Notes we take are used only for the review and are returned or deleted at the end, as the contract sets out.
What this review does not do
- It does not clean, move, store or connect data, and does not build software. That work is contracted separately.
- It is not a full audit of your data. It covers the sources for one use of AI.
- It does not certify data security or compliance with any law, and gives no legal opinion.
- It gives no score, grade or certificate of readiness.
Length and fee
Usually two to three weeks for one use of AI, depending on how many sources it relies on and how quickly their owners are available.
The fee is fixed and agreed in writing before work starts. It depends on the scope: how many sources the use of AI relies on. We quote it in writing after a free scoping call of about 45 minutes. Any data or build work that follows is quoted separately by whoever does it.
The review is complete when you have received the items listed under What you receive and the briefing has taken place.
AIR APAC also publishes research on organisations. We do not sell a review to an organisation while our research is measuring it, and no research result is used to sell a review. How AIR APAC keeps research separate from sales
Data Review
Request a scoping call
Tell us the use of AI you are planning, the information it would rely on, who owns that information today, and when you want to start.