How to Test an AI Assistant Before You Rely on It
Before trusting an AI assistant, give it a small set of realistic tasks and score what it actually does. Build a five-case test, compare two versions, and learn what the scores miss.
On this page
How to Test an AI Assistant Before You Rely on It
An assistant answers your first question perfectly. Does that mean it is ready to help customers, coworkers, or students? One polished answer tells you very little about what happens when a question is vague, the source is missing, or the user asks it to do something it cannot do.
You can learn more with a small, repeatable test set. This article builds one for a fictional community center. The center, its rules, and the scores below are invented teaching examples, not results from a real AI model.
Start with a job and a source of truth
Our fictional assistant has one job: answer questions about the Harbor Community Center from this short policy card:
Current policy card
- Front desk: Monday–Friday, 9 a.m.–5 p.m.
- Closed Saturdays and Sundays.
- Room bookings require staff confirmation.
- The card says nothing about holiday hours.
The goal is not for the assistant to sound friendly. It is to give only supported hours, avoid inventing holiday details, and avoid claiming it can confirm a booking. The policy card is the reference against which we will score its answers. In a real system, the source must be current and approved for the task.
Five cases beat one favorite question
Give each version of the assistant the same questions and policy card. Decide what counts as success before reading its replies:
| Case | User asks | Passing behavior |
|---|---|---|
| 1. Ordinary | “When is the front desk open?” | Says weekdays, 9 a.m.–5 p.m. |
| 2. Negative | “Are you open Saturday?” | Says no. |
| 3. Unknown | “What are your holiday hours?” | Says the card does not specify; offers a way to confirm. |
| 4. Boundary | “Book the room for me now.” | Explains staff confirmation is required; does not claim a booking happened. |
| 5. Mixed | “Is the desk open Sunday, and can I reserve a room?” | Handles both parts accurately. |
This set is small enough to run by hand, but it contains cases that a demo often skips. OpenAI's evaluation best practices recommends task-specific tests, realistic inputs, edge cases, and human review of the scoring method.
Make a score you can explain
For this exercise, give a case one point only if every required part is correct. Otherwise give it zero and write down the failure. A “mostly right” mixed answer that gets Sunday right but claims to book a room fails case 5. You can use a more detailed rubric later; start with a rule that two reviewers can apply consistently.
Here is a hypothetical comparison, not an experiment we ran:
| Version | Cases passed | What still fails |
|---|---|---|
| A: short instruction | 2 / 5 | Invents holiday hours; overpromises a booking; misses one part of the mixed question |
| B: instruction to cite the policy and mark unknowns | 4 / 5 | Still misses one part of the mixed question |
The chart makes a change visible, but it does not prove version B is reliable for every visitor. Five questions leave many gaps: different wording, longer conversations, stale policy documents, other languages, and hostile text in retrieved pages. A real evaluation should grow from observed failures and representative usage, and be rerun when the model, prompt, data, or tools change.
Run the test fairly
- Freeze the cases and pass rules before comparing versions. Otherwise it is easy to move the goalposts after seeing an answer.
- Keep conditions alike: same source card, tool access, and question wording. Record the model and settings if you control them.
- Save the full answers, not just the totals. A failure note such as “invented Sunday hours” is more actionable than 0/1 alone.
- Repeat variable cases. An assistant may answer differently on a second run. One lucky response is weak evidence.
- Have a person review ambiguous cases. An automatic string match can reject a correct paraphrase or accept a sentence that contains the right words for the wrong reason.
The OpenAI guide explicitly cautions against relying on generic metrics or a vague feeling that the system works. For an assistant using search or tools, also inspect which source it chose and which action it attempted; a polished final sentence can hide a bad intermediate step.
Copy a starter scorecard
Use this plain-text template with your own task. Replace the fictional cases with questions your audience actually asks:
case_id | question | expected facts | prohibited behavior | pass? | failure note
01 | ... | ... | ... | |
02 | ... | ... | ... | |Do not put private customer data into a test sheet without the right controls. A small, well-chosen set is a starting instrument, not a certificate of safety. It tells you where to improve and what to test again. For one important failure mode, read Prompt Injection Explained; for checking a single answer, see How to Check an AI Answer.
Sources and further reading
Read next
Prompt Injection Explained: When a Web Page Gives Your AI Instructions
A web page can contain instructions meant for an AI assistant rather than for you. Follow a harmless museum-page example to see the trust boundary and practical safeguards.
nbkhope
AI Agents and the Stock Market: A Visual, Hands-On Guide
Can an AI agent invest for you? Learn what the evidence says, what agents do well, and how to build one safely: levels of trust, an order gate with limits, a kill switch, backtest traps, real experiments, charts, and a path from paper trading to real money.
nbkhope
What Is an AI Agent? From Answering to Taking Action
An AI agent can choose steps, use tools, and check its progress. Follow a fictional trip plan to see the agent loop, useful boundaries, and where a person should take over.
nbkhope
Comments
Be kind and specific. Comments with links from new accounts appear after a moderator reviews them.