Implementation Guide
Testing an AI Receptionist Before Go-Live
Testing is the cheapest part of the project and the part most often cut. This guide sets out a scenario library, acceptance criteria you define yourself, and a sign-off checklist to work through before real callers arrive.
What good testing looks like
Good testing is a written list of scenarios, each with an expected outcome, run by more than one person, with results recorded and retested after every change. Ringing the number twice and deciding it sounds fine is not testing.
Test the awkward calls specifically. Any flow handles a caller who states their request clearly in one sentence; the ones that expose real problems are the caller who changes their mind, the caller on a noisy MRT platform, and the caller whose enquiry does not fit any of your categories.
Set your own pass criteria before you start. Without them, testing becomes a matter of opinion, and the loudest opinion in the room decides whether you launch.
Build a scenario library
Six categories. Write at least two scenarios in each, using wording your real callers would use.
Straightforward calls
The common enquiries, stated plainly. New booking, reschedule, opening hours, location, service availability. These should complete without a transfer.
Messy calls
Callers who interrupt, change their mind mid-sentence, give a date and then correct it, or ask three questions at once. Check that details are corrected rather than compounded.
Audio conditions
Calls from a car, a hawker centre, an MRT platform and a poor mobile connection. Confirm that unclear input leads to a polite clarification rather than a confident mistake.
Language variation
Different accents, code-switching mid-sentence, and a language you have not configured. The unsupported case must produce a clear route, not an apology loop.
Out of scope
Sales calls, wrong numbers, complaints, refund demands and questions about services you do not offer. Each should end somewhere sensible rather than in a loop.
Hard escalations
Test every fixed exclusion deliberately, including phrasing that is indirect or emotional. These must reach a person immediately, with no attempt at an answer.
Set acceptance criteria you can judge
Define what counts as a pass before testing begins, and set the thresholds yourself based on what your business can live with. The criteria below are the dimensions that matter; the standard is yours to set.
| Criterion | What you are judging | How to record it |
|---|---|---|
| Outcome correctness | Did the call reach the outcome the scenario intended | Pass or fail against the written expectation |
| Data accuracy | Name, contact number, service, date and time captured exactly as given | Compare the record with the tester’s script |
| System write | Booking or enquiry visible in the correct system, with the right owner | Check the calendar or CRM after each test call |
| Escalation integrity | Every excluded subject reached a person without an attempted answer | Any failure is a blocker, regardless of other results |
| Call length | Whether questions are proportionate to the enquiry | Note calls that felt long, and which question caused it |
| Caller experience | Whether the call would embarrass you if a customer described it | Tester’s written comment, reviewed by the flow owner |
How to run the tests
Recruit testers
Use at least three people, including someone who does not know the flow. Familiar testers unconsciously phrase requests the way the system expects.
Run the library
Each tester works through the scenarios, recording the result and any comment immediately after each call rather than from memory later.
Triage the failures
Separate genuine faults from unrealistic tests. Group the real issues by cause: wording, routing, data capture or integration.
Fix and retest
After changes, rerun the affected scenarios and a fixed core set. Changes to shared wording can affect paths you did not intend to touch.
Test out-of-hours behaviour genuinely out of hours, and test public holiday behaviour before the holiday rather than during it. Simulated conditions hide routing mistakes that only appear on a live schedule.
Sign-off checklist
- Every scenario in the library has been run and recorded
- All hard escalations reached a person, with no attempted answers
- Bookings appear in the correct calendar with the correct duration and owner
- Enquiry records contain every field your follow-up process requires
- Unsupported languages produce a clear stated route
- Out-of-hours and holiday routing tested on the live schedule
- Transfer destinations answered, and the recipients confirmed the context was useful
- No promise of price, availability or outcome was made without authorisation
- A rollback route is documented and someone owns the first fortnight of review
Escalation failures are blockers. If a test call about a child, a symptom, a security incident, an allergen, a distressed caller, a grievance or regulated advice was handled by the automation rather than routed to a person, do not proceed to go-live until it is corrected and retested.
Testing does not stop at launch. Rerun a core scenario set after any change to wording, routing, integrations or business hours, and treat that regression set as part of your operating routine.
Frequently asked questions
How many test calls are enough?
Enough to cover every scenario in your library at least once, with the common paths and all hard escalations repeated by different testers. Coverage matters more than volume.
Who should do the testing?
A mix: someone who knows the flow, someone who does not, and at least one person who takes real calls today. The last group notices things that documentation misses.
Should we test with real customers?
Not during structured testing. Once the library passes, a limited go-live on overflow or after-hours calls gives you real conditions with contained exposure.
What if a test result is a matter of opinion?
That usually means the acceptance criterion was not written precisely enough. Rewrite it as an observable outcome, then rerun the scenario.
How do we test escalation without disturbing colleagues?
Tell the escalation handlers in advance and agree a test window. Their feedback on whether the context arrived usefully is part of the test, so do not bypass them.
Do we need to retest after small wording changes?
Yes, for the affected paths and a fixed core set. Shared phrasing often appears in more places than expected, and call script writing changes can shift behaviour on paths you did not edit.
Related reading
The wider testing and implementation series is in the AI receptionist resources library.
Test it against your own awkward calls
Bring the enquiries your team finds hardest to handle. We will run them through a live flow so you can see where it holds up and where a person should take over.