How to test your AI assistant's answers before launch

Always actually ask your AI assistant questions before launch. Use a question set to check it systematically.

Preparing the question set

Three kinds of question:

• Questions with answers in the material, such as services and contact details.
• Questions without answers, such as delivery time and warranty, to see whether it honestly says there is none.
• Questions that invite over-promising, such as "can you guarantee first place" or "can I get a discount".

Ask each question several ways

Customers will not phrase things as you do. Ask the same question in casual, short and misspelled forms and check the answers agree.

What to look for

• Whether the answer is right, with no invented figures or phone numbers.
• Whether links that should be attached are attached and correct.
• When it cannot answer, whether it asks for contact details or hands over to a person.
• Whether the tone is the one you want.

Read each answer, not just the score

Automatic checks catch obvious errors, but "looks like a pass" does not mean correct. Read important questions by hand.

Keep tracking after launch

See what real customers ask and which questions are handed over, and add the missing answers to the knowledge base.