When should you use it?
- You run a chatbot that collects details for quotes, bookings or support tickets
- Your tests mostly check that replies sound right, not what gets saved
- You change prompts often and want a quick regression set
Not for: Bots with no written list of required fields and rules; Load or uptime testing
What do you need, and what do you get?
You need
- Your bot's field list: what it must collect, allowed values and formats
- Every normalization rule, such as how relative dates, units and corrections are handled
- A way to read what the bot stored after each message
You get
- One test chat per rule, with the expected stored values after each customer message
- A list of rules with no test yet
- Pass or fail per step based on stored values
How does it work, step by step?
- Write down the rulesList each required field and every rule for turning what customers say into stored values.
- Generate the casesGive that list to the prompt below and ask for one short test chat per rule, with expected stored values after each message.
- Fix the expectations firstReview the expected values before any run, so a nice-sounding wrong answer cannot pass.
- Replay against the botRun each chat through your bot and compare what it stored with what you wrote down.
- Keep the setFix what fails, add a case for any spec gap you found, and rerun the set after every prompt change.
The prompt
Our own prompt, inspired by Adam (Adam13y) and written to work in n8n, ChatGPT and Claude. Values in lime are placeholders.
Fill in the blanks (optional; the copied prompt uses your values)
I need test conversations for a chatbot that collects details from customers. Fields it must collect, with allowed values and formats: {field_spec} Rules for turning what customers say into stored values: {normalization_rules} Business time zone: {time_zone} Write one short test chat for each rule above. Label each chat with the single rule it targets. After every customer message, give the exact field values the bot should have saved at that point, using the formats in the spec. Include at least one chat where a customer corrects a single field, one with a relative date close to midnight in our time zone, and one where the customer tries to make the bot ignore its rules. Finish with a list of any rule that still has no chat.
Inspired by Adam (Adam13y): Testing an n8n lead intake bot: 10 targeted conversations found 2 bugs a 200+ dialog test set missed. The prompt above is written by playbooks.bot in our own words; nothing from the original is quoted.
What can go wrong?
- Grading the bot's reply text instead of the saved values lets wrong data through; always check what was stored.
- Generated test chats can share the same blind spots as your bot if one model writes both; review them by hand.
- Time-of-day rules need tests near midnight in your business time zone, where many date bugs hide.
- Include at least one hostile message, such as an attempt to override the bot's rules.
Our take
This is a refreshingly concrete testing story. The insight is simple and widely useful: volume does not equal coverage. If every test chat is a polite customer giving tidy answers, you learn little about how your bot handles relative dates near midnight, units it was not expecting or a correction to a single field. The post's method fixes the expected stored values before the run, so a reply that reads well but saves the wrong number cannot pass. The post has no prompt, so we wrote one to generate such cases from a field spec. We tested it on a synthetic spec for a fictional home-cleaning quote bot with six fields and five rules. It produced seven chats, one per rule plus a malformed ZIP code and a prompt-injection attempt, each with expected stored values after every message and a clear pass condition. Review generated cases by hand, since a model can share your bot's blind spots.
Where the time goes back: A dozen aimed chats take an hour to write and catch the data bugs that otherwise surface as wrong quotes or confused staff weeks later.
FAQ
Why not just add more test chats?
Does this need n8n?
Is the prompt from the post?
Source and credit
Inspired by Adam (Adam13y). We describe the idea in our own words, link to the original, and wrote the prompt ourselves; nothing is quoted.
- Inspired by
- Testing an n8n lead intake bot: 10 targeted conversations found 2 bugs a 200+ dialog test set missed
- Creator
- Adam (Adam13y)
- Published on
- n8n Community
- Original date
- Oct 13, 2026
- License
- Not openly licensed (the poster keeps all rights), so we quote nothing and link to the original
- Originally for
- n8n
- AI dry run
- Oct 22, 2026: passed. Dry-run tested by our AI test agent on sample input. Method from the post; prompt is ours.
AI dry run on sample data; not run with live connected accounts. The playbooks.bot test agent filled the prompt with synthetic inputs, produced the output and checked it against the page's stated outputs and guardrails. - Verified
- Oct 23, 2026: source link resolves to the original, the credited creator published it, our excerpt or prompt matches it, and the license allows our use.
Are you the creator? Claim or correct this page.