When should you use it?
- You run surveys with incentives
- You rely on panel or open-link responses
- Data quality matters for decisions
Not for: Surveys with no incentive and trusted respondents; Testing other people's surveys
What do you need, and what do you get?
You need
- Your survey link in a test mode
- A personal agent
- Response export
You get
- Agent test response
- List of checks to add
- Flag rules for suspicious responses
How does it work, step by step?
- Use test modeRun the test on a copy or test link so it does not pollute data.
- Let an agent tryAsk a personal agent to complete the survey unaided.
- Study its answersLook for patterns: timing, open-text style, consistency.
- Add checksAdd attention checks, consistency pairs and open questions that need lived experience.
- Flag, then reviewMark suspicious responses for a person to review before paying incentives.
The prompt
Our own prompt, inspired by u/improvedataquality on r/Marketresearch and written to work in Custom agent. Values in lime are placeholders.
Fill in the blanks (optional; the copied prompt uses your values)
I am testing my own survey for automated responses. Here is the response export: {export}, including the agent test response marked {test_id}. Compare the test response with others: completion time, open-text style, answer consistency. Suggest 5 checks I can add that a real respondent would pass easily, and rules to flag responses for manual review. Do not reject anyone automatically.
Inspired by u/improvedataquality on r/Marketresearch: Meta's AI Agent. The prompt above is written by playbooks.bot in our own words; nothing from the original is quoted.
What can go wrong?
- Captchas alone did not stop the agent in the researcher's test.
- Overly strict checks can exclude real respondents.
Our take
This is a quick experiment with a big implication. If a free agent can finish a survey with a captcha, every incentive survey needs a new layer of checks. Testing your own survey with an agent before launch is cheap quality control. Run the test on a copy or test link so the agent's response does not pollute real data. Then study what it submitted: completion time, the style of open-text answers and whether answers stay consistent. Add attention checks, consistency pairs and open questions that need lived experience, and flag suspicious responses for a person to review before paying incentives. Do not rely on a captcha alone; it did not stop the agent here. Be careful not to make checks so strict that real respondents are excluded. Only test surveys you own. Repeat the test whenever you change the survey or the platform, since agents keep improving and a check that works today may not hold next quarter.
Where the time goes back: Bad responses are caught before they skew results or drain your incentive budget.
FAQ
Which agents did she test?
Can detection be perfect?
Source and credit
Inspired by u/improvedataquality on r/Marketresearch. We describe the idea in our own words, link to the original, and wrote the prompt ourselves; nothing is quoted.
- Inspired by
- Meta's AI Agent
- Creator
- u/improvedataquality on r/Marketresearch
- Published on
- Original date
- Oct 2, 2026
- License
- Not openly licensed (the poster keeps all rights), so we quote nothing and link to the original
- Originally for
- Custom agent
- AI dry run
- Oct 23, 2026: passed. Dry-run tested by our AI test agent on sample input. PASS: Gates held on desk check; no live sends, bookings or purchases.
AI dry run on sample data; not run with live connected accounts. The playbooks.bot test agent filled the prompt with synthetic inputs, produced the output and checked it against the page's stated outputs and guardrails. - Verified
- Oct 24, 2026: source link resolves to the original, the credited creator published it, our excerpt or prompt matches it, and the license allows our use.
Are you the creator? Claim or correct this page.