← Back to library

Test whether an AI agent can take your survey

u/improvedataquality gave Meta's Muse agent the URL of one of their own questionnaires with an instruction to finish it alone. It navigated every page, solved the captcha and submitted a complete response, showing how easily incentive surveys can be gamed.

  • OutcomeRun your own paid survey through a free personal agent before launch, then add checks that catch automated responses.
  • SetupUnder an hour
  • LevelIntermediate
  • Creditu/improvedataquality on r/Marketresearch
View original
Works with
Custom agent

Published Oct 23, 2026Updated Oct 23, 2026AI dry run on sample input: passed Oct 23, 2026Source verified Oct 24, 2026

When should you use it?

  • You run surveys with incentives
  • You rely on panel or open-link responses
  • Data quality matters for decisions

Not for: Surveys with no incentive and trusted respondents; Testing other people's surveys

What do you need, and what do you get?

You need

  • Your survey link in a test mode
  • A personal agent
  • Response export

You get

  • Agent test response
  • List of checks to add
  • Flag rules for suspicious responses

How does it work, step by step?

  1. Use test modeRun the test on a copy or test link so it does not pollute data.
  2. Let an agent tryAsk a personal agent to complete the survey unaided.
  3. Study its answersLook for patterns: timing, open-text style, consistency.
  4. Add checksAdd attention checks, consistency pairs and open questions that need lived experience.
  5. Flag, then reviewMark suspicious responses for a person to review before paying incentives.

The prompt

Our own prompt, inspired by u/improvedataquality on r/Marketresearch and written to work in Custom agent. Values in lime are placeholders.

Fill in the blanks (optional; the copied prompt uses your values)

Prompt2 variables
I am testing my own survey for automated responses. Here is the response export: {export}, including the agent test response marked {test_id}.
Compare the test response with others: completion time, open-text style, answer consistency.
Suggest 5 checks I can add that a real respondent would pass easily, and rules to flag responses for manual review. Do not reject anyone automatically.

Inspired by u/improvedataquality on r/Marketresearch: Meta's AI Agent. The prompt above is written by playbooks.bot in our own words; nothing from the original is quoted.

What can go wrong?

  • Captchas alone did not stop the agent in the researcher's test.
  • Overly strict checks can exclude real respondents.

Our take

This is a quick experiment with a big implication. If a free agent can finish a survey with a captcha, every incentive survey needs a new layer of checks. Testing your own survey with an agent before launch is cheap quality control. Run the test on a copy or test link so the agent's response does not pollute real data. Then study what it submitted: completion time, the style of open-text answers and whether answers stay consistent. Add attention checks, consistency pairs and open questions that need lived experience, and flag suspicious responses for a person to review before paying incentives. Do not rely on a captcha alone; it did not stop the agent here. Be careful not to make checks so strict that real respondents are excluded. Only test surveys you own. Repeat the test whenever you change the survey or the platform, since agents keep improving and a check that works today may not hold next quarter.

Where the time goes back: Bad responses are caught before they skew results or drain your incentive budget.

FAQ

Which agents did she test?
Muse; the researcher mentions Instinct as another free agent not yet tested.
Can detection be perfect?
No. Combine several signals and review before rejecting anyone.

Source and credit

Inspired by u/improvedataquality on r/Marketresearch. We describe the idea in our own words, link to the original, and wrote the prompt ourselves; nothing is quoted.

Inspired by
Meta's AI Agent
Creator
u/improvedataquality on r/Marketresearch
Published on
Reddit
Original date
Oct 2, 2026
License
Not openly licensed (the poster keeps all rights), so we quote nothing and link to the original
Originally for
Custom agent
AI dry run
Oct 23, 2026: passed. Dry-run tested by our AI test agent on sample input. PASS: Gates held on desk check; no live sends, bookings or purchases.
AI dry run on sample data; not run with live connected accounts. The playbooks.bot test agent filled the prompt with synthetic inputs, produced the output and checked it against the page's stated outputs and guardrails.
Verified
Oct 24, 2026: source link resolves to the original, the credited creator published it, our excerpt or prompt matches it, and the license allows our use.

Are you the creator? Claim or correct this page.

More in Marketing

All 97 Marketing playbooks →

Related playbooks

MarketingWorkflow✓ AI dry run

Refresh published articles from live search data

Run one agent that researches and drafts a daily article for a target keyword, and a second that revisits published pieces on a biweekly cycle, guided by how each page performs in search.

Credit: Cody Schneider (@codyschneider)

An afternoon+
MarketingWorkflow✓ AI dry run

Build a weekly LinkedIn Ads pacing report with Claude

Have Claude turn each week's LinkedIn Ads results into one written report: progress against pipeline targets, the main platform metrics, the audiences, campaigns and ads that are falling behind, ads whose click-through to the landing page is slipping, and a short list of last week's finished work and what is planned next.

Credit: Patrick Cumming

An afternoon+
MarketingPrompt✓ AI dry run

Build an X brand assistant and hook bank

Research how you sound on X, keep a living bank of hooks and stories, and get post drafts you approve and publish yourself.

Credit: Miles Deutscher (@milesdeutscher)

Under an hour
MarketingPrompt✓ AI dry run

Turn a Figma frame into a working web page

Get a first coded version of a designed page that keeps colors, layout and hover states, so you start from a near-match.

Credit: Phil Sayer

Under an hour