When should you use it?
- You have a backlog of small, well-defined bugs
- Each touches only a few files
- You test pull requests before merging
Not for: Cross-cutting features; Codebases with no build or lint steps the agent can run
What do you need, and what do you get?
You need
- A connected repository
- Short, specific task descriptions
- Time to test each pull request
You get
- Pull requests for narrow fixes
- Rough cost per task
- A sense of where the agent fails
How does it work, step by step?
- Connect and confirm setupLet the agent scan the repo and check its inferred build and lint steps.
- Write a tight taskDescribe one change in one component with the expected result.
- Let it runStart it in the morning and check back later, as with a junior engineer.
- Test the pull requestSpend a few minutes verifying behavior before merging.
- Stop on big tasksWhen scope grows, the cost rises and success drops.
The prompt
Our own prompt, inspired by Vikram Sreekanti and written to work in Custom agent. Values in lime are placeholders.
Task for [repo]: in [component], [specific change]. Expected result: [behavior]. Only touch files under [path]. Lint, run tests, repair failures, then submit a PR that explains the change and your testing. If the change needs edits outside that path, stop and ask.
Inspired by Vikram Sreekanti: One month of using Devin. The prompt above is written by playbooks.bot in our own words; nothing from the original is quoted.
What can go wrong?
- Larger tasks consumed far more credits with poor results in their small sample.
- A small startup may run out of suitable tasks quickly.
Our take
Vikram Sreekanti of RunLLM spent a month with Devin and gives a clear rule that applies to every autonomous coding agent we have seen: one component, few files, clear expected outcome. Tight fixes such as chart styling or an API corner case needed a minute of description and a short test, at a few dollars each. Bigger jobs cost several times more, and most of those yielded nothing worth keeping. Keep a simple log of task size, cost and outcome for your first twenty tasks and you will know quickly where the line sits for your codebase. Larger teams with steady bug inflow will get more from this than small startups inventing features. Always test the pull request yourself before merging. Stop escalating scope mid-run; open a new task instead. Prefer tickets that already have a failing test or a screenshot of the bug. Track revert rate separately from merge rate so cheap merges that break production do not look like wins.
Where the time goes back: An hour of investigation and fixing becomes a few minutes of describing and testing.
FAQ
How long did small tasks take?
Did they keep using it?
Source and credit
Inspired by Vikram Sreekanti. We describe the idea in our own words, link to the original, and wrote the prompt ourselves; nothing is quoted.
- Inspired by
- One month of using Devin
- Creator
- Vikram Sreekanti
- Published on
- frontierai.substack.com (Substack)
- Original date
- Jan 23, 2025
- License
- Not openly licensed (the poster keeps all rights), so we quote nothing and link to the original
- Originally for
- Custom agent
- AI dry run
- Oct 23, 2026: passed. Dry-run tested by our AI test agent on sample input. PASS: Gates held on desk check; no live sends or purchases.
AI dry run on sample data; not run with live connected accounts. The playbooks.bot test agent filled the prompt with synthetic inputs, produced the output and checked it against the page's stated outputs and guardrails. - Verified
- Oct 23, 2026: source link resolves to the original, the credited creator published it, our excerpt or prompt matches it, and the license allows our use.
Are you the creator? Claim or correct this page.