Numbers·Failed, with a lesson·Devin·Software companies
Month of Devin tasks
Devin was given more than 20 real coding, research, and code-modification tasks over a month by an Answer.AI team to test its autonomy.
Out of 20 tasks, Devin had 14 failures, 3 successes, and 3 inconclusive results, with no pattern predicting which would work.
Overview
How it was set up
Devin, driven through Slack and its web interface, running in its own Docker environment with browser, editor, and shell.
Worth knowing
Failed tasks often took days. Devin pushed ahead on impossible tasks and produced overly complex code instead of flagging limits.
A prompt to start from
The source does not show the prompt.
Sources 1
History
- Added to Agentic Feats
- 2026-10-13
- Pick
- No
- Markdown copy
- /c/month-of-devin-tasks.md
More in software companies
15 featsResultSmall scoped fixes took about five minutes to test and merge versus roughly an hour by hand; the full-stack task failed and was rewritten.
ResultDevin found nearly every old term in a docs-wide rename, but the edits still needed 20+ manual commits.
ResultThe author shipped 120 models, an MCP server for Claude Desktop, and 140 chaos tests with a 100% pass rate over 22 months.