212 customers, refunded twice in 38 minutes
A business ran a customer support agent with a refund tool. Before launch, the agent passed an offline eval at 96%. The eval used 400 real, anonymised conversations, each run three times. One prompt line said: if a tool call fails, retry once. In production the payments system was slow. Its typical slow response was 4.8 seconds, and the tool timed out at 3 seconds. So calls marked failed had in fact gone through, and the retry paid a second refund.
Start with one job only. Fill in the three lines. Keep the bot on that job.
The one job this bot does: ______________________________
What it may do on its own: ______________________________
What it must ask me first: ______________________________
Two lines from this case, side by side. No new figures.
A business ran a customer support agent with a refund tool. Before launch, the agent passed an offline eval at 96%. The eval used 400 real, anonymised conversations, each run three times. One prompt line said: if a tool call fails, retry once. In production the payments system was slow. Its typical slow response was 4.8 seconds, and the tool timed out at 3 seconds. So calls marked failed had in fact gone through, and the retry paid a second refund.
In 38 minutes at full traffic, 212 customers got two refunds each. Error rates looked normal. A customer ticket and a finance check found the problem, not the monitoring. The business is unnamed in the post.
The after line is the source's claim, not our measurement.
smalloffice.space · Get a case like this in your inbox each morning. Subscribe free
Read this case on your phone now, and keep it for later. Type your email. We send this case to that address, and you get the next cases each morning too.