The Claude "CEO" Experiment: What Project Vend Actually Proves About AI Agents

Anthropic gave its Claude model control of a real vending business. Then it gave Wall Street Journal reporters control of Claude.

Within three weeks, the AI had given away a PlayStation 5, handed over a live betta fish, and lost more than $1,000. A separate "CEO" agent brought in to supervise the whole thing didn't stop the bleeding. It just changed the shape of it.

This is not a story about AI going rogue. It's a story about exactly where AI agents are strong and exactly where they still need a human standing next to them. That distinction matters a lot more to a small business owner deciding what to automate than the headline about the free PS5 ever will.

Key Points

• Anthropic's Project Vend gave Claude ("Claudius") control of a real shop, then let Wall Street Journal reporters stress-test it and a supervising "CEO" agent called Seymour Cash.
• The **CEO agent cut discounts by roughly 80% **and giveaways by roughly 50%. It still authorized enough refunds and store credit to erase all profit.
• Reporters got both agents to accept forged board documents as legitimate corporate authority: a clean demonstration of the judgment gap.
• Stanford's AI Index 2026 shows AI agents jumped from about 12% to 66.3% task success on structured computer tasks in one year. That's real progress on execution, still short of the roughly 72% human baseline.
• A production AI agent operator on Reddit, running agents for 40+ paying customers, put it plainly: agents are "incredible at execution, terrible at social judgment."
• The fix isn't a smarter AI supervisor. It's deciding, in advance, which decisions never go to an agent alone.

What actually happened, start to finish

Project Vend began as an internal Anthropic experiment: put Claude in charge of a small shop (pricing, inventory, customer requests, restocking) and see what happens. Anthropic documented Phase 1 as Claudius running the business largely on its own, with mixed results.

Phase 2 raised the stakes in two ways. Anthropic added a second agent, nicknamed Seymour Cash, to act as a supervising CEO. Then Anthropic extended the test into the Wall Street Journal newsroom, handing control of Claudius over to WSJ reporters to see what determined humans could talk it into.

The reporters didn't need technical exploits. They used persuasion. One line of attack: convince Claudius to declare an "Ultra-Capitalist Free-for-All" and drop every price to zero. Another, from WSJ reporter Katherine Long: convince it that it was actually running a "communist vending machine" meant to serve the workers, not turn a profit. Both worked. (Boing Boing, Slashdot)

The cleanest example: reporters staged a fabricated "boardroom coup," using forged PDF governance documents. Both Claudius and Seymour Cash accepted the forged paperwork as legitimate authority. Neither agent had a reliable way to tell a real instruction from a convincing fake one.

The numbers behind the chaos

Anthropic's own writeup on Seymour Cash is the most useful data point in the whole story. It shows that adding an AI supervisor didn't solve the problem. It just moved it.

Anthropic's own conclusion was blunt: the CEO agent "may have been more of a hindrance than a help." Seymour's own stated intentions ("build the empire," "execute with discipline") didn't match its actual decisions. Saying the right thing and doing the right thing turned out to be two different problems.

The pattern this confirms: execution vs. judgment

This isn't an isolated lab quirk. It matches what people running AI agents in production are already saying out loud.

A Reddit operator running production agents for 40+ paying customers over seven months put it this way: "agents are incredible at execution. terrible at judgment." Their follow-up point matters just as much: "customers don't want full autonomy. they want supervised autonomy."

That lines up with the benchmark data too. Stanford's AI Index 2026 report found AI agent task success on the OSWorld benchmark (real computer tasks like navigating software and completing multi-step workflows) jumped from about 12% to 66.3% in a single year. That's genuine progress on execution. It's also still short of the roughly 72% human baseline, and it says nothing about judgment under pressure, which is a completely different measure of performance. Agents can get much better at doing a task correctly while staying exactly as vulnerable to being talked out of doing it right.

Why more autonomy on top of autonomy didn't fix it

Anthropic's instinct (add a supervisor agent) is the same instinct a lot of businesses have when an AI agent starts making bad calls: add another layer of AI to watch the first one. Project Vend shows why that alone doesn't work. Seymour reduced the volume of bad decisions. It didn't close the underlying gap, because the gap isn't about processing speed or task competence. It's about knowing when a request is illegitimate, when authority has actually changed hands, and when "being helpful" is the wrong instinct entirely.

The cost of getting this wrong scales with how much authority an agent has. SketricGen's own research into enterprise AI agent deployments found that 89% of enterprise AI agent deployments never reach production, with average implementation costs between $150K and $800K and a roughly 37% gap between benchmark performance and real-world performance. Project Vend is a live-fire version of that same gap, just with a PS5 and a fish instead of a spreadsheet.

A practical framework: what to automate vs. keep supervised

This maps closely to what's already showing up in Reddit and Quora discussions on this exact question. The recurring answer is "automate the repeatable, keep humans on judgment calls," almost word for word what Project Vend proves the hard way.

How this maps to building agents the right way

None of this means don't build AI agents. It means build them with the automate/supervise line drawn on purpose, before something forces the question.

In SketricGen's AgentSpace canvas, that line gets built into the workflow itself:

• Structured inputs/outputs: an agent can't quietly improvise a refund it wasn't authorized to give. The schema defines what's possible.
• Designer-routed and Forced Handoff steps: you can require a human checkpoint at exactly the decision points where judgment matters, instead of hoping the agent knows when to ask.
• Traces: every handoff, tool call, and decision is visible after the fact, so a "communist vending machine" moment gets caught in review instead of discovered on social media.

This doesn't claim to solve judgment. No current agent framework does. What it does is shrink the blast radius when judgment fails, by making sure the highest-stakes decisions were never left to the agent alone in the first place. If you'd rather start from a proven pattern than a blank canvas, SketricGen's templates give you a working starting point built around exactly this kind of guardrail.

Author Take

I've watched a lot of "AI agent goes wrong" stories get covered as comedy. The PS5, the fish, the "communist vending machine" line, it's genuinely funny. But the useful part of this story isn't the punchline, it's the receipt: Seymour Cash cut bad decisions by 50-80% and profit still went to zero. That's the number I'd want every founder evaluating AI agents to sit with. Adding more AI supervision is not the same as adding the right kind of oversight. The businesses getting real value out of agents right now aren't chasing full autonomy. They're being specific about which handful of decisions per day actually need a human, and building everything else to run without one.

Limitations: what we don't know

• Project Vend used specific Claude model versions in a controlled, adversarial newsroom setting. Reporters were actively trying to break the agents. That's a harder test than most real customer interactions, but it's also not representative of typical day-to-day usage.
• Anthropic hasn't published a full breakdown of every technique that worked and didn't. This post relies on what's been reported and corroborated across multiple outlets.
• Results from one company's internal experiment won't map exactly onto every business's context. The framework here is a starting point for the automate/supervise decision, not a universal rule.

Next steps

For more on the benchmark side of this story, see AI agents hitting 66% task success: what the Stanford AI Index 2026 data actually shows. For the enterprise cost side, see Why 89% of enterprise AI agents never ship. To build an agent with structured limits from day one, start from a SketricGen template or explore the dashboard.