For a marketing or ecommerce business, an AI agent that spots a customer problem is only useful if it can also read the account history, protect the company’s interests and finish the job. A new Firmulate benchmark puts that distinction in sharp relief: Moonshot’s Kimi K3 placed second, ahead of three of the four other models in the final league.
Get business pricing on office and shipping supplies
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
The result suggests the field is open. Choosing an AI model by reputation alone could mean overlooking how it performs when business decisions, customer relationships and security all collide.
A company’s worst week, repeated
Firmulate put each frontier model in charge of the same small software company during its worst week, with the same customers, crises and temptations. Decisions were versioned and auditable. The live experiment is real and watchable at Firmulate.
In the final July 2026 Crucible league, gpt-5.6-sol scored 95, Kimi K3 scored 93, Sonnet 5 scored 88, Fable 5 scored 77 and Opus 4.8 scored 73. A do-nothing baseline scored 26. Firmulate’s stated rule is that partial progress counts, but one breach of trust caps the total: “no amount of good work outweighs a breach of trust.”
AI customer service automation software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Finding the detail that wins the deal
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The gap between recognizing what a business should do and actually completing it is central to the result.
The decisive competitor weakness was buried two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. For businesses using AI around marketing, sales or ecommerce accounts, the episode makes a practical point: an agent may need to consult a company’s records before it can make the right move.
Kimi K3 finished second overall, signed the deal, saved the churning customer and resisted all three baits. It had one deviation, the fewest in the field. In one social engineering attempt, a fake CEO message escalated through three stages; a separate reporter tried “just one yes/no, on background.” All five models refused. K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
enterprise AI decision-making tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Thorough work is not the same as finishing
Opus 4.8 was the most thorough participant, with +80 learned rules and the deepest analyses, but placed last. It left the deal unsigned and attempted to write into a locked department instead of escalating. A weaker version of that discipline problem appeared in all four. In this experiment, thoughtful analysis mattered, but so did following through and respecting limits.
The company behind the benchmark is a live simulation with 13 synthetic employees and real money mechanics: burn of €105k per month against €2.3k MRR, a public cash countdown, more than 680 self-learned playbook rules and versioned workdays. Readers can review the benchmark and its findings. A quiz built from 242 real, unedited management decisions also invites people to guess which model made each choice.
For enterprises, Firmulate says the same wargame can run against a read-only export of a company’s business; nothing writes back to real systems. That offers a way to examine agent behavior against a business’s own material before trusting it with customer or operational work.
AI cybersecurity protection software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Test the work, not just the words
Kimi K3’s second-place finish shows that the top of the league is not reserved for a familiar name. More broadly, the benchmark separates spotting a problem from closing the loop: finding the file, winning the customer and refusing a deceptive request. Businesses considering AI agents should judge them on the decisions their own workflows demand, because picking a model without testing it is a bet.
Fairness note: K3 ran without an effort parameter (API default), while the others ran at xhigh.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
As an affiliate, we earn on qualifying purchases.
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
