Back
LLMsNEW

Why Harvey Ditched OpenAI and Anthropic for an Open Model

Legal AI startup Harvey watched its gross margins collapse from 50% to negative 50% as enterprise LLM pricing rose 20x on rising token usage. It fixed that by training its own model on the open-weight Kimi K3, and it is not the only startup doing this.

4 min read
Updated Sep 22, 2026
QUICK ANSWER

Harvey, the legal AI startup valued at $15.6 billion, saw its gross margins collapse from roughly 50% at the start of the year to negative 50% by June 2026, as Bloomberg reported.

Key Takeaways
  • This guide provides comprehensive, actionable information
  • Consider your specific workflow needs when evaluating options
  • Explore our curated LLMs tools for specific recommendations

What actually happened

Harvey, the legal AI startup valued at $15.6 billion, saw its gross margins collapse from roughly 50% at the start of the year to negative 50% by June 2026, as Bloomberg reported. The cause was straightforward: token usage under OpenAI and Anthropic's usage-based enterprise pricing rose twentyfold as customers used the product more, and the bill scaled with it. Running at negative margins on your core product is not a sustainable position for a company of that size, whatever the growth numbers look like.

Harvey's fix was to stop renting frontier intelligence and build its own. In August 2026 it launched Tenet, an in-house model post-trained on top of Moonshot AI's open-weight Kimi K3. Harvey says Tenet runs at less than a quarter of the cost of leading foundation models while holding strong performance on its legal benchmarks, and margins turned positive again after the switch.

50% → -50%
Harvey's gross margin swing, start of year to June 2026
20x
rise in token usage under OpenAI and Anthropic's enterprise pricing
<25%
of leading foundation-model cost, Harvey's reported cost for Tenet

Harvey is not the only one

Bloomberg's reporting names other startups making the same move: Abridge, Decagon and Ramp are each building on cheaper open-weight models rather than staying fully dependent on closed frontier APIs, backed by investors including Sequoia Capital and General Catalyst. The pattern is the same in each case: a company with real usage volume finds that per-token enterprise pricing scales worse than its own revenue, and that training or post-training an open-weight base now costs less than continuing to rent.

Why this is economically viable now, and was not before

This move only works if the open-weight base is actually good enough, and that gap has been closing fast. By the same reporting, Moonshot AI's open-weight Kimi K3 has narrowed the performance gap to Anthropic's closed Fable 5 to about four months, at roughly a fifth of the cost. A year ago, the capability gap between an open-weight base and a frontier closed model was wide enough that most companies had no real alternative to paying whatever the API charged. That is no longer true for workloads, like Harvey's legal analysis, that are well-scoped enough to post-train a smaller open base against.

The broader effect Bloomberg points to is pressure on OpenAI and Anthropic's own margins, and a shift in where AI's actual profit lands, potentially toward chips and infrastructure rather than the model layer itself, as more of the companies that used to be their biggest customers become their competitors on the model layer.

Who should actually care

Your product's COGS scale with LLM token usage
Run the same math Harvey did: plot your margin against usage growth, not against a fixed per-seat price. If the curve points negative, post-training an open-weight base is now a real option, not a research project.
You are deciding whether to build or keep renting
The gap that used to make renting the obvious choice, capability, has narrowed to months on well-scoped tasks. The remaining question is whether your workload is narrow enough to post-train for, the way Harvey's legal analysis was.
You're evaluating AI vendors as an investor or operator
Watch gross margin trends at usage-heavy AI startups specifically, not just growth. Harvey's swing from 50% to -50% happened inside two quarters; it is a fast-moving number, not a slow one.

For what open-weight models still cannot do, and where this strategy runs into a wall, see open models: what they cannot do. For the build-vs-rent decision itself, see AI infrastructure: run, host, or rent. For the model at the center of this, see Kimi K3 and every LLM in the directory.

FREQUENTLY ASKED QUESTIONS
Why are AI startups like Harvey moving from OpenAI and Anthropic to open-weight models?
Harvey, the legal AI startup valued at $15.6 billion, saw its gross margins collapse from roughly 50% at the start of the year to negative 50% by June 2026, as Bloomberg reported.
EXPLORE TOOLS

Ready to try AI tools? Explore our curated directory:

SHARE THIS GUIDE

Harvey, the legal AI startup valued at $15.6 billion, saw its gross margins collapse from roughly 50% at the start of the year to negative 50% by June 2026, as Bloomberg reported.

Share on X LinkedIn Reddit Email
Copied to clipboard