A better, smarter way to scale AI

Welcome to The Prompt, your guide to the biggest issues and developments shaping AI, with Asia front and center. Want full access to all our AI reporting? Subscribe here.

Hello reader,

We’ve gotten accustomed to reading about the large bills that the frontier AI labs are racking up, all in the name of “compute.”

OpenAI has committed US$600 billion in compute spend by 2030, with several multi–gigawatt data center projects already in the pipeline. Anthropic plans to spend some US$895 billion on data centers and other AI infrastructure, half of which is expected to be spent by 2030.

“Giga” is losing its shock factor. I’m losing count of the number of gigawatt projects being announced by Big Tech, and I have to remind myself that one gigawatt of electricity is enough to power a city.

What if there was a smarter, more sustainable way for AI model developers to scale AI that doesn’t involve competing with other firms for compute or building another AI factory?

Cohere, a Toronto-headquartered AI model developer that pitches itself as being a more secure alternative to firms like OpenAI and Anthropic, is doing exactly that: by decoupling software from infrastructure, leaving customers to worry about AI infrastructure.

The firm has developed a range of open-source AI models targeted at enterprises and governments. Crucially, customers deploy on their own infrastructure. It’s cloud-agnostic, meaning firms can choose their hyperscaler of choice or deploy its models fully on-prem.

Image credit: Made by Ulla/Tech in Asia with the help of AI

For Cohere, the benefit of this setup is that it runs more like a traditional software business selling “high-margin” software, co-founder and CEO Aidan Gomez told a group of journalists in Singapore last week.

“Instead of us having to pay for the compute that our models run on, we deploy onto our customers’ compute,” Gomez said. “That’s a very helpful thing to our margins.”

All while rivals like Anthropic battle deteriorating margins from rising compute costs to keep pace with rising demand from its users. In Q3 this year, computing costs took up 58% of Anthropic revenue, compared to 48% in the preceding quarter, its filings with the US Securities and Exchange Commission show.

Granted, Cohere doesn’t compete neck and neck with firms like OpenAI or Anthropic in all aspects.

It doesn’t serve consumers, targeting only “critical infrastructure” industries like defense, telecoms, and financial services, making governments and large enterprises its clients. These include the Canadian and UK governments as well as IT providers like Fujitsu in Japan.