Honey, I shrunk the LLM: why leaner is meaner

Large language models (LLMs) – AI systems like ChatGPT that process text data to comprehend and generate human language – have been touted as a game changer.

Despite the potential of LLMs to make workers more productive – or even replace them in certain jobs – industry watchers believe that these systems have to be leaner and less expensive before they are more widely adopted.

Sri Ambati, founder and chief executive of AI startup H2O.ai, notes that LLMs have demonstrated that computers can answer questions about a wide range of topics.

But for simpler use cases, applying LLMs can seem like overkill. Moreover, the cost of AI applications powered by LLMs can be prohibitive, Ambati adds.

A single LLM-powered AI use case costs anywhere from US$500,000 to US$1 million, he estimates. If a company has 1,000 use cases spread across different functions, it could spend up to a billion dollars just to apply AI.

See also: Patsnap invested ‘7 digits’ to develop an LLM. Will it pay off?

So it makes sense to create specialized models for companies to use, instead of creating larger LLMs that can be more expensive to operate, he explains.

Enter small language models (SLMs), which are essentially streamlined versions of LLMs.

H2O.ai’s own SLM, dubbed Danube, has been trained with 6 trillion tokens (words, sub-words, or characters).

Versions of the model were then created with 4 billion, 2 billion, and 500 million parameters, which is significantly fewer than those of larger and more complex models.

For instance, OpenAI’s GPT-4 is estimated to have about 1.8 trillion parameters.

Good things in small packages

Security concerns