AI changed the cloud-first gospel, sparking on-premise resurgence

For almost a decade, the tech industry has been preaching a gospel of “cloud-first.”

Pushed by industry giants like AWS and Google Cloud, and supported by industry reports, cloud has been heralded as faster and cheaper for ambitious companies chasing digital transformation.

Image credit: Timmy Loen

But following the rise of AI, this gospel is being reexamined. Some enterprises are now bringing their applications and data back in-house, saying that on-premise systems, (which includes mainframes) are critical in today’s market.

Some of the reasons they give for the switch: regulatory pressures, latency, and security.

Protecting data in critical systems like transport, emergency services, and healthcare has long been a concern, but with today’s “data is the new gold” mindset, security has become a bigger talking point for more players.

A recent report by Tenable shows that 70% of AI-related cloud services globally had at least one critical vulnerability, higher than non-AI cloud services. It also found that, in certain cases, restricted or confidential information had been exposed in Southeast Asia due to misconfigurations and weak access controls.

But perhaps the biggest motivator in favor of this shift is the bottom line. The cost of the top performing GPUs have risen 800% since 2015, and trade bans on exports have raised concerns about the limited availability of cloud-based GPUs in Asia.

This return to on-premise systems opens a door for companies like IBM and Weka, a software storage platform built to handle data-intensive workloads such as AI training.

IBM – born the same year that the assembly line was invented — is positioning itself to be the backbone for a new era of on-premise AI.

It aims to do this with the Z17, a mainframe computer which it rolled out in Southeast Asia earlier this year. It can handle 450 billion AI inference operations per day. Inferences are when an AI uses data to make predictions, decisions, or answers

There is no listed price of the Z17, but estimates based on a previous model of the system put the cost between US$250,000 to US$1 million.

When they get the first bill, and it’s US$10,000 – it’s a shock. They think, hey, I can’t afford that.

For Arun Menon, principal analyst at MTN Consulting, IBM’s intensified push into Asia with the Z17 platform, signals a strategic play that could have meaningful implications for the region’s AI data centers.

The cost of a reality check

Dark clouds for hyperscalers?

Can the cloud go poof?