Ziroh Labs, an AI startup in India, has introduced Kompact AI, a system that runs large AI models without advanced GPUs.

Developed with Indian Institute of Technology (IIT) Madras, Kompact AI works on standard CPUs found in regular devices, reducing the need for expensive and scarce GPUs. It has been tested by Intel and AMD.

The platform focuses on the inference stage of AI and has successfully run models like Meta’s Llama 2 and Alibaba’s Qwen 2.5 on a laptop with an Intel Xeon processor.

This helps Indian developers who struggle to access high-end GPUs.

IIT Madras Director V. Kamakoti said, “We’re demonstrating that you don’t need a revolver to kill a mosquito.”

🔗 Source: Bloomberg

🧠 Food for thought

1️⃣ The economic implications of CPU-based AI in developing markets

Ziroh Labs’ innovation addresses a critical bottleneck in AI democratization: the cost and availability of specialized hardware.

GPUs have traditionally been considered essential for AI workloads because they contain thousands of cores optimized for parallel processing, making them significantly more efficient than CPUs for complex machine learning tasks 1.

This hardware requirement has created an “AI divide” where only organizations with access to expensive GPU infrastructure can develop advanced AI applications—a particularly acute challenge in India where high-performance computing infrastructure remains limited 2.

The economics of this divide are stark: India spends less than 0.7% of its GDP on R&D compared to much higher percentages in AI-leading nations, restricting its ability to compete in hardware-intensive AI development 2.

By enabling AI inference on widely available CPUs, Ziroh’s approach could significantly reduce the entry barrier for AI deployment across India’s rapidly growing tech ecosystem, where over 90% of businesses are already integrating AI technologies despite hardware constraints 3.

2️⃣ Strategic alignment with India’s push for technological self-reliance

Kompact AI represents a practical implementation of India’s broader three-pillar strategy for AI autonomy, which emphasizes developing homegrown solutions linked to national development goals 4.

The Indian government has committed substantial resources to this vision, allocating ₹10,300 crore (approximately $1.4 billion) to the IndiaAI Mission specifically to build domestic AI infrastructure and capabilities 5.