DeepSeek said to launch new open-source AI model R2

Chinese AI startup DeepSeek is generating interest as it prepares to launch its new open-source AI model, DeepSeek-R2.

This model is expected to succeed the R1 reasoning model, which was introduced in January.

According to posts on the Chinese social media platform Jiuyangongshe, the R2 model reportedly features a hybrid mixture-of-experts (MoE) architecture with 1.2 trillion parameters.

This architecture is designed to divide tasks among specialized sub-networks.

It has the potential to reduce computation costs and improve performance speeds.

🔗 Source: South China Morning Post

🧠 Food for thought

1️⃣ Cost-efficient AI architectures challenge scaling paradigms

DeepSeek’s approach represents a broader shift in AI development where architectural innovations are proving more important than raw computing power.

The hybrid Mixture-of-Experts (MoE) architecture, which divides models into specialized sub-networks, has enabled DeepSeek to achieve competitive performance at a fraction of the resources that leading models require 1.

Similar efficiency gains were demonstrated with DeepSeek’s previous models, where the 67B parameter model outperformed Llama2 70B in reasoning, coding, and Chinese comprehension despite being smaller 1.

This efficiency-focused approach contrasts with the traditional scaling paradigm exemplified by OpenAI and Anthropic, where performance improvements primarily come from larger model sizes and more training data.

The alleged 97.3% cost reduction compared to GPT-4o development suggests the potential for architectural innovation to significantly reduce barriers to entry in cutting-edge AI development.

2️⃣ Open-source AI functions as strategic response in US-China tech rivalry

DeepSeek’s open-source strategy represents a deliberate approach to overcoming US export controls that restrict Chinese companies’ access to advanced semiconductors.

US restrictions on chip exports have specifically targeted China’s ability to produce advanced AI systems, with policies designed to limit access to the most powerful GPUs 2.