DeepSeek signals bigger, cheaper AI models with new paper

DeepSeek, a Hangzhou-based AI startup, has published a new technical paper proposing changes to deep learning architectures to improve training efficiency for large models.

The paper, co-authored by founder and CEO Liang Wenfeng and 18 other researchers, introduces Manifold-Constrained Hyper-Connections (mHC).

The team tested mHC on models with 3 billion, 9 billion, and 27 billion parameters, reporting that it scaled without major increases in computational costs.

Released on arXiv, the paper highlights ongoing research activity at DeepSeek, which has gained attention in China’s AI sector.

MHC builds on the hyper-connections concept introduced by ByteDance researchers in 2024 to address rising memory costs in large-scale training.

Industry observers see these publications as signals of future model development, with anticipation that DeepSeek could launch a new AI model ahead of the Spring Festival in February.

🔗 Source: South China Morning Post

🧠 Food for thought

Implications, context, and why it matters.

Manifold-Constrained Hyper-Connections (mHC) real-world use depends on reproducibility

The article does not say if DeepSeek released mHC training code, hardware settings, or if outside teams verified the memory savings. These gaps decide whether others can use it beyond DeepSeek.
DeepSeekMath released base, instruct, and reinforcement learning (RL) checkpoints under an MIT License with model use under a Model License 1. DeepSeek-V3 offers inference across frameworks 2. The deepseek-ai GitHub organization, a code hosting hub, hosts active repos 3.
Without reproducibility data, readers cannot tell if mHC only runs on DeepSeek’s internal setup or if it works on standard cloud GPU configurations (the typical setups from public clouds). That affects adoption and any claim of lower training costs.

Cloud GPU providers can gain if mHC scales for memory-optimized training

If mHC reduces memory use during training, cloud GPU providers with lower spec cards like NVIDIA A100 40 GB could sell tailored bundles. A100 40 GB rentals range from $0.66 per hour on budget hosts to $4+ per hour on major clouds 4.
Verda lists A100 SXM instances (SXM is NVIDIA’s server module form factor) from $0.22 per hour 5. Civo prices NVIDIA GPUs at $0.69 per hour 6. Memory efficient setups could make 40 to 48 GB GPUs workable for larger experiments while avoiding 80 GB cards.
Machine Learning Operations (MLOps) vendors can add tooling that applies mHC during training jobs, then sell cost focused workflows. DeepSeek-V3 consumed 2.788 million NVIDIA H800 data center GPU hours 2, which strains startup and lab budgets.