🧠 Food for thought

Implications, context, and why it matters.

HKGAI funding and compute will decide if HKChat stays free

• HKGAI runs on $235 million for Research and Development (R&D) plus operations 1, a HK$200 million donation from the Ng Teng Fong Charitable Foundation (a local philanthropic foundation) 2, with no model cost detail 1.
• A first‑phase government Large Model Smart Computing Center (a public AI training and inference facility) using NVIDIA DGX H800 systems was near HK$600 million 3.
• Heavy first‑week traffic raised inference (running the model to produce answers) costs, so slow replies reveal capacity strain that needs more compute or efficiency steps like quantization (reducing precision to shrink then speed up models) to keep access free.
• HKGAI has not shared a runway (how long current funding lasts) or monetization plans, so without near‑term revenue staying free depends on InnoHK funds plus donations covering compute, with lower per‑query costs via technical gains.

GPU clouds can ease Hong Kong’s low‑latency gap for Large Language Models (LLMs)

• HKChat latency exposes demand for nearby inference that HKUST’s H800 SuperPOD (a clustered GPU system) 4 plus Cyberport’s Artificial Intelligence Supercomputing Centre (AISC) subscriptions 5 serve only for research and startups.
• Cyberport offers H800 access from HK$1,800/month 5, but air‑gapped conditions (no internet connectivity) plus 20GB transfer caps 5 block real‑time chatbots.
• TraxComm runs Tier III+ (high‑availability) sites with carrier‑neutral links plus direct GPU provider connections 6, so firms use colocation (renting space to host your own servers) for hybrid production AI.
• MLOps vendors can help with inference tuning. Techniques include quantization, caching (reusing prior results) and dynamic batching (grouping requests to maximize GPU utilization) which cut H800 cost per query while improving odds access stays free.