Kimi K3: Moonshot ships 2.8T-param open-weight MoE, trailing only Fable 5 and GPT-5.6

Moonshot AI has released Kimi K3, a Mixture-of-Experts model with 2.8 trillion total parameters but only 104 billion activated per token, native vision, and a one-million-token context window. The architecture leans on three ideas the team credits for roughly 2.5x better scaling efficiency than Kimi K2: Kimi Delta Attention for cheap long-sequence mixing (with periodic Gated MLA layers for global attention), Attention Residuals that let each layer draw on all earlier layers, and a ‘Stable LatentMoE’ design that routes among 896 experts while activating just 16, held stable at that extreme sparsity by normalization and quantile-based load balancing. Post-training is built around reinforcement learning across coding, agentic, reasoning, and knowledge tasks at multiple ‘thinking effort’ levels, with specialized policies folded back into one model via multi-teacher on-policy distillation.

The headline framing is that open models have been stuck near the 1T-parameter class while proprietary systems pulled ahead, and Kimi K3 pushes the open frontier to 3T-class scale on both pre-training and test-time compute. On Moonshot’s own benchmark suite spanning long-horizon coding, agentic work, vision, and reasoning, K3 posts frontier-level numbers and consistently beats other open and proprietary models tested—including GPT-5.5, Claude Opus 4.8, and GLM-5.2—while still trailing the two strongest proprietary systems, Claude Fable 5 and GPT-5.6 Sol. It leads the pack outright on several tasks such as ProgramBench, BrowseComp, SWE-Marathon, and AutomationBench.

Much of the report is about the systems engineering needed to train and serve a model this large: fused KDA kernels and context parallelism, a ‘MoonEP’ expert-parallel scheme with balanced execution and zero-copy communication, and a million-token agentic RL stack using partial rollouts, external KV-cache retention, and resumable microVM sandboxes to keep long-lived model and environment state alive across thousands of tool calls. Notably, Moonshot is releasing the full model weights on Hugging Face, continuing its pattern of putting frontier-scale open weights into researchers’ and builders’ hands.