Kimi K3 from the inside. What it takes to run it yourself
If your organisation is considering running an AI model in-house, you have probably come across Kimi K3. Before any decision, it is worth knowing how it was built, because its architecture decides what it costs you to run.
In short: The Chinese company Moonshot released Kimi K3 in July 2026 and published its weights and technical report on 27 July. It has 2.78 trillion parameters, 104.2 billion of them active per token, and reads one million tokens in a single request. It ranked third on the Artificial Analysis index on launch day.
1. Experts: large knowledge, small compute
Each of its 92 mixture-of-experts layers holds 896 small networks, and a router reads every token and sends it to the best 16, alongside two shared experts that run on every token. On our arithmetic each expert holds about 33 million parameters, so the experts together hold 2,722.7 billion, about 98% of the model's weights.
What matters: on our arithmetic each token costs about 208 billion operations, and a dense model of the same size would cost 26.7 times as much. The price is that every expert must stay in memory, whichever ones a token picks.
2. The transport trick
When a token is sent to its experts, it travels between GPUs. In the previous generation, K2, it went to 8 experts at a width of 7,168 numbers. In K3 it is first compressed to 3,584 numbers, then sent to 16 experts. The total is the same in both cases: 57,344 numbers. Moonshot doubled the number of active experts without adding traffic between GPUs.
3. Hybrid attention
Definition: attention is the step where the model compares each token with what came before it, and it normally keeps a memory that grows with every new token.
In K3, 69 of 93 layers use a linear attention Moonshot calls Kimi Delta Attention: a fixed-size state updated with each token, so it does not grow however long the text. The remaining 24 layers keep a compressed memory with no positional encoding, so the model needed no special tricks to reach one million tokens. The result is about 26 kilobytes per token, against about 70 in K2, so a one-million-token conversation needs about 26 gigabytes.
4. Stability in training
Moonshot trained the model with the Muon optimiser, which learns from tokens faster than the common AdamW but tends to inflate attention values until training collapses in a sudden spike. A spike in a run of trillions of tokens can cost days of rollback. In K2 it fixed this with a step called QK-Clip, which rescales the query and key weights whenever the largest value passes a threshold, and K2 trained on 15.5 trillion tokens without a single spike. K3 adds updating each attention head separately.
5. Running it: usage sets the price
The published weights are 1.56 terabytes, because the experts are stored at four and a quarter bits per parameter. Eight B200 GPUs hold 1,440 gigabytes, which is not enough; eight B300s are enough, and 16 H200s barely fit.
The price is set by the number of users more than by the type of GPU. An expert only runs efficiently when it receives many tokens per step. On our arithmetic, a batch of 8 tokens touches about 13% of the experts, 64 tokens 68%, and 256 tokens 99%. That is why a million tokens cost $0.16, input and output, in a wide deployment on rented B200s, and about $139 per million output tokens on 16 H200s serving 54 users on our arithmetic, and about $2.31 on average through Moonshot's API. Electricity at the Kingdom's industrial tariff is about half a cent per million tokens at full load.
6. What independent tests measure
K3 scored 57 on the Artificial Analysis index on launch day, in third place. Its accuracy on their knowledge test rose from 33% to 46% over the previous generation, and its hallucination rate rose with it, from 39% to 51%. In October Moonshot announced K3.1 Pro, which it says addresses this, and promised to publish its weights by 4 November.
The licence has conditions
The Kimi K3 licence allows running, modifying and redistributing the model. A group with revenue above $20 million that sells the model as a service needs a separate agreement with Moonshot; internal use falls outside that condition.
And the Kingdom?
SDAIA put out for consultation, until 5 November, a draft amendment to the implementing regulation of the Personal Data Protection Law that would require personal data to be stored inside the Kingdom. Localisation for cloud computing has moved from the National Cybersecurity Authority's controls to the National Data Management Office. And the computing is arriving: AMD MI355X systems have run in the Kingdom since 31 August 2026, and HUMAIN holds a US licence for up to 35,000 GB300 GPUs.
The question now
The model is available, and the computing is coming. Two questions decide it: how many users will the model serve each hour inside your organisation, and who on your team will test it before it goes live?
This analysis comes from research by the Horizon Studio RDI Center. We checked every figure at its source, and we say so where a number is our own calculation.
To assess your organisation's readiness to run its own model: https://horizon.bishernp.com/enquire
References
Moonshot AI, Kimi K3 model card and technical report, July 2026.
Moonshot AI, Kimi Linear, October 2025, and the Kimi K2 post, July 2025.
Artificial Analysis, Intelligence Index and Kimi K3 page, July and October 2026.
vLLM, Kimi K3 performance optimisation, September 2026; SemiAnalysis InferenceX, October 2026.
Moonshot AI, Kimi K3 licence, July 2026.
Al Tamimi, SDAIA consultation on the implementing regulation, October 2026.
National Cybersecurity Authority, Cloud Cybersecurity Controls, 2024.
Your data is in the Kingdom. What changed in 2024?
Where must your data be kept in Saudi Arabia? In 2024 the localisation rule moved to SDAIA, but the real questions now are about regulators, people and contracts.
Who owns Saudi space? A map of the sector in 2026
Working in Saudi Arabia's space sector in 2026 requires understanding its structure. This guide maps out the key players: the regulators, builders, PIF funders, and service providers like Starlink.
Your first bank or defence pilot: what should it prove?
A tech pilot in a highly regulated institution is more than a simple demo; it is proof of regulatory strength. Discover the six main metrics for success, like rapid audit records, and three to ignore.