DeepSeek open-sources DeepInfer, its production inference engine

What happened: DeepSeek open-sourced DeepInfer under the Apache 2.0 licence. Its GitHub README calls it "the high-performance inference framework that powers DeepSeek's large-scale production services."

The details: DeepInfer targets DeepSeek-V4.1-Flash and supports Nvidia SM100 GPUs and Huawei Ascend 950 NPUs. The listed kernels are FlashMLA, DeepGEMM, DeepEP, TileKernels and DeepSelect, alongside TileLang.

The features: It offers attention data parallelism with balanced context parallelism, and MoE expert parallelism with EPLB. It also has draft-token acceleration with confidence-scheduled verification (DSpark), an async zero-overhead batch scheduler and a batch-invariant mode. Constrained generation works by regex, JSON schema or EBNF grammar.

The numbers: DeepSeek claims about 155 trillion tokens per day per cluster, with a 96% input cache hit rate. That figure has not been independently verified.

The context: The README says DeepInfer was "originally built primarily on vLLM", with ideas drawn from SGLang and TensorRT-LLM. The release follows DeepSeek's statement last week that it plans to pre-train on Huawei Ascend 950 chips by the end of 2026.

Why it matters: Outside developers can now run the same serving stack DeepSeek says it uses in production. Support for Ascend 950 NPUs sits next to Nvidia support in the same codebase.

The catch: Reports differ on the repository's popularity, from 5.5k stars to more than 8,000 within hours. One report says the launch came a day after a US DHS ban on DeepSeek in federal agencies. That is unverified.

What's next: Check the repository directly for current star counts and updates.