# DeepSeek open-sources DeepInfer, its production inference engine

> DeepSeek released DeepInfer, its production inference framework, as open source under Apache 2.0.

*The Apache 2.0 framework targets DeepSeek-V4.1-Flash and runs on both Nvidia and Huawei hardware.*

By Bez · FeaturedDaily
Canonical: https://featureddaily.com/news/deepseek-open-sources-deepinfer-its-production-inference-engine

**What happened:** DeepSeek open-sourced DeepInfer under the Apache 2.0 licence. Its [GitHub README](https://github.com/deepseek-ai/DeepInfer) calls it "the high-performance inference framework that powers DeepSeek's large-scale production services."

**The details:** DeepInfer targets DeepSeek-V4.1-Flash and supports Nvidia SM100 GPUs and Huawei Ascend 950 NPUs. The listed kernels are FlashMLA, DeepGEMM, DeepEP, TileKernels and DeepSelect, alongside TileLang.

**The features:** It offers attention data parallelism with balanced context parallelism, and MoE expert parallelism with EPLB. It also has draft-token acceleration with confidence-scheduled verification (DSpark), an async zero-overhead batch scheduler and a batch-invariant mode. Constrained generation works by regex, JSON schema or EBNF grammar.

**The numbers:** DeepSeek claims about 155 trillion tokens per day per cluster, with a 96% input cache hit rate. That figure has not been independently verified.

**The context:** The README says DeepInfer was "originally built primarily on vLLM", with ideas drawn from SGLang and TensorRT-LLM. The release follows DeepSeek's statement last week that it plans to pre-train on Huawei Ascend 950 chips by the end of 2026.

**Why it matters:** Outside developers can now run the same serving stack DeepSeek says it uses in production. Support for Ascend 950 NPUs sits next to Nvidia support in the same codebase.

**The catch:** Reports differ on the repository's popularity, from 5.5k stars to more than 8,000 within hours. One report says the launch came a day after a US DHS ban on DeepSeek in federal agencies. That is unverified.

**What's next:** Check the repository directly for current star counts and updates.

## Key takeaways

- DeepInfer is Apache 2.0 and built for DeepSeek-V4.1-Flash
- Supports Nvidia SM100 GPUs and Huawei Ascend 950 NPUs
- Throughput claims come from DeepSeek and are unverified

## Sources

- [DeepInfer repository](https://github.com/deepseek-ai/DeepInfer) — GitHub (deepseek-ai), 2026-10-20