DeepSeek has released the code of the engine it uses to run its own models. On October 20, 2026, the Chinese lab put DeepInfer on GitHub under the permissive Apache 2.0 license and announced it on X, describing it as the in-house framework that serves DeepSeek-V4.1-Flash to its users. For now that is the only model it supports, though DeepSeek says inference code for later models and further performance work will be published in the same repository.
Open weights have been DeepSeek's calling card, but weights alone do not decide what a model costs to operate. The serving layer that batches requests, splits work across chips and generates tokens can change throughput and spending dramatically. In April 2025 the company had said that releasing its whole inference engine was impractical and that it would contribute pieces to existing projects instead, a stance The New Stack notes this release reverses in part.
According to the repository, DeepInfer bundles DeepSeek's own kernel libraries, including FlashMLA, DeepGEMM and DeepEP, with attention data parallelism, expert parallelism for its mixture-of-experts layers, asynchronous scheduling, structured outputs and a speculative decoding scheme called DSpark that decides how many drafted tokens are worth checking. A companion library, deepseek-recipe, available in Rust and Python, reproduces how the official API converts requests into tokens and back, so third-party hosts can match its behavior.
Hardware support is the detail that drew the most attention. The code targets two platforms: NVIDIA's SM100 architecture, which powers its Blackwell data center chips, and Huawei's Ascend 950 NPUs. Older NVIDIA generations such as Hopper are not listed. Two reproducibility features, batch-invariant inference and deterministic sampling, work only on NVIDIA hardware, and the published benchmarks were run on NVIDIA chips (prefill on four SM100 GPUs, decode on 64), leaving Ascend performance unmeasured in public.
It is not a turnkey service. The README says a complete deployment still needs a KV cache system, separation of prefill and decode stages across machines, and load balancing across replicas, none of which ship in this release. A long deployment guide in the repository describes how DeepSeek runs those parts internally, and the lab says a single production cluster handles 155 trillion tokens a day, earning about 38 dollars per GPU-hour at off-peak prices. Those numbers come only from DeepSeek.
Independent coverage confirms the release and its limits. The Next Web, citing Bloomberg, reported that no outside party had yet benchmarked DeepInfer and that the repository passed 8,000 stars within a day. The South China Morning Post quoted a DeepSeek researcher framing the contest between open and closed AI in stark terms. The vLLM project, on which DeepSeek says DeepInfer was originally built, thanked the company and said it would borrow ideas for its own V4.1-Flash support.
For cloud providers already hosting DeepSeek models, the code offers a reference for the company's own optimizations. For everyone else, the practical value may be studying it and porting techniques into engines such as vLLM or SGLang, at least until independent tests and the missing serving components arrive.