DeepSeek Opens DeepInfer Source, Backs 2 Chips [2026]

DeepSeek pulled back the curtain on its production infrastructure this week. On October 20, 2026, Hangzhou DeepSeek Artificial Intelligence Basic Technology Research Co. Ltd. open-sourced DeepInfer, the inference framework the company has used internally to run its large-scale AI services. The Information first flagged the release as a rare look inside the system behind DeepSeek’s low-cost AI serving. The code went up on GitHub under the Apache 2.0 license, alongside a second, smaller release called deepseek-recipe.

DeepSeek announced the release on X, describing DeepInfer as “our in-house inference framework, now fully open source,” and adding that it is “Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency,” according to the company’s own post. For a company that built its reputation on undercutting Western AI labs on price, opening the engine room is a notable move. It gives outside developers, and rival labs, a direct look at how DeepSeek keeps its inference costs down.

Google · Preferred Sources

Don't miss new tech stories on Google

Add TrendinTech once in the Google app and our stories appear in your news suggestions.

Add Now

What DeepSeek Actually Released on October 20

DeepInfer is described in its own GitHub documentation as the high-performance inference framework that powers DeepSeek’s large-scale production services. The initial open-source drop covers offline inference: optimized kernels, a model and sampler runtime, attention data parallelism, balanced context parallelism, mixture-of-experts expert parallelism with expert-parallel load balancing (EPLB), the DSpark speculative-decoding system, confidence-scheduled verification, sliding-window-attention bounded replay, asynchronous scheduling, batch-invariant inference, deterministic sampling, and structured outputs.

That is a long feature list for a first release, and it matches what The New Stack reported when it broke down the framework’s scope. DeepSeek paired the main release with deepseek-recipe, a smaller companion library that converts incoming API requests into model inputs and converts model outputs back into API responses, matching the behavior of DeepSeek’s own hosted API. Developers building a DeepSeek-compatible service now have both halves of the pipeline: the engine that runs the model and the glue code that makes it speak the same request and response format as DeepSeek’s production API.

Right now, DeepInfer supports exactly one model: DeepSeek-V4.1-Flash, the same model family DeepSeek pushed an update to just three days earlier. That update, DeepSeek-V4.1-Flash-1017, shipped on October 17, 2026, and is now the default checkpoint referenced in DeepInfer’s own example configurations and evaluation scripts. Bloomberg reported that DeepSeek positioned the update as faster and leaner, part of its ongoing effort to challenge OpenAI and Anthropic on both cost and speed.

Why Open-Sourcing Inference Code Is Different From Open-Sourcing a Model

DeepSeek has open-weighted models before. What it had not done, until this week, was open the production stack that actually serves those models to paying customers at scale. That distinction matters more than it might sound. A model’s weights tell you what the AI knows. An inference framework tells you how a company turns those weights into fast, cheap, reliable answers for millions of concurrent requests. The second problem is, in many ways, the harder engineering challenge, and it is the one that determines whether an AI company’s unit economics work at all.

SiliconANGLE’s coverage of the release framed it the same way, noting that DeepSeek open-sourced the exact framework behind its production services, not a simplified or stripped-down demo version. Other labs, including the teams behind vLLM and SGLang, maintain open inference engines too, but those are general-purpose tools built to serve many different model architectures. DeepInfer, by contrast, is purpose-built around DeepSeek’s own models and its own hardware mix, which gives it narrower scope but a tighter fit with the workload it was designed for.

The Hardware Story: Nvidia and Huawei, Treated as Equals

The detail that caught the most attention in trade coverage was hardware support. DeepInfer’s public release supports NVIDIA SM100 data-center GPUs and Huawei Ascend 950 NPUs, listed side by side as first-class targets. The New Stack’s headline on the story put it bluntly: Huawei gets equal billing with Nvidia. That is a meaningful signal. A framework built to run equally well on both an American chip architecture and a Chinese one tells the market that DeepSeek is actively hedging against export restrictions that have made Nvidia hardware harder to source inside China.

The Huawei support did not appear out of nowhere. According to reporting tied to the release, DeepSeek and Huawei had already collaborated to bring the TileLang kernel-programming language to Ascend chips in a separate release on September 30, 2026, which laid the groundwork for Ascend 950 becoming a first-release target in DeepInfer three weeks later. Some features in the current release, including batch-invariant inference and deterministic sampling, are documented as available only on the Nvidia side for now, so the two hardware paths are not yet at full feature parity.

DeepInfer’s own deployment documentation on GitHub describes a split architecture: ordinary 8-GPU machines handle the prefill stage, while larger 64-to-128-GPU supernodes handle decoding, with key-value cache retention extending beyond 72 hours. That architecture detail matters for anyone trying to estimate what it would actually cost to replicate DeepSeek’s production setup rather than simply running the model on a single machine.

What’s Missing From the Public Release

DeepSeek was upfront about the limits of what it shipped. The public version of DeepInfer provides offline inference only. It does not include a key-value cache management system, infrastructure for separating the prefill and decode stages across different machines, or replica load balancing across a fleet of servers. Those three pieces are exactly what a company needs to turn an inference engine into a complete, internet-facing online service capable of handling unpredictable traffic at scale.

In practice, that means developers can study DeepInfer’s kernels, its parallelism strategies, and its speculative-decoding approach, and they can run batch workloads against DeepSeek-V4.1-Flash on their own hardware. What they cannot do, at least not yet, is stand up a drop-in replacement for DeepSeek’s hosted API with the same reliability characteristics DeepSeek itself runs in production. That gap between “the code that powers our service” and “a complete service you can run yourself” is a deliberate line, and it is one worth watching, because DeepSeek could close it with future releases or could leave third parties to build that layer themselves.

DeepInfer vs vLLM: What the Numbers Can and Cannot Tell You

DeepInfer’s own README documents benchmark configurations for both the prefill and decode stages. The prefill configuration uses four SM100 GPUs with an expert-parallel-by-4, data-parallel-by-4 layout, 384 routed experts plus 32 shared experts, and a 131,072-token context window. The decode configuration scales up to 64 SM100 GPUs with an expert-parallel-by-64, data-parallel-by-64 layout, the same 384-plus-64 expert mix, a fixed next-token lookahead of three positions for speculative decoding, and a reported 75% draft-token acceptance rate.

Those are DeepInfer measuring its own engine’s throughput-latency frontier on its own hardware configuration, and they have not been independently reproduced by a third party as of this week. Separately, the team behind vLLM, a widely used open-source inference engine, has published its own comparison figures: a 1.9x speedup at low concurrency and a 5.3x throughput improvement at a 150-tokens-per-second interactivity constraint, measured on the SemiAnalysis AgentX benchmark, a workload built around long inputs and heavy prefix-cache reuse. Those numbers come from a different workload on different hardware assumptions, so they are not a like-for-like comparison with DeepInfer’s own figures. Readers should treat both sets of numbers as vendor-reported until independent benchmarking firms run their own tests across both engines on identical hardware.

The Pricing Backdrop: Why Inference Cost Matters to DeepSeek

DeepSeek’s business runs on API pricing that undercuts most Western competitors, and inference efficiency is the lever that makes that pricing possible. Pricing trackers covering DeepSeek’s API in October 2026 report peak pricing for DeepSeek-V4.1-Flash at $0.30 per million input tokens and $1.20 per million output tokens, with cached-input tokens priced far lower at roughly $0.006 per million. DeepSeek’s heavier model, V4 Pro, is reported at $1.32 per million input tokens and $3.96 per million output tokens at peak. Off-peak pricing, tied to DeepSeek’s UTC+8 peak and off-peak schedule, is reported at roughly half the peak rate.

Those figures come from third-party pricing aggregators rather than a DeepSeek press release, so they should be read as reported rather than officially confirmed. Still, the pattern lines up with why DeepSeek would want to open its inference stack: every percentage point of throughput DeepInfer squeezes out of a fixed GPU fleet turns directly into either a lower price for customers or a wider margin for DeepSeek. The framework’s DSpark speculative-decoding system and its confidence-scheduled verification step are both aimed squarely at that goal, since speculative decoding lets the model draft several tokens ahead and verify them in a batch rather than generating one token at a time.

It is worth being precise about what is and is not confirmed here. The framing that this release exposes a “high-margin” inference operation has circulated in coverage of the story, but no DeepSeek disclosure reviewed for this piece includes an actual margin figure, a cost-per-token figure for running DeepInfer internally, or a profit number tied to the framework. The efficiency claims in DeepInfer’s own documentation are about throughput and latency, not about dollars and cents.

DeepSeek’s Broader October: A Busy Month for One Lab

The DeepInfer release did not happen in isolation. DeepSeek has had an unusually active month. On October 17, 2026, it shipped DeepSeek-V4.1-Flash-1017, an update to its flagship fast model that TechNode and Malay Mail both covered as a response to recent service disruptions, with DeepSeek billing the new version as faster and leaner. The same week, separate reporting from Yahoo Finance described DeepSeek introducing paid Search and Fetch APIs aimed at AI agents, reportedly priced at $0.75 per 1,000 successful calls, with billing scheduled to begin at 00:00 Beijing time on November 1, 2026. That detail is drawn from a single source and has not been independently corroborated, so it should be read as a reported plan rather than a confirmed price.

Taken together, the month’s releases point to a company trying to compete on three fronts at once: better models, cheaper infrastructure, and new monetizable surfaces like agent-facing APIs. The DeepInfer open-source release fits into that pattern as the infrastructure piece, the part of the stack that makes the other two cheaper to run. For context on how crowded the open-model field has already become, DeepSeek is competing directly against other open releases covered elsewhere this year, including the comparison at Kimi K2 vs DeepSeek vs Qwen3 and the earlier API opening detailed in DeepSeek V4.1-Flash-1017’s API rollout.

How This Compares to Other Open-Source Inference Moves in 2026

DeepSeek is not the only AI company that has opened up parts of its serving stack this year. Google released TorchTPU, exposing roughly 450 PyTorch operators for running on its TPU hardware rather than Nvidia chips, a move covered in depth in Google’s TorchTPU open-source release. Nvidia itself open-sourced Run:ai tooling for GPU orchestration earlier this year, detailed in NVIDIA Run:ai’s open-source push. Meta’s SGLang-backed Muse Spark model got day-zero inference support from the SGLang project, a different flavor of the same infrastructure-openness trend, covered in SGLang’s day-zero support for Muse Spark.

What sets DeepInfer apart from most of those releases is the dual-hardware commitment. Nvidia’s own China market share has reportedly fallen to around 34%, with Huawei’s Ascend line climbing to roughly 28%, a shift tracked in Nvidia’s China share decline. A framework that runs natively on both chip families, built by one of China’s most closely watched AI labs, is as much a geopolitical statement as it is an engineering one. It also puts pressure on US-based AI labs that have, so far, kept their production inference stacks closed, a contrast to the broader open-source momentum described in the open-source AI bet gaining ground across the industry.

Market and Industry Reaction

Reaction across trade press was largely framed around two angles: cost transparency and the Nvidia-Huawei parity. SiliconANGLE’s write-up emphasized that DeepSeek is “sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable,” quoting DeepSeek’s own announcement language. The New Stack’s framing leaned harder into the hardware-politics angle, treating the Huawei support as the more consequential story than the open-sourcing itself. Neither outlet reported a stock-market reaction tied directly to the release, which is consistent with DeepSeek’s private ownership structure; the company is not a publicly traded entity, so there is no ticker to move on this kind of announcement.

Developer-facing coverage, including analysis from AlphaSignal and NxCode, focused on the gap between what DeepInfer ships today and what a production-grade hosted service actually needs. Both outlets flagged the missing KV-cache management layer and the missing multi-machine load balancing as the reason DeepInfer, by itself, will not let a third party spin up a DeepSeek-API clone overnight. That caution lines up with DeepSeek’s own documented limitations and suggests the developer community is reading this release with appropriate skepticism about how far it actually goes.

Historical Context: DeepSeek’s Pattern of Infrastructure Disclosure

DeepSeek built its global profile in early 2025 by showing that a relatively small, efficient team could train a frontier-competitive model for a fraction of the compute budget that larger US labs were reportedly spending. That story was always partly about the model and partly about the infrastructure discipline behind it. The DeepInfer release extends that same narrative into 2026: instead of just claiming efficient training, DeepSeek is now showing its efficient serving code directly. The company’s MLA and mixture-of-experts architecture choices, which it has written about extensively on its own technical blog, were always positioned as cost-saving measures relative to traditional dense transformer models. DeepInfer is the serving-side counterpart to those training-side efficiency claims.

It also continues a pattern of DeepSeek timing releases closely together. The V4.1-Flash-1017 model update, the DeepInfer open-source release, and the reported Search and Fetch API rollout all landed within a four-day window in mid-to-late October 2026. Whether that reflects a coordinated product calendar or simply a busy engineering team shipping whatever is ready is not something the public record settles either way.

DeepInfer Feature Snapshot

Component	Status	Key detail
DeepInfer core engine	Open-sourced Oct 20, 2026	Apache 2.0 license, offline inference only
deepseek-recipe	Open-sourced Oct 20, 2026	Companion library; converts API requests/outputs; Rust with Python bindings
Supported model	DeepSeek-V4.1-Flash	Default example checkpoint is the Oct 17, 2026 “-1017” update
NVIDIA hardware path	Supported	SM100 data-center GPUs; full feature set including batch-invariant inference
Huawei hardware path	Supported	Ascend 950 NPUs; some features, including deterministic sampling, Nvidia-only for now
KV-cache management	Not included	Needed for a complete online-serving stack
Prefill/decode disaggregation infra	Not included	DeepSeek’s own production setup separates these stages; public code does not
Replica load balancing	Not included	Required to spread traffic across multiple serving machines

Reported DeepSeek API Pricing, October 2026

Model	Peak input ($/M tokens)	Peak output ($/M tokens)	Cached input ($/M tokens)
DeepSeek-V4.1-Flash	$0.30	$1.20	~$0.006
DeepSeek-V4 Pro	$1.32	$3.96	~$0.044
Search API (reported)	~$0.75 per 1,000 successful calls, billing from Nov 1, 2026
Fetch API (reported)	~$0.75 per 1,000 successful calls, billing from Nov 1, 2026

Off-peak pricing under DeepSeek’s UTC+8 schedule is reported at roughly half these peak rates. These figures are drawn from third-party pricing trackers rather than a DeepSeek rate card, so treat them as reported rather than officially confirmed, and expect them to shift as the company adjusts its surge-pricing policy around the V4 family.

What Developers Can and Cannot Do With This Release

For teams running batch inference workloads, offline evaluation pipelines, or research experiments on DeepSeek-V4.1-Flash, DeepInfer is immediately useful. It gives them the same kernels, parallelism strategy, and speculative-decoding approach DeepSeek itself uses, running on either Nvidia SM100 or Huawei Ascend 950 hardware, with a documented setup involving Linux, GCC 13 or newer, Python 3.12, and either CUDA 13 or CANN 9.2 depending on the hardware path. For teams trying to build a competing hosted inference service, the picture is murkier: they get the fast engine, but they still have to build the cache layer, the disaggregation infrastructure, and the load balancer themselves before they can offer anything resembling DeepSeek’s own API reliability.

That gap is also a business opportunity. Cloud infrastructure providers and inference-hosting startups are likely to treat DeepInfer as a starting point rather than a finished product, wrapping it in exactly the missing layers and reselling access. Readers tracking how developer tooling is evolving around open-source AI infrastructure more broadly may also find relevant context in coverage of how local AI agent setups are being built around open frameworks, since the same KV-cache and load-balancing gaps show up whenever a model-serving framework moves from internal tool to public release.

Predictions: Where This Goes From Here

Expect DeepSeek to add support for additional model checkpoints in DeepInfer beyond V4.1-Flash within the next few months, following the same pattern as its model release cadence.
Third-party cloud providers will likely package DeepInfer with their own KV-cache and load-balancing layers, offering “DeepInfer-compatible” hosting faster than DeepSeek closes that gap itself.
The Huawei Ascend 950 support will draw more attention from Chinese cloud operators seeking to reduce Nvidia dependency, accelerating Ascend adoption in domestic AI serving stacks.
Independent benchmarking firms will publish head-to-head throughput comparisons between DeepInfer, vLLM, and SGLang on identical hardware within weeks, since DeepInfer’s current figures are self-reported and not yet independently verified.
Pressure will grow on OpenAI and Anthropic to explain their own inference economics, even without matching DeepSeek’s move toward opening production code, as DeepSeek’s efficiency claims the public narrative around who runs AI more cheaply.

Where DeepInfer Fits: DeepInfer vs vLLM vs SGLang

It helps to place DeepInfer against the two most widely used open-source inference engines already in circulation. vLLM and SGLang are both general-purpose engines designed to serve a wide range of model architectures from multiple AI labs, which makes them the default choice for most companies hosting open-weight models. DeepInfer takes the opposite approach: it is narrowly tuned for DeepSeek’s own mixture-of-experts architecture and its own speculative-decoding system, DSpark, which the company says can meaningfully improve throughput at DeepSeek’s specific production operating point.

That specialization is both DeepInfer’s strength and its limit. A team running only DeepSeek models may get better performance out of DeepInfer’s tightly tuned kernels than out of a general-purpose engine. A team running a mix of models from different AI labs will likely stick with vLLM or SGLang simply because those tools already support everything they need in one place. The fact that DeepSeek chose to open-source its specialized engine rather than simply contribute optimizations upstream to vLLM or SGLang suggests the company sees enough of a performance edge in its own approach to make a standalone release worthwhile.

What This Means for the Open-Source AI Inference Race

The timing is notable. Inference cost, not training cost, has become the dominant line item for AI companies serving models at scale, since training happens once but inference happens on every single request forever. Every major lab is under pressure to drive that cost down, and DeepSeek’s decision to show its work, even a limited, offline-only version of it, gives the rest of the industry a public benchmark to measure against. Whether other labs follow with similar disclosures, or treat their own inference stacks as a closely guarded competitive advantage, will say a lot about how open the next phase of the AI infrastructure race actually becomes.

Frequently Asked Questions

What is DeepInfer?
DeepInfer is the inference framework DeepSeek uses internally to run its large-scale AI production services. DeepSeek open-sourced it on October 20, 2026, under the Apache 2.0 license.

Which DeepSeek model does DeepInfer support?
The public release currently supports DeepSeek-V4.1-Flash, with example configurations defaulting to the updated DeepSeek-V4.1-Flash-1017 checkpoint released October 17, 2026.

What hardware does DeepInfer run on?
DeepInfer officially supports NVIDIA SM100 data-center GPUs and Huawei Ascend 950 NPUs, giving it a dual-vendor hardware path that few open-source inference frameworks offer.

Can I run a full hosted API service with DeepInfer?
Not out of the box. The public release provides offline inference only and does not include key-value cache management, infrastructure to separate prefill and decode across machines, or replica load balancing, all of which DeepSeek says its own production service uses internally.

What is deepseek-recipe?
It is a companion library DeepSeek released alongside DeepInfer that converts API requests into model inputs and converts model outputs back into API responses, matching the behavior of DeepSeek’s official hosted API. It is written in Rust with Python bindings.

Is DeepInfer faster than vLLM?
DeepSeek’s own documentation reports throughput figures for DeepInfer on its own benchmark configuration, and vLLM’s team has published separate comparison figures on a different workload. Neither set of numbers has been independently reproduced by a third party on identical hardware, so a clear head-to-head verdict is not yet available.

Does this release confirm DeepSeek’s profit margin on inference?
No. While the release has been widely framed as exposing DeepSeek’s cost-efficient inference approach, no DeepSeek disclosure tied to this release includes a specific margin, revenue, or cost figure.

Why does Huawei chip support matter here?
Nvidia’s share of the Chinese AI chip market has reportedly declined as Huawei’s Ascend line has grown, and a major Chinese AI lab building equal support for both chip families into its core inference framework signals a deliberate hedge against further export restrictions on Nvidia hardware.

DeepSeek Opens DeepInfer Source, Backs 2 Chipmakers [2026]