NVIDIA has quietly put a new local inference server into the open. The repository NVIDIA/flare-local-inference appeared on GitHub on October 20, 2026 under the Apache 2.0 licence, describing Flare as a server for NVIDIA SM120 and SM121 GPUs, the class that covers RTX PRO Blackwell workstation cards and the GB10 chip in DGX Spark. Its stated goal is to make the memory, bandwidth and compute of desk-side machines go further for long conversations, coding agents and repeated tool use, while keeping generation responsive. NVIDIA labels it a research prototype released to enable technical collaboration and gather early feedback.
The design splits the work in two. A build-time tool called Ignition uses PyTorch, Transformers and FlashInfer kernels to load a model, tune kernels, compile the forward pass and capture CUDA graphs into a runtime package. A Rust runtime called Helios then serves that package with no Python installed. All compilation and graph capture finish before serving starts, and buffers and capacities are planned ahead of time. The project also treats a conversation's whole state, including recurrent state alongside the key-value cache, as one unit, so that reusing, branching and resuming a session keep it intact.
Flare does not try to run everything. Its documentation lists model recipes that include Nemotron 3.5 Bolt, Qwen3.8 27B and Qwen3.8 Flash, DeepSeek V4 Flash and V4.1 Flash, MiMo V2.6 Pro, Step-5 Preview and Kimi K3. Every recipe stores the key-value cache in FP8 by default. Models too large for a single DGX Spark can run with tensor parallelism across two or four Sparks linked through their ConnectX-7 ports, with one machine building the package and a coordinating command starting every rank.
The README is candid about where it fits. It says vLLM covers a much broader range of models, hardware and deployment patterns, and presents Flare as a specialised option that trades breadth for a focused serving contract around selected models and local workloads. What a user can run is bounded by the supported recipes and the capacities prepared in advance. On DGX Spark, where the GPU and CPU share the same memory, the guide asks users to build packages on the Spark itself and to reserve memory for the operating system.
There was no launch announcement. A post on NVIDIA's developer forum on October 20 flagged the repository as having arrived with no announcement and single-digit stars, and early users in that thread began posting their own throughput numbers. Independent write-ups by explainx.ai and Tokenstead summarised the architecture the same day. By October 22 the repository had about 560 stars, no tagged releases and no performance figures in its README.
The release matters because NVIDIA has been selling DGX Spark and RTX PRO Blackwell machines as a home for private, local AI agents, while most people running them rely on community engines such as llama.cpp or general servers such as vLLM. A first-party runtime built around agent-style sessions, where long contexts and repeated tool calls dominate, gives those owners another option and shows how NVIDIA thinks local serving for agents should be structured.
Important gaps remain. NVIDIA has published no benchmarks, the project is explicitly a prototype that may change quickly, it supports only Linux on SM120 and SM121 hardware, and the README does not include install commands, pointing instead to separate design and preparation documents.