The team behind the EAGLE family of speculative-decoding methods released EAGLE-4 on October 20, 2026, publishing inference code on GitHub, a Python package called eagle-harness on PyPI and trained draft models on Hugging Face. Speculative decoding speeds up a large language model by letting a small drafter guess several tokens ahead, which the large target model then checks in a single pass. EAGLE-4's pitch is that one trained drafter per target model can stay useful from a single request up to a busy server handling hundreds of requests at once, on different kinds of hardware.
The central piece is EAGLE-Harness. During a warm-up it profiles how long drafting and verification take on the machine at different batch sizes, context lengths and numbers of speculative tokens, and it stores the results in lookup tables. While generating, it combines those cost estimates with the drafter's confidence to decide how deep and how wide each draft tree should be and how many drafted tokens to send for verification. At low load it can use a branching tree; under heavy load it can switch to a single chain, because a wide tree that pays off for one user can waste compute when hundreds of users are waiting.
Two other changes target the drafter's own overhead. A low-rank output head first shortlists candidate tokens across the whole vocabulary, and only those candidates are scored by the full output layer; Eagle says this cut total draft-step latency by 55.6% in its profiled configuration. The drafter also uses sliding-window attention over the most recent 4,096 positions and an append-only draft cache, so its attention cost stays flat as conversations grow. According to the project, the Qwen drafter is a single Transformer layer with about 0.44 billion parameters.
The headline figures are Eagle's own. On NVIDIA Nemotron 3.5 Super VL, a 120B-A12B model, running on four NVIDIA B300 GPUs, the company reports 647 tokens per second for one request, 2.74 times plain decoding, and 33,480 aggregate tokens per second at 512 concurrent requests, a 1.69 times gain. For Qwen3.8-27B it reports beating two rival drafting methods, DSpark and DFlash 2, at both one and 512 concurrent requests. AMD contributed day-0 results on an Instinct MI350X, where single-request throughput rose 3.49 times, and the launch post also describes CPU tests with a 4-bit Gemma model through llama.cpp, including a phone run in which energy per generated token fell by about 53%.
The code installs as a plugin into patched builds of SGLang and vLLM, and into TokenSpeed, an inference engine from the LightSeek Foundation. LightSeek published its own post the same day describing how its TorchSpec tool trains EAGLE-4 drafters and how TokenSpeed serves them. The patched engines ship as pinned wheels and container images. At release, the ROCm container was not yet published, qualification of the vLLM image was still pending according to the install guide, and TokenSpeed support was marked experimental. A 0.1.1 update followed on October 21, and the repository had about 780 GitHub stars on October 22.
One detail matters for anyone planning production use: EAGLE-4 is not open source. The README describes it as source-available software under the Eagle 4 Source-Available License. That licence allows some commercial use only below a US$20 million aggregate revenue threshold and requires a separate commercial agreement for competing offerings regardless of revenue. Model weights stay under their owners' own licences.
Independent benchmarks were not yet available when this story was checked. All throughput figures come from Eagle, its paper and its integration partners, and most were measured on a single high-end GPU configuration at temperature 0, so real-world gains will depend on the model, hardware, workload and sampling settings.