FreeToken roadmap targets AMD, Apple Silicon and multi-GPU expansion

FreeToken is already known for running very large Mixture-of-Experts models on consumer and workstation hardware, but its next challenge is broader than squeezing more parameters into limited VRAM. The project has now published a 2026 roadmap that aims to move FreeToken beyond its current NVIDIA-focused runtime and turn it into a more heterogeneous local-AI platform.

The official roadmap, published by the maintainers on August 23, lists native support for Apple Silicon through a Metal engine, AMD GPUs through ROCm, and DGX Spark through aarch64 wheels, sm_121 kernels and a unified-memory mode.

It also targets multimodal image input for Qwen3.5, Qwen3.6, Qwen3.8 and Gemma-4 families, tensor parallelism across multiple GPUs, broader GGUF support across model architectures and speculative decoding techniques including MTP, DFlash and Dspark.

Current support is still NVIDIA-only

The roadmap is important partly because it highlights the limits of the current release.

FreeToken's maintainer FAQ still defines official support as x86_64 hardware with an NVIDIA GPU from the Ampere generation onward, effectively RTX 30 series and newer, with driver r580+ and CUDA 13. Windows and Linux are supported through FreeToken Desktop, while the Python package is supported on Linux.

The project's installation documentation makes the same requirement explicit: Linux x86_64, NVIDIA GPU and CUDA 13 for the packaged runtime.

That means AMD, macOS and DGX Spark are not newly supported products today. They are roadmap commitments.

This distinction matters because the roadmap itself is marked as work in progress. Treating it as a shipping announcement would overstate the current state of the project.

Community ports are already testing the AMD path

The interesting part is that the AMD target is no longer only theoretical.

A community contributor reported a Windows 11 plus ROCm port running FreeToken on an AMD RX 9070 XT without an NVIDIA toolchain. The report says model loading, prefill, decode, SSE streaming and the bundled web interface were all working on the fork, with a Qwen2.5-3B test reaching roughly 57 tokens per second in BF16.

That port is not official upstream support. The same report says MoE expert offload had not yet been fully tested and several GGUF and quantization paths were incomplete.

A second community report went further. On an RX 9060 XT, a contributor says the packed-GGUF path was brought up end-to-end for both dense and MoE workloads, while also identifying low-level host-memory and device-pointer problems behind an RDNA4 offload-decode crash.

Those reports are useful because they show that at least part of FreeToken's CUDA-centric architecture can be adapted to ROCm. They do not prove that AMD support is production-ready or close to an official release.

Why the roadmap matters

FreeToken's original value proposition is hardware elasticity: use GPU memory, system RAM, CPU execution and available interconnect bandwidth together instead of treating VRAM as a hard boundary.

That design becomes more strategically interesting if it works across multiple GPU vendors.

Today, local AI infrastructure is highly fragmented. NVIDIA has the strongest software ecosystem, but developers increasingly want to run models on AMD desktops, Apple Silicon systems and smaller unified-memory machines without rewriting their serving stack for each platform.

If FreeToken can keep the same serving model across CUDA, ROCm and Metal, its role changes. It would no longer be just a specialized MoE runtime for RTX hardware; it could become a portability layer for large local models across heterogeneous personal machines.

The multi-GPU roadmap pushes in the same direction. Tensor parallelism would let a single model span multiple GPUs in one workstation, while planned DGX Spark support would extend the system toward linked compact AI systems.

Multimodal and speculative decoding broaden the target

The roadmap is not only about hardware.

Image input for Qwen and Gemma vision-language models would move FreeToken beyond text-only local agents. Broader GGUF support would make it easier to use the large existing ecosystem of community-quantized checkpoints rather than relying primarily on safetensors and a narrower set of supported quantization paths.

Speculative decoding could also matter for interactive agents. Techniques such as MTP and DFlash aim to reduce decode latency by predicting multiple tokens or draft sequences ahead of the main model. That would complement FreeToken's existing focus on memory movement and bandwidth utilization.

Together, these features suggest that the maintainers are trying to turn FreeToken from a research implementation into a broader local serving platform.

What to watch next

The roadmap is ambitious enough that execution matters more than the feature list.

The first milestone to watch is official AMD support. Community forks have shown that RDNA4 bring-up is possible, but upstream integration, packaging, kernel coverage, MoE offload reliability and repeatable performance remain separate engineering problems.

Apple Silicon may be even more significant. A native Metal backend would open FreeToken to machines with large unified-memory configurations, a hardware profile that is naturally attractive for oversized local models.

Multi-GPU tensor parallelism is another major test because it changes FreeToken from a single-device-plus-host-memory runtime into a distributed local serving system.

For now, the key development is not that FreeToken suddenly supports every major hardware platform. It does not. The meaningful news is that the project has publicly defined a path beyond its NVIDIA-only launch, and its community is already beginning to validate pieces of that path on AMD hardware.

Sources
- FreeToken Roadmap (2026)
- FreeToken FAQ
- FreeToken installation requirements
- Community Windows/ROCm RDNA4 port
- Community RX 9060 XT GGUF and MoE bring-up