Soup v0.74.0 fixes a memory-management mistake that could make LoRA training look far more GPU-hungry than the model configuration implied. The release, published on September 4, corrects three model-loading paths that were materializing a frozen base model in fp32 even though that base was not being updated by the optimizer.
The project's v0.74.0 release notes report that, on an H100 with Llama-3.1-8B and LoRA, the change reduced peak memory from 48,241 MiB to 18,658 MiB. Soup describes that as a 2.59x reduction and says the result was byte-identical across three repeats. Those are project-reported measurements, not an independently reproduced benchmark. The package itself is available as soup-cli 0.74.0 on PyPI.
The bug was a default, not a larger model
The important technical detail is that the frozen base was being loaded at a wider numeric precision than necessary. Soup says its text, vision and audio from_pretrained paths did not pass an explicit dtype, so the base model was materialized in fp32. For LoRA-style training, that base remains frozen while comparatively small adapter parameters are trained.
Soup now loads the frozen base at the checkpoint's intended precision while deliberately keeping a trainable base in fp32 for full fine-tuning. That distinction matters because the memory reduction comes from correcting representation of parameters that are not being optimized, not from changing the model architecture or claiming a new compression method.
The release also unifies the logic used to decide whether a run is full fine-tuning. Previously, the trainer and the VRAM pre-flight check used separate implementations that could disagree. In practical terms, the pre-flight estimate and the actual loading path now share the same decision about whether the base is trainable.
A 28.9 GB difference can change the hardware decision
Aipolix's main engineering conclusion is that training-stack defaults belong in capacity planning, not only in debugging. A team can choose a model size, quantization setting and LoRA configuration correctly and still get a misleading GPU requirement if the framework silently changes the effective dtype at load time.
In Soup's H100 test, the reported difference was 28.9 GB of peak memory. That is large enough to change whether a job fits on a particular accelerator, whether it needs a larger instance, and whether an engineer concludes that parameter-efficient tuning is economically viable on the intended hardware.
The operational lesson is broader than Soup. GPU budgeting should verify the effective dtype and trainability of the loaded base model, then compare measured peak memory with the planner's estimate. Configuration intent is not sufficient evidence when a framework or dependency can alter how weights are materialized.
The release also tightens failure and security boundaries
Version 0.74.0 is not only a memory fix. Soup also changes several controls around its serving and tooling surfaces.
The project says /v1/tools/bash is re-enabled under operating-system namespace and sandbox isolation. Because that endpoint executes code, soup serve now exits with status 2 when bound to a non-loopback host without --tool-auth-token; the previous behavior only emitted a warning. The release also closes four alternate-IPv4 notation bypasses in telemetry, webhook and OTLP endpoint validation, and makes the SGLang backend respect the --trust-remote-code gate instead of enabling remote model code automatically.
These are first-party release claims about shipped behavior. They are significant because they move unsafe conditions from warnings or implicit defaults toward explicit refusal or opt-in controls.
Compatibility improved, but one dependency floor is still wrong
Soup 0.74.0 adds support for Transformers 5.x, TRL 0.29 and PEFT 0.20, restores a previously unsatisfiable train,mlx extras combination, and allows Qwen3.5-family text decoders on the Transformers path. It also adds a plan-first Lambda cloud training command whose API key remains local while a local controller owns termination.
The same release notes disclose a concrete limitation: the declared torch>=2.5.0 floor does not actually work with trl>=0.29. The project measured failure on torch 2.5.1 because the required FSDPModule symbol is absent. A fresh install that resolves a newer Torch can be unaffected, but a pinned 2.5.x environment can lose DPO, KTO, GRPO and BCO support. Soup did not raise the floor because it had not yet measured the next version as working.
That disclosure is useful operational evidence. A version constraint that resolves successfully is not necessarily a tested compatibility guarantee.
Treat the 2.59x figure as a scoped measurement
The memory result is compelling, but its scope should remain narrow. It comes from the project's own H100 test using Llama-3.1-8B with LoRA. Soup reports three byte-identical repeats, but Aipolix did not find an independent reproduction of the v0.74.0 result.
That means teams should not convert 2.59x into a universal expectation for every model, accelerator, quantization mode or training path. The defensible claim is that Soup fixed an fp32 loading path and measured a large reduction in one documented configuration.
Layer streaming, another major Soup capability for training large frozen bases on small GPUs, also remains explicitly marked beta. The release does not change that status.
What teams should verify after upgrading
For teams already using Soup, the upgrade is worth validating with the workload that drove the original capacity estimate. Record the loaded dtype, peak allocated and reserved GPU memory, whether the base is frozen, the adapter configuration and the exact Soup, Torch, Transformers, TRL and PEFT versions.
The most useful comparison is not simply whether v0.74.0 uses less memory. It is whether measured runtime memory now matches the planning assumptions closely enough that instance sizing and cost estimates can be trusted.
Soup's release is a reminder that efficient fine-tuning can fail at a layer below the algorithm. A LoRA configuration may be parameter-efficient on paper while a framework default quietly materializes the frozen weights at an unnecessarily expensive precision. Fixing that boundary can be as consequential for deployment economics as changing the tuning method itself.