LightOn, the French AI company known for its open retrieval models, published two new sparse retrievers on Monday, October 19, 2026. SlideOn is an English model with 174 million parameters, and mSlideOn is a 339 million parameter multilingual model that also handles long documents and source code. The company released the fine-tuned models, the sparse autoencoder checkpoints used to build them and the training code under what it describes as a permissive license, with the models on Hugging Face and the code on GitHub.
The release targets a practical problem in search and retrieval-augmented generation. Dense embedding models compress each document into one vector and usually need GPU-accelerated vector search to stay fast at scale, while late-interaction models such as LightOn's own LateOn keep many vectors per document and cost more to store and query. Sparse retrieval represents queries and documents as vectors in which almost every value is zero, so the results can be stored in an inverted index, the same structure classic keyword search engines use. Search then only touches documents that share active features with the query. LightOn cites earlier research showing that such indexes can run on CPUs with latency similar to GPU-backed dense search.
Rather than training from scratch, LightOn converted retrievers it had already built. SlideOn starts from its DenseOn dense model and mSlideOn from its multilingual late-interaction model mLateOn. The team first trained a Top-K sparse autoencoder on each model's token embeddings, with 32,768 learned features and only the top 32 kept per token, then fine-tuned the backbone and sparse encoder for retrieval. Token vectors are max-pooled into a single sparse vector per query or document. Instead of SPLADE-style models that weight words from a tokenizer vocabulary, the features are learned concepts, which LightOn says can capture shared meaning across phrasings and languages.
Training used about 16.3 million query and positive-document pairs across nine natural languages plus code, each with 10 mined hard negatives scored by the mxbai-rerank-large-v2 cross-encoder as a teacher. A Matryoshka-style objective concentrates information in the strongest features, so users can prune query and document vectors at inference time to trade a little accuracy for lower search cost and smaller indexes without retraining.
On the English BEIR-15 benchmark, LightOn reports that SlideOn scores 56.68 average nDCG@10, slightly above DenseOn at 56.20 and about half a point behind LateOn. It scores 2.2 points higher than SPARSEUP, a recent sparse retriever, although SPARSEUP needs fewer retrieval operations by default. With documents pruned to their top 512 features, LightOn says SlideOn sits on the quality-versus-search-cost and quality-versus-storage frontiers among the sub-2B models it compared.
The multilingual model shows bigger gains over its dense sibling. On MIRACL, mSlideOn reaches 62.78 across all 18 languages, 4.76 points above mDenseOn but below mLateOn at 67.04. On the long-document MLDR benchmark it reaches 61.68 across 13 languages, roughly 10 points above mDenseOn, though still well behind mLateOn at 77.92. On MTEB Code it scores 74.78, ahead of mLateOn at 73.48 and every other sparse and late-interaction model in LightOn's comparison. LightOn notes that native MIRACL supervision covered only Arabic, English, French and Spanish, which it says helps explain why rivals trained on 16 languages remain stronger.
For teams building search or RAG systems, the appeal is a smaller model that can serve from standard inverted-index infrastructure while keeping semantic matching. The caveats are clear. All benchmark numbers come from LightOn's own evaluation and have not been reproduced independently, the multilingual model still trails late interaction on most tasks, and real-world speed will depend on the sparse index engine used. LightOn says future work will extend supervision to more languages and scripts and better preserve late-interaction matching during sparsification.