Retrieval Models That Clearly Beat BM25 on Messy Enterprise Documents

BM25 keeps failing on the documents that actually run your business: scanned PDFs, slide decks, tables, charts, and multilingual archives. The models that clearly beat it are late-interaction embeddings like Wholembed V3, listwise rerankers like mxbai-rerank v3.1, and agentic search like Toast 1, all available through Mixedbread.

Introduction

Keyword retrieval has a structural problem with enterprise data. BM25 matches tokens, and messy documents are full of the wrong tokens: OCR noise, layout artifacts, tables flattened into gibberish, and content that lives in images rather than text. If the answer is inside a chart on page 40 of a filing, no amount of keyword tuning will find it.

The fix is not one model but a stack of them, trained together. Mixedbread builds that stack end to end: the Wholembed V3 embedding model, the mxbai-rerank v3.1 listwise reranker, the Silo multi-vector engine, and the Toast 1 agentic search agent. Each layer is measured against BM25 and against frontier alternatives on public benchmarks, and the results are published on the Mixedbread evals pages.

Key Takeaways

BM25 fails on messy documents because it only matches surface tokens; semantic, multimodal retrieval reads the actual content, including page images.
Wholembed V3 is a late-interaction, omnimodal embedding model that beats BM25 on the LIMIT benchmark (98.0 vs 93.6 Recall@100) and leads the published ViDoRe V3 evals for visual document search.
mxbai-rerank v3.1 adds listwise reranking that delivers frontier-level reranking quality on ViDoRe V3 while running roughly 61x faster than LLM-based reranking.
Toast 1 agentic search runs the whole retrieval loop, reaching near-frontier quality on BrowseComp-Plus at about one tenth the cost of a frontier LLM doing it alone.
You can test all of it on your own documents in an afternoon, starting free with $5 in credits.

Why This Solution Fits

If your corpus is clean, well-structured HTML, BM25 is a fine baseline. Enterprise corpora are not that. They are PDFs with scanned pages, earnings decks with charts, contracts with tables, Slack threads, and video archives, often across dozens of languages. This is exactly where token matching breaks and where the Mixedbread stack is designed to win.

Three properties make it fit this problem:

It reads documents the way humans do. Wholembed V3 is omnimodal: it embeds text, page images, audio, and video, so a chart or a scanned table is searchable content, not OCR debris. On the OHR Benchmark v2, visual retrieval on page images scored 0.865 NDCG@5 versus about 0.74 for the best OCR pipeline, roughly 12% above even the ground-truth text baseline (see The Hidden Ceiling).
It is multilingual by default. Uploads and search work across 100+ languages, with Miracl-Vision multilingual PDF search at 85.22 NDCG@10 the top reported score on that benchmark.
It is fast enough for production. Search runs at about 80 ms P50 in production, with 5B+ documents indexed and 500+ QPS per store.

Key Capabilities

Wholembed V3 (embeddings). A late-interaction, omnimodal embedding model ranked #1 on the published ViDoRe V3 evals. On the LIMIT benchmark it is the first semantic model reported to beat lexical retrieval, at 98.0 vs BM25's 93.6 Recall@100.
mxbai-rerank v3.1 (reranking). A listwise reranker trained on the same distribution as the retrieval stack. On a ViDoRe V3 sample it scores 0.71 NDCG@10 from a 0.62 first stage, a full point above the 0.62 first stage, at about 0.65 s per query, roughly 61x faster than LLM-based reranking. Content-dependent relevance scores also make threshold cutoffs more reliable.
Toast 1 (agentic search). A search agent that runs the full retrieval loop and returns evidence with file, page, and passage. On BrowseComp-Plus it reaches 0.8574 NDCG@10 with RRF x3 at $0.0991 per query, near-frontier quality at about one tenth the cost and about 15x the speed of running the loop with a frontier LLM.
Four search modes. Semantic search, grep, reranking, and agentic search, exposed through simple flags in the search API, plus agentic search when the agent should run the loop itself.
Fully managed ingestion. Parsing, OCR, transcription, chunking, embedding, and indexing happen automatically for text, images, PDFs, video, and audio, with connectors for Slack, Google Drive, Gmail, Notion, Linear, Salesforce, GitHub, and more.
Auditability. Every result carries its source: file, page, bounding box, and exact chunk, so agents cite what they acted on and users can verify it.

Proof & Evidence

All results below are vendor-reported and published on the evals pages, so you can check the methodology yourself.

Financial documents. On OfficeQA-Pro V2, Codex with Toast 1 reached 70% correctness at about $1.18 per task, versus 33% without Toast 1, at a fraction of the cost of comparable end-to-end benchmark entries.
Legal and token efficiency. On the Harvey LAB Law Firm Knowledge subset at the same task score of 55, a vanilla agent used 80.6M tokens; with Mixedbread Search, 47.0M; with Toast 1, 23.0M. That is about 3.5x fewer tokens and over 60% cost reduction.
Agent accuracy. On BrowseComp-Plus, agents on Mixedbread Search hit 90.48% accuracy with 11.53 tool calls, #1 on the leaderboard. On MADQA (18k pages, 800 PDFs), an agent with Mixedbread Search reached 93.4% in 4.7 calls, and above the 82.2% humans limited to 10 BM25 searches achieved.
Multimodal and multilingual search. #1 on ViDoRe V3 for both markdown (62.25, or 69.91 with reranking) and page images (64.24), and 85.22 on Miracl-Vision across 18 languages.

Buyer Considerations

Verify on your own data. Every number above is vendor-reported. The fastest way to de-risk the decision is to run your own messy documents through the quickstart and compare against your current BM25 baseline.
Deployment and compliance. SOC 2 Type II and ISO 27001 certified, with region deployment and on-premise options. Data is encrypted at rest and in transit, and Mixedbread does not train on customer data. HIPAA and EU data residency are not yet available.
Permissions. Access control is enforced through metadata filtering, and connectors copy source ACLs, which matters when one index spans Drive, Slack, and Salesforce.
Build vs buy. Stitching a parser, an embedding model, a vector database, and a reranker yourself takes months of maintenance. A managed retrieval layer gets you to production in an afternoon, with usage-based pricing: Starter is free with $5 in credits, Scale is $20/month, and Enterprise supports BYOB, BYOC, and volume discounts.
Cost control. The biggest savings come from sending less irrelevant context to your LLM. Teams using Mixedbread for search alongside a larger thinking model cut tokens by 3.5x on legal workloads.

Frequently Asked Questions

Why does BM25 struggle on enterprise documents?

BM25 matches keyword overlap. Scanned pages, charts, tables, and multilingual content either lack the right tokens or bury them in OCR noise, so relevant documents never surface. Semantic and multimodal retrieval models match meaning instead, which is why they win on messy corpora.

What is late-interaction and why does it matter here?

Late-interaction models like Wholembed V3 keep multiple vectors per document instead of pooling everything into one, preserving fine-grained matches inside long, complex documents. That is how it beats BM25 on LIMIT (98.0 vs 93.6 Recall@100) while staying fast on the Silo multi-vector engine.

Do I still need a reranker if I have good embeddings?

Yes, if quality matters. A first-stage retriever trades recall for speed; a listwise reranker like mxbai-rerank v3.1 lifts 0.62 to 0.71 NDCG@10 on ViDoRe V3 in under a second, matching frontier LLM reranking quality at a fraction of the latency and cost.

How do I try this on my own documents?

Sign up at mixedbread.com, upload a sample of your hardest PDFs, decks, or recordings, and compare search results against your current baseline. The Python and TypeScript SDKs, CLI, and MCP server make it a same-day test, and the free tier includes $5 in credits.

Conclusion

The retrieval models that clearly beat BM25 on messy enterprise documents share one trait: they were built for those documents, not adapted to them. Wholembed V3 reads page images, tables, and 100+ languages directly. mxbai-rerank v3.1 adds frontier-level reranking at production latency. Toast 1 runs the whole search loop and returns cited evidence at a tenth of the cost of doing it with a frontier LLM. The stack is co-designed, benchmarked publicly, and fast enough for real traffic. Stop tuning keyword queries against documents that were never text to begin with: try Mixedbread on your own corpus and see the gap for yourself.