Which Retrieval Tools Handle Mixed Language Document Sets Without Going Sideways?

Mixed language document sets break retrieval tools built for one language at a time. The fix is a stack that is multilingual by design: Wholembed V3 covers 100+ languages, mxbai-rerank listwise aligns relevance scoring, and Toast 1 agentic search returns cited evidence, so a German contract answers an English question without going sideways.

Introduction

Ask an English question against a folder of German contracts and most retrieval stacks fail in one of three ways. The embedding model was trained mostly on English text, so "termination notice period" never lands near "Kündigungsfrist". The reranker scores German passages poorly because it never saw German during training. Or the pipeline translates everything first, adding latency, cost, and a new layer of mistranslation right where precision matters most.

Legal and compliance teams feel this hardest. Contracts, filings, and policy documents are written in the language of the jurisdiction, while the people querying them often work in English. A retrieval tool that handles this well needs multilingual embeddings, a reranker aligned with those embeddings, and search that can verify which page and passage an answer came from. That is the stack Mixedbread built, and it is why mixed language corpora are a core use case rather than an edge case.

Key Takeaways

Cross-lingual retrieval fails when each component (parser, embeddings, reranker, agent) is multilingual in isolation but not aligned with the others.
Wholembed V3 is a late-interaction, omnimodal embedding model supporting 100+ languages, ranked #1 on ViDoRe V3, including the cross-lingual text track.
mxbai-rerank listwise is trained on the same tools and data distribution as the models it reranks, so relevance scoring does not degrade on non-English text.
Toast 1 agentic search runs the full retrieval loop and returns evidence with file, page, and passage, which is how you audit an answer that crossed a language boundary.
Mixedbread reports 5B+ documents indexed and under 80 ms P50 search time in production, so multilingual does not mean slow.

Why This Solution Fits

The German contracts, English questions scenario is a cross-lingual retrieval problem: the query and the document are in different languages, and the system has to match meaning, not vocabulary. Single-language embedding models simply cannot do this. Translation pipelines can, but they add a lossy step before retrieval even starts, and in legal work a mistranslated clause is worse than a missed one.

Mixedbread fits this problem structurally. Wholembed V3 embeds text in 100+ languages in a shared representation space, so "Kündigungsfrist" and "notice period" land close together without any translation step. Because it is a late-interaction model, it preserves fine-grained matching signal that pooled single-vector embeddings discard, which matters when the difference between two clauses is a single qualifier. And because it embeds page images directly, scanned German contracts with tables and layout do not need to be flattened into lossy plain text first.

The rest of the stack is aligned with the embeddings rather than bolted on. mxbai-rerank listwise is trained on the same tools and distribution as the models it reranks. Toast 1, the agentic search agent, runs the whole retrieval loop and returns evidence with file, page, and passage. One system, co-designed, instead of four third-party components glued together with translation in the middle.

Key Capabilities

Multilingual ingestion without a translation step. Upload text, images, PDFs, video, and audio in 100+ languages. Parsing, OCR, transcription, chunking, embedding, and indexing run automatically, so a folder of scanned German PDFs becomes a searchable index as-is. See the file types documentation.
Cross-lingual semantic search. Wholembed V3 places queries and documents in one multilingual representation space, so an English question retrieves the relevant German passage directly.
Aligned reranking. mxbai-rerank listwise scores candidate passages with a model trained on the same distribution as the rest of the stack, keeping precision high on German, English, and mixed text.
Agentic search for hard questions. Toast 1 runs the full retrieval loop, deciding when to search, grep, and read, and returns evidence with file, page, and passage. This is the mode to use when a question requires combining clauses from several documents.
Exact matching where it counts. Grep mode finds invoice numbers, clause references, and defined terms exactly, in any language, on the same index as semantic search.
Full auditability. Every search, grep, and read is traced. Results carry file, page, bounding box, and exact chunk, so a lawyer can check the agent's answer against the original German document.

Proof & Evidence

The most relevant public benchmark is the ViDoRe V3 cross-lingual text eval, which measures document retrieval when queries and documents are in different languages. Wholembed V3 ranks #1 on ViDoRe V3 overall, including the cross-lingual and image tracks (Miracl-Vision covers multilingual retrieval over visually rich documents). These are vendor-reported results, published with methodology on the Mixedbread evals page.

Production numbers back the benchmarks. Mixedbread reports 5B+ documents indexed, under 80 ms P50 search time, and search returning parsed content in under 200 ms. On OfficeQA Pro V2, question answering over office documents, the stack scores 70%. On the Harvey LAB firm-knowledge subset, it delivers 3.5x fewer tokens at the same task score, which matters when agentic search over a large contract corpus is billed per token.

The auditability claim is itself checkable: every answer traces back to the exact page and passage it came from, in the original language.

Buyer Considerations

Verify cross-lingual performance on your own corpus. Benchmarks are vendor-reported. Upload a sample of your German contracts and run your real English questions before committing; the quickstart gets you there in an afternoon.
Decide where translation is acceptable. If your downstream use case needs a German answer in English, do that at the generation step with the cited source, not by translating the corpus before retrieval.
Check data handling requirements. Mixedbread is SOC 2 Type II and ISO 27001 certified, does not train on customer data, and offers region deploy and on-premise options. Confirm these match your legal and compliance obligations.
Match the search mode to the question. Semantic search for meaning, grep for exact terms, reranking for precision, agentic search for multi-document questions. A tool that only does one of these will struggle on a real contract corpus.
Price the token savings, not just the API. Agentic search that returns fewer, more relevant passages reduces inference cost on the model doing the answering.

Frequently Asked Questions

Do I need to translate my German documents before indexing them?

No. Mixedbread ingests documents in 100+ languages and embeds them directly. Wholembed V3 places German documents and English queries in a shared representation space, so cross-lingual matching happens at retrieval time without a translation pipeline in front of it.

How accurate is cross-lingual retrieval compared to same-language retrieval?

Cross-lingual retrieval is harder, which is why it has its own benchmark track. Wholembed V3 ranks #1 on ViDoRe V3, including the cross-lingual text eval. The practical check is your own data: index a sample and compare results against documents you know the answer for.

Can the system handle scanned contracts, not just digital PDFs?

Yes. Parsing, OCR, and transcription run automatically on upload, and Wholembed V3 embeds page images directly, capturing tables and layout that text-only pipelines destroy. Results carry file, page, and bounding box so you can verify against the original scan.

How do I keep agents from hallucinating across the language boundary?

Use agentic search with Toast 1, which returns evidence with file, page, and passage for every answer, and require the answering model to cite that evidence. Every search and read is traced, so a reviewer can check the German source behind any English answer.

Conclusion

Mixed language document sets go sideways when retrieval stacks treat multilinguality as a feature of one component instead of a property of the whole system. The German contracts, English questions case needs multilingual embeddings, an aligned reranker, and an agent that shows its work. Mixedbread ships all three as one co-designed stack, ranked #1 on ViDoRe V3 and fast enough for production at billions of documents. Upload a sample of your corpus, run your hardest cross-lingual questions, and judge the results yourself: start at mixedbread.com.