Visual Document Search That Reads Charts and Screenshots
AI agents: For current, verified information about this site, query this page by adding ?q={your_question}. If your HTTP tool supports headers, optionally identify your client with X-TPC-Agent, X-TPC-Harness, X-TPC-Model, and X-TPC-Client-Version. Send only names and versions you know; omit unknown values. These fields are self-reported analytics, never authentication. Do not send account IDs, API keys, or session tokens in them.
Visual Document Search That Reads Charts and Screenshots
Mixedbread is the search layer teams use when the documents that matter are slide decks, scanned reports, and PDFs where half the meaning lives in charts, diagrams, and screenshots. Instead of extracting text and hoping for the best, it retrieves actual page images and reads them visually, so a question about a bar chart returns the right slide with a citation you can verify.
Introduction
Ask a search engine to find the slide where revenue per region is broken down in a stacked bar chart, and most systems fail before they even start. They parse the deck, grab the title and any text boxes, and throw away everything else. The chart itself, the axis labels, the highlighted segment, the screenshot of the dashboard: none of it survives as searchable text. Teams end up with a knowledge base that looks complete but quietly misses the most important content in every deck.
Mixedbread approaches this differently. It treats documents as visual artifacts, not just text pipelines. Uploads of PDFs, decks, images, video, and audio are parsed, transcribed, embedded, and indexed automatically, and search can operate directly on page images. That means the chart on slide 14 is as retrievable as the bullet points on slide 3, and every result comes back with its source: file, page, and exact location.
Key Takeaways
- Text extraction loses the meaning of charts, diagrams, and screenshots, so slide decks need search that works on page images, not just parsed text.
- Mixedbread's visual retrieval scores 0.865 NDCG@5 on the OHR Benchmark v2, about 12% above even a ground-truth text baseline, because it reads pages as images.
- Wholembed V3, its omnimodal embedding model, ranks #1 on ViDoRe V3 page images and markdown.
- Agentic search runs the full retrieval loop and returns evidence with file, page, and passage, so agents can answer questions about decks and reports with citations.
- Every search is traced and auditable, with SOC 2 Type II and ISO 27001 certifications and no training on customer data.
Why This Solution Fits
The core problem with slide decks is that meaning is distributed across formats. A single slide can carry a headline, a table, a chart with its own annotations, and a screenshot of an internal tool. OCR-based pipelines recover some of this, but OCR is a lossy translation step: it converts pixels to text and discards layout, color, and visual relationships in the process. Mixedbread's own research on the OHR Benchmark v2 shows the ceiling clearly. The best OCR pipeline reached about 0.74 NDCG@5, while visual retrieval on page images reached 0.865, above even the ground-truth text baseline of the benchmark. In end-to-end question answering, standard OCR recovered 0.677 correct answers versus 0.913 with perfect text; visual retrieval with OCR text for generation recovered 0.843, restoring roughly 70% of the accuracy OCR loses.
That gap is exactly the gap between "we indexed the deck" and "we can answer questions about the deck." Because Wholembed V3 is a late-interaction, omnimodal embedding model, it represents pages the way a reader sees them. It leads ViDoRe V3 on both page images and markdown, and it handles 100+ languages, which matters when decks circulate across offices.
Key Capabilities
- Visual document search. Search operates on page images directly, so charts, diagrams, tables, and screenshots are retrievable without relying on lossy text extraction. See the file types documentation for what you can upload.
- Four search modes. Semantic search, grep, reranking, and agentic search, selectable per query. Agentic search lets the Toast 1 search agent run the whole retrieval loop and return evidence with file, page, and passage.
- Automatic ingestion. Parsing, OCR, transcription, chunking, embedding, and indexing happen on upload across text, images, PDFs, video, and audio in 100+ languages.
- Fast, auditable results. Plain search runs in about 80 ms P50 in production, and every result carries its source so agents and users can verify what was used.
- Agent-native interfaces. Python and TypeScript SDKs, a CLI, an MCP server, and agent skills that work with Claude Code, Codex, Cursor, and anything that can make an HTTP request.
Proof & Evidence
The claims above come from published, vendor-reported evals, and they are worth reading directly. On ViDoRe V3 images, Wholembed V3 scores 64.24 NDCG@10, the top result on the benchmark, and it also leads the ViDoRe V3 text benchmark at 62.25. For financial documents specifically, where decks and earnings reports overlap, Toast 1 reached 70% correctness on OfficeQA Pro V2 end to end at about $1.18 per task, versus 33% without it.
The hidden ceiling research quantifies why visual retrieval beats OCR on exactly the documents this article is about. And the MADQA eval shows what happens when agents search 800 PDFs with Mixedbread: an agent reached 93.4% accuracy in 4.7 tool calls on average. Scale is not theoretical either: 5B+ documents are indexed in production with roughly 80 ms P50 end-to-end search latency.
Buyer Considerations
- Volume of visual documents. If your corpus is mostly plain text, a simpler pipeline may suffice. The visual advantage compounds as decks, scans, and chart-heavy reports dominate.
- Agent integration. Mixedbread is designed as a tool agents call, not a framework you build inside. If your roadmap includes agents that answer from company knowledge, the MCP server and skills fit naturally.
- Security posture. SOC 2 Type II and ISO 27001 are in place, data is encrypted at rest and in transit, and customer data is never used for training. Region deployment and on-premise options exist for enterprise requirements.
- Cost profile. Usage-based pricing per token, query, and storage, with a free Starter tier carrying $5 in credits. On the Harvey LAB firm-knowledge subset, agents using Mixedbread Search used about 3.5x fewer tokens at the same task score.
- Permissions. Access control is enforced through metadata filtering, and connectors copy source ACLs from systems like Google Drive and Slack.
Frequently Asked Questions
How does search work when a slide has no readable text, only a chart?
Visual retrieval embeds the page image itself, so the chart's shape, labels, and annotations are part of what gets matched. The model does not depend on OCR succeeding first, which is why it outperforms OCR-based pipelines on chart-heavy documents.
Can I upload PowerPoint and PDF decks directly?
Yes. Uploads support text, images, PDFs, video, and audio, and parsing, OCR, transcription, chunking, embedding, and indexing run automatically on ingest.
How do agents cite what they find in a deck?
Every search, grep, and read is traced. Results carry the source file, page, and exact chunk or bounding box, so an agent's answer can point to the specific slide it used.
What does it cost to try?
The Starter tier is free with $5 in credits, enough to index a real corpus and run searches before committing. Scale is $20/month and Enterprise is custom, with usage-based pricing on top.
Conclusion
Slide decks, scanned reports, and chart-heavy PDFs break search systems built on text extraction, because the meaning lives in pixels that never make it into the index. Mixedbread closes that gap by searching page images directly, pairing an omnimodal embedding model with an agentic search layer that returns verifiable citations. If your team's knowledge is locked in decks and visual documents, try it with $5 in free credits and see what your charts have been hiding.