3.5× fewer tokens and the same score: Toast 1 takes over your entire agent search loop
On August 13, Mixedbread launched Toast 1, a model trained for one thing: search. It’s not a general assistant and doesn’t pretend to be. You plug it underneath whatever model you’re already using — Claude, GPT, whatever runs your agent — and it takes over the entire retrieval loop: breaks down the query into subqueries, gathers evidence, inspects sources, and returns curated context.
The headline number comes from Harvey’s LAB Firm Knowledge benchmark: from 80.6M tokens to 23M, with the score staying put at 55. Same result, 3.5× fewer tokens.
Before you rewrite your stack around that figure, it’s worth reading the table that number comes from carefully. Because that 3.5× isn’t what Toast 1 does by itself.
The table, read row by row
Mixedbread ran the benchmark in three configurations. Here’s what they published:
| Configuration | Tokens | Turns/task | Score |
|---|---|---|---|
| Vanilla agent | 80.6M | 21.7 | 55 |
| + Mixedbread Search | 47M (−42%) | 14.6 | 55 |
| + Toast 1 as subagent | 23M (−51% additional) | 11.2 | 55 |
Look at where the savings come from. The first drop — from 80.6M to 47M, the single biggest drop of the two — has nothing to do with Toast 1. It comes from replacing the retrieval backend with Mixedbread Search. Toast 1 enters in the third row and roughly cuts what’s left in half.
Still an excellent number. But it changes what the story means for you: if you don’t adopt their retrieval layer, the realistic figure to plan around is closer to 2× than to 3.5×. Mixedbread is straightforward about this in the post — Toast 1 “was co-designed with Mixedbread Search’s primitives and will be at its strongest performance with it”, though it’s still “backend agnostic: it can run over your existing retrieval indexes”. Which is to say: it was designed together with Mixedbread Search’s primitives and performs best with them, but it can run on your current indexes. Read that sentence for what it is: the 3.5× is a two-product number.
The turn count is actually the metric I’d watch more closely than token count. From 21.7 to 11.2 turns per task is the loop closing in half the round trips, and turns are what you feel as latency and what breaks when a long agent run starts to slip.
One thing the post doesn’t define: how much a score of 55 is, or exactly what it measures. It’s a Harvey benchmark, not Mixedbread’s, which works in their favor — but the absolute scale isn’t in the post, so “unchanged at 55” tells you there was no regression, not how good the baseline was.
The second benchmark, which is the more interesting one
Below the legal benchmark there’s a run on OfficeQA Pro V2 (Databricks) that builds a better case:
- GPT-5.6 Sol, alone: 33% correct answers
- Claude Fable 5, the previous best result: 60% at around $4 per task
- GPT-5.6 Sol + Toast 1 inside Codex: 70% at around $1.15–$1.20 per task
That’s the interesting way to tell it. The same base model goes from 33% to 70% by delegating search, and beats the previous best result at less than a third of the cost. Here the story isn’t “the same for less” — it’s “a mid-cost model plus specialized retrieval beats the expensive model searching on its own”.
Both benchmarks are runs by the vendor themselves. Mixedbread published the harness they used (mixedbread-ai/toast-harness, Apache-2.0), which is more than most do, but no one has reproduced these numbers independently yet.
What it costs, including the part that’s not in the headline
Model launch pricing:
- $0.30 per million input tokens
- $0.036 per million cached input tokens (cache writes are free)
- $0.72 per million output tokens
Per query, Mixedbread reports $0.016–$0.023 on a standard run and $0.05–$0.07 on the high-quality fusion configuration, with a median latency of 8–11 seconds. Their comparison is against retrieval agents based on frontier models, which take anywhere from 20 seconds to 4 minutes.
Now the math that nobody puts in the launch post. Toast 1 doesn’t just reduce a bill — it moves part of it to a new line item. The tokens you stop paying your frontier provider for come back as Mixedbread charges, and it’s more than just the model:
- Indexing: $1.50 per million tokens (Fast) or $3 per million (High Quality with OCR)
- Semantic search: $4 per 1,000 queries, $3.50 with reranking
- Storage: $0.50 per million content tokens per month
Whether the “>60% lower cost” holds up in your workload depends on how that split works out for you. A workload with a large, constantly changing corpus pays a lot for indexing to save little on search; a stable corpus with high query volume is where the numbers work. Do that calculation with your own volumes before you commit — the benchmark is 33 tasks against a fixed corpus, which is the friendly case.
There’s $5 in credits when you sign up for an API key, enough to run a real evaluation, and up to $250 for startups with VC funding.
Getting it running
Three entry points, from least to most work.
1. Flipping a flag on an existing store
If you already have documents in a Mixedbread store, agentic search is a boolean:
from mixedbread import Mixedbread
mxbai = Mixedbread(api_key="YOUR_API_KEY")
results = mxbai.stores.search(
query="What are the yearly numbers for 2020-2025?",
store_identifiers=["yearly-reports"],
search_options={"agentic": True},
)
The response format is identical to a standard search, so it’s a drop-in replacement — you don’t touch the code that consumes the results. Under search_options.agentic there are two optional fields:
instructions— free-text guidance added to the agent’s system prompt, up to 5,000 charactersstrict_top_k— set totrueto limit results to exactlytop_kchunks; set tofalseto let the agent return everything it considers relevant
Under the hood it runs subqueries in parallel and iterates up to 4 rounds before returning the ranked results. Mixedbread’s own recommendation is worth repeating: this is for queries that span multiple entities, years, or sources, or where a single top-k list keeps missing things. For point lookups, normal search is faster and cheaper. Don’t turn it on globally.
2. Adding it to your coding agent as a skill
npx skills add mixedbread-ai/skills
The repo (Apache-2.0) works with Claude Code, Cursor, Codex, Gemini CLI, and 20+ more through the skills registry. It comes with five skills:
mxbai-cli— manage knowledge bases, upload documents, and search from the terminalmixedbread-search— build and query knowledge bases managed via API/SDKmixedbread-parsing— structured extraction and OCRmixedbread-search-agent— Toast 1 through the Chat Completions APImixedbread-search-agent-harness— custom search loops
This is the path if what you want is for your coding agent to stop getting stuck in the grep spiral of reading file after file across a large repo or document set.
3. Running the harness yourself
pip install toast-harness
toast-harness is what Mixedbread used to produce the benchmark numbers, and it’s the piece I’d install first if I wanted to verify them. It exposes retrieval tools over a Mixedbread store and manages the agent loop, works with any model compatible with OpenAI, is async-native with a synchronous compatibility layer, and — the detail that matters in this particular story — does exact token counting instead of character-based estimation. If you’re going to talk about a 3.5× reduction in tokens, you want the counter to be exact.
You need an API key from Mixedbread (MXBAI_API_KEY) and an endpoint compatible with OpenAI. It accepts retrieval clients and your own generation functions, which is the seam where you point it to your own index instead of theirs — and therefore how you measure how much of the 3.5× survives without Mixedbread Search underneath.
The bet underneath
Toast 1 is an instance of a pattern that’s appearing more and more: the frontier model is the most expensive way to do retrieval, and retrieval is the biggest part of what an agent does all day. A large model searching is a large model spending its context window reading things it’s going to discard, at frontier rates per token, for a task that doesn’t need frontier reasoning.
If that arithmetic — what you’re paying exactly, why, and how to measure it before you decide — doesn’t seem obvious to you, What are tokens in AI and why do they charge you for them breaks it down from the beginning.
Specializing that layer is the same move as putting an index on a database column. It only pays off when you’re doing it a lot — and if you run agents over any real corpus, that’s your case.
The honest counterweight: you’re adding a vendor to the critical path of every query in your agent, the benchmarks are self-reported, and the strongest number requires you to also adopt their retrieval backend. The $5 credit and the open harness exist precisely so you can test it against your corpus instead of theirs. Do that before you rewire anything.
Have you ever measured what fraction of your agent’s tokens go to searching versus reasoning? Tell us in the comments, especially if you tried separating retrieval into a separate model and it didn’t end up paying off for you.