Hosted Tools

Hosted tools are server-executed store tools. Declare one and the server runs the whole search loop: Toast 1 searches your Stores, reads the results, searches again if it needs to, and answers in plain text. You send one request and receive the final answer; no tool calls come back for you to execute.

The tools behave the same on the Responses API and Chat Completions. This page documents the tools, their scope, the context window, and how a run ends, and shows each request in both shapes. To run the tool loop yourself instead, see Build Your Own Harness.

Declare a hosted tool

Add a hosted tool type to tools. Everything else about the request stays as documented for the endpoint you call.

response = client.responses.create(
    model="toast-1",
    input="Which suppliers had recalls in 2023?",
    tools=[{"type": "search_corpus", "store_identifiers": ["my-store"]}],
)

print(response.output_text)

The hosted tools are opt-in: a request that declares none of them is a single model turn, exactly as sent. When they run, every server-executed call is recorded in hosted_tool_calls, beside output on the Responses API and beside choices on Chat Completions: one item per call with its type (for example search_corpus_call), id, status (in_progress, completed, or failed), and the call's arguments echoed as fields named for the tool (queries for a search, pattern for a grep). The chunk results of a call are stored but not returned by default; request them with include, e.g. include=["search_corpus_call.results"].

Function tools and hosted tools combine freely: your functions still come back as tool calls for your application to execute, while the hosted calls run server-side in the same loop. A function tool that duplicates a hosted tool's name is rejected with a duplicate_tool_name validation error.

tool_choice can force a hosted tool by its type. With hosted tools it applies to the first model turn; later turns of the server loop use auto.

Store scope

Every store-scoped tool takes store_identifiers, the IDs or names of the stores it runs against, and all declared store tools must share one scope; mixed scopes are rejected with a validation error. An API key restricted to specific stores must cover the scope.

To let the model pick the store itself, omit store_identifiers and declare list_stores alongside (required; the request is rejected without it):

tools=[{"type": "search_corpus"}, {"type": "list_stores"}]

Each store-scoped tool then takes a required store argument: the model discovers your stores with list_stores and names exactly one store on every call. get_chunks never takes one; its chunk IDs already identify where the run saw them.

Instructions

Your instructions (or system/developer messages) are the model's system prompt, passed through as written. A request that declares a hosted tool but sends no instruction text gets this default system message:

You are a search agent over the user's connected stores. Use the search tools you were given to explore the corpus regarding the user's query.

The default is strictly a fallback: any instruction text you send replaces it entirely, and requests without hosted tools never receive it.

The hosted tools

The hosted tools share their names, schemas, and behavior with the open-source toast-harness repository, the reference for exact schemas, defaults, and truncation behavior. Tool calls are budgeted together by max_tool_calls, and at most 8 server-executed calls run per model turn (extra calls receive a structured error result instead of running).

search_corpus and grep results are deduplicated across the run: chunks the model has already seen come back as compact references, in deduped_existing_or_deleted for a search and as seen: true entries for a grep.

Chunk references

Every chunk reference in hosted payloads (result entries and the IDs get_chunks accepts) has one uniform shape, chunk_id = "<file_id>:<chunk_index>", stable across requests. The <file_id> prefix resolves directly against the store files API, so your application can fetch the underlying file or chunk with one exact lookup. Each chunk also carries a document_id shared by all chunks of the same document, so results can be grouped by it.

Manage the context window

Toast 1 is served with a 131,072-token context window. A long hosted run can approach it; opt-in context editing gives the model room to keep exploring:

response = client.responses.create(
    model="toast-1",
    input="Compare the recall policies of all our suppliers.",
    tools=[{"type": "search_corpus", "store_identifiers": ["my-store"]}],
    context_management={"edits": [{"type": "prune_context"}]},
)

With it declared, the model gets a prune_context tool and calls it to clear tool results it considers stale as the conversation approaches the window. Every tool result it has read, server or client, can be pruned. context_management is accepted with any tool set, including pure function-tool requests, and is a no-op until something prunable accumulates.

What was edited is reported on the response, and this is the only trace of it you see:

• context_management.applied_edits carries one prune_context entry aggregating the model's prune calls and the input tokens they cleared, and one truncate_tool_result entry per client tool result the server had to shorten in overflow recovery (with its tool_call_id).
• The internal result ids and renderings the model prunes by are never returned; your conversation, transcripts, and continuations keep every tool result exactly as sent.
• Your function results are never capped by the server outside of overflow recovery, and pruning never rewrites what is stored, only what the model sees.

Prune calls count against max_tool_calls like every server-handled call.

How a hosted run ends

The loop ends when the model answers in plain text or calls one of your function tools; those calls always come back to you, the search limit included. Two bounds apply to server-executed calls: max_tool_calls (default 16) and the context window. When either is reached, the model receives this message once:

You have reached the search limit. Do NOT search further. You must now reply with your final answer to the user query as plain text, with NO tool calls. Base it only on retrieved evidence; if the evidence is insufficient to answer, say so.

A plain-text reply completes the response normally with whatever evidence it gathered; a call to one of your function tools is returned to you as usual. If the model does neither, the run ends as it stands: on the Responses API, status: "incomplete" with incomplete_details.reason set to "max_tool_calls" or "context_window"; on Chat Completions, finish_reason: "length" on the last choice. Either way the response carries whatever text the model produced. The API never writes an answer for the model.

A context_window ending means the run filled the window before it finished exploring. If your runs hit it, declare context_management alongside your search tools to give the model room to prune.

Streaming

Hosted runs stream their progress alongside the endpoint's standard events. When a server-executed call is dispatched, an event carries the call item with status: "in_progress"; when its round returns, another carries the finished item.

On the Responses API these are response.output_item.added and response.output_item.done events. The items are a Mixedbread extension: they claim an output_index in the shared item order but appear in hosted_tool_calls, not output, in the terminal payload. On Chat Completions, each call item arrives on a chunk once as in_progress and once finished, and context_management rides on the final usage chunk.

Non-streaming and streaming report identical final content; the hosted call items and their include behavior are the same on both surfaces.