Responses API

The Mixedbread Responses API is an OpenAI-compatible Responses endpoint for Mixedbread specialized models and agentic workflows. It provides a simpler input and output model than Chat Completions, supports stored multi-turn conversations, and works with client-executed function tools. It is also the primary surface for the hosted store tools: declare them in tools and the server runs the whole search loop for you.

You can use the OpenAI SDK by changing base_url and api_key, then selecting a model from the supported models. For message-based integrations, see the Chat Completions API; to run the tool loop yourself, see Build Your Own Harness.

Prerequisite

Get a Mixedbread API key from the API Keys page. A scope-restricted key needs the Completions scope for these endpoints; see API keys. You can install the OpenAI SDK or use cURL directly.

pip install openai

Make your first request

Point base_url to Mixedbread, use your Mixedbread API key, and select one supported model. Pass a string or a list of input items to input.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.mixedbread.com/v1",
    api_key="YOUR_API_KEY",
)

response = client.responses.create(
    model="toast-1",
    input="What is a search agent?",
)

print(response.output_text)

The Python and TypeScript SDKs expose the generated text through response.output_text. On the wire, generated messages and function calls are separate items in output.

OpenAI compatibility

Mixedbread accepts the OpenAI Responses shape for supported fields. The tables below cover important request and response behavior.

Request fields

Response fields

Mixedbread currently supports text input and output, client-executed function tools, hosted store tools, remote MCP servers, opt-in context editing via context_management, and streaming. Background responses, structured text formats, multimodal input, and OpenAI service connectors (connector_id) are not supported. Unsupported options return a validation error instead of being ignored.

See the advertised request and response fields in the API reference.

Continue a conversation

Responses are stored by default. Pass the previous response ID with only the new input to continue the conversation. Replace resp_123 with the id returned by the previous request.

response = client.responses.create(
    model="toast-1",
    input="When should I use one?",
    previous_response_id="resp_123",
)

print(response.output_text)

Set store=False when you do not need retrieval or continuation. Stored responses can be retrieved with client.responses.retrieve(response.id), and their input items can be listed with client.responses.input_items.list(response.id).

Tool use

Toast 1 takes three kinds of tools in tools, and they combine freely in one request: functions your application executes, hosted tools the server runs against your Stores, and the tools of a remote MCP server. tool_choice applies across all of them, and max_tool_calls budgets every server-executed call, hosted and MCP alike.

Your own tools

Function tools let Toast 1 request data or actions from your application. The model returns a function_call output item with JSON-encoded arguments. Execute the function in your application, then send a function_call_output item with the same call_id to receive the final answer.

import json

from openai import OpenAI


client = OpenAI(
    base_url="https://api.mixedbread.com/v1",
    api_key="YOUR_API_KEY",
)


def search_documents(query: str) -> list[dict[str, str]]:
    return [
        {
            "title": "Search agents",
            "text": f'Search agents plan queries like "{query}", inspect evidence, and synthesize an answer.',
        }
    ]


tools = [
    {
        "type": "function",
        "name": "search_documents",
        "description": "Search the application's documents for relevant passages.",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "The search query.",
                }
            },
            "required": ["query"],
        },
        "strict": False,
    }
]

response = client.responses.create(
    model="toast-1",
    input="What is a search agent? Use search_documents.",
    tools=tools,
    tool_choice={"type": "function", "name": "search_documents"},
    parallel_tool_calls=False,
)

tool_call = next(item for item in response.output if item.type == "function_call")
result = search_documents(**json.loads(tool_call.arguments))

final_response = client.responses.create(
    model="toast-1",
    previous_response_id=response.id,
    input=[
        {
            "type": "function_call_output",
            "call_id": tool_call.call_id,
            "output": json.dumps(result),
        }
    ],
    tools=tools,
    tool_choice="none",
)

print(final_response.output_text)

The example forces one call for a deterministic first turn, executes the application-owned search function, and sets tool_choice="none" on the second turn so Toast 1 produces the answer from the returned results. In production, the function can call any search engine, database, or internal API.

Hosted tools

Declare a hosted store tool and the server runs the whole search loop: Toast 1 searches your Stores, reads the results, searches again if it needs to, and answers in plain text. You send one request and receive the final answer; no tool calls come back for you to execute.

response = client.responses.create(
    model="toast-1",
    input="Which suppliers had recalls in 2023?",
    tools=[{"type": "search_corpus", "store_identifiers": ["my-store"]}],
)

print(response.output_text)

The answer arrives as message output items; the calls the server ran are recorded in hosted_tool_calls beside output. The Hosted Tools page documents the tools, the store scope, context_management, and how a run ends.

MCP servers

The mcp tool gives Toast 1 the tools of a remote Model Context Protocol server over its Streamable HTTP transport. Declare the server with a server_label and an HTTPS server_url; pass headers or an OAuth authorization token when the server needs them. Mixedbread lists the server's tools when the response starts, offers them to the model, and runs the calls the model makes.

response = client.responses.create(
    model="toast-1",
    input="What does the mixedbread-ai/mixedbread repo do?",
    tools=[
        {
            "type": "mcp",
            "server_label": "deepwiki",
            "server_url": "https://mcp.deepwiki.com/mcp",
            "require_approval": "never",
        }
    ],
)

print(response.output_text)

The response records the MCP work as output items: one mcp_list_tools item per server, then an mcp_call item for each call with its arguments, output, and error. Streaming emits the response.mcp_list_tools.* and response.mcp_call.* events around them. Use allowed_tools to offer only some of the server's tools, by name or with {"read_only": true}.

Mixedbread speaks the MCP Streamable HTTP transport and the initialize, tools/list and tools/call methods. The legacy HTTP+SSE transport, MCP resources, prompts and elicitation, and OpenAI service connectors (connector_id) are not supported.

Approve calls

As in the OpenAI API, require_approval defaults to always: instead of running a call, the response ends with an mcp_approval_request item carrying the tool name and arguments. Answer it on the next request with an mcp_approval_response item. An approved call runs before the model continues; a rejected one is reported to the model with your reason.

# The first response ends with an approval request instead of running the call.
approval = next(item for item in response.output if item.type == "mcp_approval_request")
print(approval.name, approval.arguments)

# Approve (or reject) it on the next request; the call runs before the model continues.
response = client.responses.create(
    model="toast-1",
    previous_response_id=response.id,
    input=[{"type": "mcp_approval_response", "approval_request_id": approval.id, "approve": True}],
    tools=[{"type": "mcp", "server_label": "deepwiki", "server_url": "https://mcp.deepwiki.com/mcp"}],
)

print(response.output_text)

Set require_approval to never, or to a filter such as {"never": {"tool_names": ["ask_question"]}}, for tools that can run without a check. Declare the server on every request, including continuations: the headers and authorization values are used for that response only and are never stored or echoed back.

Streaming

Set stream: true to receive the standard OpenAI semantic events (response.created, response.output_item.added, response.output_text.delta, …) and a terminal response.completed (or response.incomplete) event carrying the full final response, context_management included.

Hosted runs also stream their progress as response.output_item.added and response.output_item.done events carrying the call items; see streaming hosted runs.

Reasoning and thinking

Toast 1 has no thinking channel: thinking is disabled at the chat template, and chat_template_kwargs is not a parameter of this API. The response emits no reasoning output items, and usage.output_tokens_details.reasoning_tokens is always 0.

A hosted run returns the model's answer as message output items in output; the tools it ran to get there are recorded in hosted_tool_calls.