OpenAI Responses API & Agents SDK

Unified Tool Execution

Built-in tools — web_search_preview, file_search, computer_use_preview — are first-class citizens of the API. You declare which tools are available; the model decides when to invoke them; OpenAI executes them and returns results as part of the response. No polling loops, no separate Run state machine.

Stateful Conversations

Pass store: true on any call and OpenAI stores the response server-side. On subsequent turns, pass previous_response_id instead of a full message array. OpenAI reconstructs context automatically. Opt-out anytime by returning to manual message history management — the two patterns interoperate.

Streaming First-Class

All response types — text, tool calls, reasoning summaries — stream as typed events. The ResponseStream abstraction in the SDK surfaces events like response.text.delta, response.tool_call.arguments.delta, and lifecycle events. Streaming is not an afterthought; it is the primary design mode.

Reasoning Models

The o1, o3, and o3-mini series are fully supported. Set reasoning.effort to low, medium, or high to trade latency for answer quality. Streaming reasoning summaries (not full chain-of-thought) are available where models support it.

Structured Outputs

JSON Schema enforcement is built into the API via text.format. When you define a schema, the model is constrained to produce valid JSON matching it. This removes the need for output parsers and retry logic, and makes downstream validation deterministic.

Background Mode

Long-running agentic tasks (multi-step research, code generation, document analysis) can run asynchronously via background: true. You receive a response ID immediately and poll for completion — a correct pattern for tasks that exceed HTTP timeout limits or need to survive client disconnects.

Web Search

web_search_preview gives the model live web access. Results are cited inline with source URLs. Useful for agents that need current information beyond training data cutoffs — news analysis, pricing research, documentation lookup. The model decides when a query warrants a search; you can also force tool use via tool_choice.

File Search

Semantic search over Vector Stores you create via the Files API. Supports hybrid search (keyword + semantic), multiple vector stores per call, and attribute filters. Replaces Assistants API File Search with the same underlying infrastructure but a simpler invocation model. Max 10,000 files per vector store.

Computer Use (Preview)

computer_use_preview allows the model to control a desktop or browser environment — click, type, scroll, screenshot. Built on the CUA (Computer Use Agent) model. Appropriate for automation workflows where structured APIs don't exist. Currently preview; production use requires careful sandboxing and human oversight.

Function Calling (Custom Tools)

Your own tools alongside built-in ones. Define JSON Schema function specs; the model generates valid arguments; your code executes and returns results. The Responses API processes function results within the same request-response cycle via the tool_result input type — simpler than the Assistants API Runs pattern.

Agents

The core primitive. An Agent is a named configuration object: system instructions, a model, a list of tools, and optional output type (for structured outputs). Agents are stateless — all state lives in the Runner context. You can define dozens of specialized agents (triage, research, writer, validator) and wire them together via handoffs.

Handoffs

Agents can transfer control to other agents via handoff tools. The current agent calls transfer_to_[agent_name], passes a context message, and execution continues with the target agent in the same conversation thread. This is how you build triage → specialist workflows. Handoffs are implemented as regular function calls — fully transparent in traces.

Guardrails

Parallel safety and validation checks that run alongside the main agent response. An input guardrail fires before the main agent response — useful for topic classification, PII detection, or abuse screening. An output guardrail fires after — useful for policy compliance, hallucination checks, or format validation. Guardrails can trip a GuardrailTripwireTriggered exception that halts the workflow.

Runner & Context

The Runner.run() method executes an agent loop: call model, process tool calls, repeat until a final output or handoff. The RunContext carries user-defined state across the entire run — database connections, user session data, accumulated results. Context is typed and dependency-injected; agents receive it via tool function signatures.

Tracing

Every SDK run generates a full trace: agent invocations, tool calls, handoff decisions, guardrail evaluations, token usage. Traces send to the OpenAI platform by default (visible in the dashboard) and can be exported to third-party observability tools. This is the main observability advantage of the SDK vs. raw API calls.

Voice Pipeline

The SDK includes a VoicePipeline abstraction for speech-to-speech agent applications: STT → agent → TTS, with streaming audio in and out. Useful for phone agents, voice assistants, and real-time meeting bots. Audio input goes through Whisper; output through TTS; the agent runs on the Responses API in between.

Store Mode Data Residency

When store: true is set, conversation history is stored in OpenAI's infrastructure. For healthcare, legal, or financial applications with data governance requirements, this may be incompatible with compliance obligations.

Our approach: Default to manual message history management with store: false. Maintain conversation history in your own database, giving you full control, portability, and auditability. Only use store mode for non-sensitive, internal tooling.

Web Search Hallucinated Citations

The web_search_preview tool occasionally returns citations that do not match the actual retrieved content — particularly on low-traffic URLs or paywalled sources the model has seen in training data.

Our approach: Always surface citations to end users with clickable links — let humans verify. For automated pipelines, treat web search as a signal, not ground truth. Implement a validation step that fetches and spot-checks cited URLs before including them in downstream outputs.

Handoff Loops in Multi-Agent Workflows

Agents can get into circular handoff patterns — agent A defers to agent B, which defers back to A, indefinitely. The SDK does not enforce a hard handoff depth limit by default. This surfaces as a hanging run rather than an error.

Our approach: Set explicit max_turns on the Runner. Design triage agents with clear escalation rules and a “cannot resolve” fallback path. Monitor handoff counts per run in traces and alert on runs exceeding your expected depth.

Guardrail Latency Overhead

Guardrails run in parallel but still add measurable latency — particularly output guardrails that must wait for the primary response to complete before evaluating. A guardrail that calls a second model adds a full inference round-trip to every user interaction.

Our approach: Use fast, cheap models (gpt-4o-mini, o3-mini low) for guardrails. Design guardrails as binary classifiers, not complex evaluators. For latency-sensitive applications, consider async guardrail evaluation with post-hoc intervention rather than blocking the user response.

Background Mode Polling Complexity

Background runs require your application to manage polling — request status, handle timeouts, surface partial results, and recover from failures. Teams underestimate the infrastructure required: job queue, status persistence, webhook or SSE for completion notification.

Our approach: Treat background runs as async jobs from day one. Use a proper job queue (BullMQ, Celery, or similar) to manage background run lifecycles. Store run IDs with job metadata; implement idempotent polling with exponential backoff; surface progress to users via a status endpoint rather than leaving them with a spinner.

Context Injection Costs at Scale

Agentic runs accumulate context rapidly — tool call results, intermediate reasoning, handoff messages. A 10-step agent run with moderate tool outputs can easily consume 50,000+ input tokens. At GPT-4o pricing, this makes per-session costs an order of magnitude higher than simple chat applications.

Our approach: Model each agent run's token budget before deployment. Use gpt-4o-mini for intermediate steps that don't require flagship reasoning. Compress intermediate tool results to relevant excerpts before injecting into the next step. Set per-session cost caps with automatic termination.