Case Study: OpenAI Responses API and Agent Orchestration

OpenAI’s Responses API (March 2025) targets unified agent orchestration. If you build agents, understand this path—it differs from MCP’s open horizontal model.

Pre-Responses Pain

Multiple APIs, stitched by hand:

Chat Completions  → dialogue
Assistants API    → agents with tools (deprecated 2025)
Code Interpreter  → execution
File Search       → retrieval
Function Calling  → custom tools

Building one agent meant juggling APIs, IDs, and state. Assistants helped but was slow, state-heavy, inflexible.

Responses API Design

One entry point:

response = client.responses.create(
    model="gpt-4o",
    input="Analyze this sales CSV and find anomaly trends",
    tools=[
        {"type": "code_interpreter"},
        {"type": "file_search", "vector_store_ids": ["vs_abc123"]},
        {"type": "web_search_preview"},
        {"type": "function", "function": {
            "name": "query_crm",
            "description": "Query customer CRM",
            "parameters": {...}
        }}
    ],
    instructions="You are a data analyst. Reply in English."
)

One request—model picks tools, order, and synthesis.

Built-in Tools vs MCP

	Responses API	MCP
Tool schema	OpenAI functions	MCP Tools/Resources
Built-ins	code interpreter, file search, web	none (all servers)
Cross-platform	OpenAI only	any MCP host
Local execution	cloud sandbox	stdio local servers
State	`previous_response_id`	host-managed
Best for	fast OpenAI-native agents	open, local, multi-model

OpenAI = vertical integration. MCP = horizontal standard.

Internal Agent Loop (Simplified)

1. input + tools
2. LLM → answer or tool call
3. if tool: execute (built-in or function) → append → goto 2
4. return final response + trace

Same idea as harness loop controller—orchestration lives in OpenAI cloud.

Multi-Step Example

Input: "Compare Q3 vs Q4 sales, explain decline, write report"

Step 1: file_search → CSVs
Step 2: code_interpreter → pandas + charts
Step 3: web_search → industry Q4 trends
Step 4: code_interpreter → markdown report
Step 5: final output

One API call from your app; OpenAI runs the loop.

Computer Use

Computer tool via Responses API:

Agent can:
  - view screenshots
  - move mouse, click
  - type
  - operate GUI apps

Same class as Devin / Claude computer use—UI automation without APIs.

Good for legacy systems with no API.

Production Tradeoffs

Pros

Fastest time-to-agent (built-in tools)
Strong internal tool routing
Simple multi-turn via previous_response_id

Cons

Vendor lock-in
Multi-step token burn (each step re-calls LLM)
Limited local/private tooling
Higher P99 latency vs local harness

Cost Sketch (5-step GPT-4o task)

Steps accumulate context → ~37K tokens total ≈ $0.15–0.30 per task

1000 users × 3 tasks/day → $450–900/day

Agent orchestration often 5–10× chat cost.

When to Choose Which

Responses API if:
  ✅ MVP speed, OpenAI models enough
  ✅ no on-prem requirement
  ✅ built-ins cover needs
  ✅ don't want to maintain harness

MCP + custom harness if:
  ✅ multi-model (Claude + GPT + local)
  ✅ local/private execution
  ✅ deep harness customization
  ✅ long-term cost and vendor control

References

Summary

Responses API = cloud-all-in-one agent orchestration—simple, fast, OpenAI-bound. MCP + harness = open ecosystem—flexible, heavier engineering. Choose by stage and constraints.

Return to the AI series index for the full catalog.