Agent architecture

Code mode: letting the model write code instead of calling tools

MCP made it easy to give an agent a hundred tools. Code mode is a slight correction: hand the model a typed API and let it write a program, so schemas and intermediate results stop flowing through the context window.

Nightshift EngineeringAugust 30, 202610 min read

The Model Context Protocol solved the connection problem. Any agent can talk to any tool, and the community answered by building thousands of servers. The bill for that arrives in the context window: every tool an agent might use has to be described to the model before the model reads the request, and every result a tool returns has to pass back through the model on its way to the next tool.

Instead of exposing MCP tools as callable functions the model selects between, you present them as a typed code API and give the model one tool: run this program. The model writes the program, a sandbox runs it, and only what the program prints comes back. This piece covers what that changes, what it costs, and how to try it.

Two costs of a bunch of tools

Connect an agent to a handful of MCP servers and two things happen.

The first is that tool definitions occupy context before any work starts. Each tool contributes a name, a description, and a full JSON Schema for its inputs and outputs. That is the price of admission, paid on every single request, for every tool the agent might conceivably reach for. Anthropic describes agents connected to enough servers that they process hundreds of thousands of tokens before reading the request.

The second is subtler. When you chain tools, results travel through the model. Read a meeting transcript from Google Drive, write it into Salesforce, and that transcript enters the context once as the output of the read and again as the input to the write. Anthropic gives the figure for a two hour sales meeting: roughly 50,000 extra tokens, spent transporting text the model had no reason to read.

Neither cost is a bug in MCP. They are what it looks like when a function-call interface meets a workload that wants a pipeline.

Figure 1: what a tool-calling turn leaves in the window

Model

MCP client: context window

MCP server

Model

System prompt

Tool 1 deftools/list

Tool 2 deftools/list result

User msg 1reads

Assistant msg 1writes

Call tool 1tools/call

Call tool 1 resulttools/call result

Assistant msg 2writes

User msg 2

MCP server

Nothing in the window leaves it. The next turn pays for all of it again.

The green rows are the two costs: schemas that arrive before any work starts, and results that land in the window on their way somewhere else. Diagram after the one in Anthropic’s post on code execution with MCP.

The idea

Code mode inverts the interface. Rather than advertising every tool and asking the model to pick one, you convert the MCP server schema into a typed API and expose a single entry point that runs model-written code against it.

Cloudflare, whose engineering blog popularised the name, generates a TypeScript declaration from the server schema, doc comments included.

The API handed to the model, generated from the MCP schematypescript

declare const codemode: {
  search_agents_documentation: (
    input: SearchAgentsDocumentationInput
  ) => Promise<SearchAgentsDocumentationOutput>;
};

Cloudflare generates this from the server schema at connect time. The model never sees a tool list, only a type it can program against.

The model then writes an ordinary async function. Loops, conditionals, destructuring and filtering all work, because it is just code.

What the model emits instead of a sequence of tool callstypescript

async () => {
  const projects = await codemode.list_projects({ status: "active" });
  const tasks = [];
  for (const project of projects) {
    tasks.push(...(await codemode.list_tasks({ projectId: project.id })));
  }
  return tasks.filter((task) => task.status === "blocked");
};

One model turn. Direct tool calling would need one call per project, each round-tripping its full result through the context window.

Why code beats tool calls

The efficiency argument is the obvious one, but there is a second argument that matters more than the cost.

Tool calling is a synthetic interface. Models learn it from purpose-built training data, because nothing resembling a JSON tool call appears in the wild at scale. Code is the opposite: models have read enormous quantities of real TypeScript and Python, written by people solving real problems, complete with error handling and edge cases. Asking a model to orchestrate through tool calls is like teaching Shakespeare Mandarin for a month and then asking him for a play.

That is why code mode tends to hold up as tools get more numerous and more complicated. You are asking the model to do something it has practiced for millions of examples rather than something it was drilled on.

Where the tokens actually go

Here is a simple pipeline. Pull a meeting transcript out of Google Drive and attach it to a Salesforce record. Under direct tool calling the transcript passes through the context twice. Under code mode the agent writes this instead.

./agent-script.tstypescript

import * as gdrive from './servers/google-drive';
import * as salesforce from './servers/salesforce';

const transcript = (await gdrive.getDocument({ documentId: 'abc123' })).content;
await salesforce.updateRecord({
  objectType: 'SalesMeeting',
  recordId: '00Q5f000001abcXYZ',
  data: { Notes: transcript }
});

The transcript moves between two systems inside the sandbox. It is never serialized into the conversation.

Anthropic reports this taking the task from about 150,000 tokens to about 2,000, which is 98.7% lower. The transcript still moves; it just moves through the runtime rather than through the model.

Those tools come from a generated tree, one file per tool, which is what makes the imports above work and what lets a model discover tools by listing a directory rather than by loading every schema up front.

Generated wrapper treebash

servers
├── google-drive
│   ├── getDocument.ts
│   ├── ... (other tools)
│   └── index.ts
├── salesforce
│   ├── updateRecord.ts
│   ├── ... (other tools)
│   └── index.ts
└── ... (other servers)

./servers/google-drive/getDocument.tstypescript

import { callMCPTool } from "../../../client.js";

interface GetDocumentInput {
  documentId: string;
}

interface GetDocumentResponse {
  content: string;
}

/* Read a document from Google Drive */
export async function getDocument(input: GetDocumentInput):
  Promise<GetDocumentResponse> {
  return callMCPTool<GetDocumentResponse>('google_drive__get_document', input);
}

Each file is a thin typed wrapper over one MCP tool call. The model reads the ones it needs and ignores the rest.

Filtering where the data already is

The sharpest win is on large results. Ask for a spreadsheet through a tool call and all of it lands in the context, where the model reads every row to find the few that matter. In code mode the filtering happens next to the data.

Filter in the sandbox, log a sampletypescript

const allRows = await gdrive.getSheet({ sheetId: 'abc123' });
const pendingOrders = allRows.filter(row =>
  row["Status"] === 'pending'
);
console.log(`Found ${pendingOrders.length} pending orders`);
console.log(pendingOrders.slice(0, 5)); // Only log first 5 for review

What reaches the modeloutput

Found 42 pending orders
[
  { id: 'ORD-1041', customer: 'Northwind', total: 1840.00, status: 'pending' },
  { id: 'ORD-1044', customer: 'Contoso',   total:  310.50, status: 'pending' },
  { id: 'ORD-1051', customer: 'Fabrikam',  total: 2210.75, status: 'pending' },
  { id: 'ORD-1052', customer: 'Northwind', total:   96.20, status: 'pending' },
  { id: 'ORD-1060', customer: 'Tailspin',  total:  740.00, status: 'pending' }
]

Ten thousand rows were read and filtered. Six lines entered the context window. The other 9,995 rows never existed as far as the model is concerned.

Control flow gets the same treatment. Polling, retries and backoff become a loop in the sandbox rather than a sequence of turns, each one costing an inference.

Polling without burning a turn per checktypescript

let found = false;
while (!found) {
  const messages = await slack.getChannelHistory({ channel: 'C123456' });
  found = messages.some(m => m.text.includes('deployment complete'));
  if (!found) await new Promise(r => setTimeout(r, 5000));
}
console.log('Deployment notification received');

Progressive disclosure, and a fixed context bill

Generating a wrapper per tool solves the chaining cost but not the schema cost, if the model still has to read every wrapper. The answer is to let it search.

Cloudflare collapses an entire MCP portal into two tools, a search and an execute, and reports the effect on an internal deployment: four servers exposing 52 tools, roughly 9,400 tokens of definitions, become 2 tools at roughly 600 tokens. The important property is not the ratio, it is that the number stops moving. Connect a fifth server and the context cost is unchanged.

The same trick applied to the Cloudflare API itself puts more than 2,500 endpoints behind about 1,000 tokens of context. The model queries the OpenAPI spec as data, then calls what it found.

Search: the model greps the spec instead of reading ittypescript

async () => {
  const spec = await codemode.spec();
  return Object.entries(spec.paths)
    .filter(([path]) => path.includes("/rulesets"))
    .map(([path, operations]) => ({path, methods: Object.keys(operations)}));
};

Returned to the modeloutput

[
  { path: '/zones/{zone_id}/rulesets',      methods: [ 'get', 'post' ] },
  { path: '/zones/{zone_id}/rulesets/{id}', methods: [ 'get', 'put', 'delete' ] }
]

Two entries, discovered from a spec of thousands. Only the relevant slice is paid for.

Execute: call what the search turned uptypescript

async () => {
  const response = await codemode.request({
    method: "GET",
    path: `/zones/${zoneId}/rulesets`
  });
  return response.result.map(({ id, name, phase }) => ({ id, name, phase }));
};

Note the projection on the way out. The full response never leaves the sandbox.

The privacy property

In code mode, intermediate results are private by default. The model sees what the program logs and nothing else, which means data can move between two systems without ever entering the context window.

That turns a policy question into an architecture. A client can intercept sensitive fields on the way through and substitute tokens, so the values are real where they land and opaque where the model can see them.

Real values move; the model is not in the pathtypescript

const sheet = await gdrive.getSheet({ sheetId: 'abc123' });
for (const row of sheet.rows) {
  await salesforce.updateRecord({
    objectType: 'Lead',
    recordId: row.salesforceId,
    data: {
      Email: row.email,
      Phone: row.phone,
      Name: row.name
    }
  });
}
console.log(`Updated ${sheet.rows.length} leads`);

What the model would see if it logged the rowsoutput

[
  { salesforceId: '00Q...', email: '[EMAIL_1]', phone: '[PHONE_1]', name: '[NAME_1]' },
  { salesforceId: '00Q...', email: '[EMAIL_2]', phone: '[PHONE_2]', name: '[NAME_2]' },
  ...
]

The MCP client swapped the values for tokens. Salesforce received the real ones.

Running the code in an isolate

You are now executing model-written code, so what isolation do we need?

Cloudflare runs each program in a V8 isolate that starts in milliseconds on a few megabytes, and, more importantly, gives it no general network access. MCP servers arrive as live bindings on the environment object rather than as URLs to fetch, so the sandbox holds no credentials at all. Calls on a binding go to the supervisor, which holds the tokens. A leaked key is not a risk if the key was never in the room.

Loading a program with a binding rather than a network routejavascript

let worker = env.LOADER.get(id, async () => {
  return {
    compatibilityDate: "2025-06-01",
    mainModule: "foo.js",
    modules: {"foo.js": "export default {...}"},
    env: {SOME_RPC_BINDING: ctx.exports.MyBindingImpl({props})}
  };
});

The boundary is a JavaScript interface, not an allowlist of hosts. That is a much easier thing to reason about.

FastMCP takes the same posture in Python and ships defaults you would otherwise have to invent: a 30 second wall clock, 100 MB of memory, and a cap of 50 tool calls per program.

Code mode on an existing FastMCP serverpython

from fastmcp import FastMCP
from fastmcp.experimental.transforms.code_mode import CodeMode

mcp = FastMCP("Server", transforms=[CodeMode()])

@mcp.tool
def add(x: int, y: int) -> int:
    """Add two numbers."""
    return x + y

The transform wraps tools you already wrote. Nothing about the tool definition changes.

Tightening the limitspython

from fastmcp.experimental.transforms.code_mode import (
    CodeMode,
    MontySandboxProvider,
)

sandbox = MontySandboxProvider(
    limits={"max_duration_secs": 10, "max_memory": 50_000_000}
)
mcp = FastMCP("Server", transforms=[CodeMode(sandbox_provider=sandbox)])

What it costs

Code mode is a trade, and the write-ups tend to be quiet about the other side of it.

Approvals get harder to read. "Call jira_list_issues with these arguments" is reviewable by anyone. Fifteen lines of TypeScript is a code review, and you are asking a user to do one before every action.
Error handling moves. A failed tool call comes back to the model, which can reason about it and try something else. A failed line inside a program throws in a sandbox, and unless the generated code handles it, what surfaces is a stack trace.
You are operating a sandbox now. Isolates or containers, resource limits, timeouts, and a supervisor holding credentials. That is real infrastructure with real failure modes.
It does not pay for itself at small scale. Three tools that never chain are fine as tool calls. The pattern earns its keep when tool counts climb, when calls compose, or when results are large.

Choosing between them

	Direct tool calls	Code mode
Context cost	Grows with every tool added	Fixed, with discovery on demand
Chained work	One model turn per call	One turn for the whole program
Large results	Enter the context in full	Filtered before anything is logged
Reviewing an action	Read one call and its arguments	Read a program
Infrastructure	None beyond the client	A sandbox and a supervisor
Best when	Few tools, little chaining	Many tools, composition, big payloads

What we take from it

The reason code mode interests us is not the token arithmetic, though that is real and it compounds. It is that the pattern moves the interesting decisions to a place you can govern.

When an agent emits tool calls, the only enforcement point is the tool boundary, and everything the tool returns is already in the model. When an agent emits a program that runs against a typed API behind a supervisor, you get a seam: you decide what the API exposes, what the sandbox may reach, which fields are tokenized on the way through, and what the program is permitted to log. Those are the same questions you would ask about any service that touches customer data, and code mode is what makes them askable.

It is early. The tooling is labelled experimental in FastMCP and the approval story genuinely is not solved. But the direction looks right: give the model the interface it has actually been trained on, and keep the data somewhere you can put a policy in front of.