Token-Efficient Apps with TOON and the Vercel AI SDK

To use TOON with the Vercel AI SDK, serialize your data context as a TOON string and pass it in the message content to generateText or streamText. Keep generateObject and tool-call schemas on JSON. This pattern cuts input tokens by about 43% on average — with no changes to your SDK configuration or model setup.

Why Format Context Data Differently from Structured Output?

The Vercel AI SDK gives you two distinct concerns: feeding data into a model (context, retrieved records, tool results) and getting structured data out of a model (generateObject, tool-call arguments). These two directions have different optimal formats, and conflating them costs you either tokens or accuracy.

The evidence is clearest in a February 2026 arXiv paper (arXiv 2603.03306), "Token-Oriented Object Notation vs JSON: A Benchmark of Plain and Constrained Decoding Generation." It confirmed that TOON has a strong accuracy-per-token advantage for comprehension and retrieval (input), but for generation — asking the model to produce structured output — plain JSON had the best one-shot and final accuracy. TOON's edge on output is reduced by the prompt tax: the instructional overhead needed to teach or constrain the model to emit TOON format.

The official toonformat.dev benchmarks (v4.1) ran 5,856 LLM calls across 244 questions, six formats, and four models. TOON achieved 72.2% retrieval accuracy versus JSON's 71.4% — effectively a tie — while using 42.6% fewer tokens: 29.2 accuracy-points per thousand tokens against JSON's 16.6. On flat uniform arrays (retrieved rows, table scans, tool results), TOON reduced tokens by 58.7%.

The practical rule: feed context as TOON, receive structured output as JSON.

What Is the Input vs. Output Split in the Vercel AI SDK?

Before writing the code, it helps to be precise about which parts of an AI SDK call are "input" and which are "output."

The bottom row is the most impactful in agentic loops. Tool results are injected back into the context on every iteration. See our post on building an MCP server that returns TOON for a deeper treatment of that pattern, or point the AI SDK MCP client at the hosted TOON MCP server.

How to Pass TOON Context Data into generateText

The Vercel AI SDK accepts a plain string in the content field of any message. You do not need any SDK configuration change to use TOON — you simply serialize your data before building the messages array, then pass the resulting string as the content.

The example below shows the same product catalog query built twice: once with the records serialized as JSON, once as TOON. Both are valid SDK calls; the TOON version sends significantly fewer input tokens.

import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";
import { encode } from "@toon-format/toon";

// Sample dataset: 50 product records fetched from your database
const products = [
  { id: 1, name: "Widget A", category: "hardware", price: 9.99, inStock: true },
  { id: 2, name: "Widget B", category: "hardware", price: 14.99, inStock: false },
  // ... 48 more rows
];

// ── Option A: JSON context (verbose) ──────────────────────────────────────────
const jsonContext = JSON.stringify(products);
// Token cost: all keys repeated for every object → ~high token count

// ── Option B: TOON context (compact) ──────────────────────────────────────────
const toonContext = encode(products);
// Resulting TOON string (conceptually):
//
// products[50]{id,name,category,price,inStock}:
//   1, Widget A, hardware, 9.99, true
//   2, Widget B, hardware, 14.99, false
//   ...
//
// Keys declared once in the header → about 59% fewer tokens on flat arrays

// ── Vercel AI SDK call — context as TOON, output as JSON (via generateObject) ─
const { text } = await generateText({
  model: openai("gpt-4o"),
  messages: [
    {
      role: "system",
      content:
        "You are a product assistant. " +
        "The catalog uses TOON format: a header declares the array length and field names, " +
        "followed by comma-separated rows. Answer questions using only the provided data.",
    },
    {
      role: "user",
      content:
        "Here is the current product catalog:\n\n" +
        toonContext +
        "\n\nHow many items are currently out of stock?",
    },
  ],
});

console.log(text);

The system message adds a brief format explanation — the prompt tax. On 50 rows with 5 fields, this overhead is trivial compared to the per-row token savings. At fewer than roughly 10 objects, the math flips; see our API cost optimization guide for the full break-even analysis.

How Much Can TOON Actually Save in Practice?

The savings depend heavily on data shape. The toonformat.dev benchmark measured real token counts across several dataset types:

• Flat / uniform tables (flat-only track): 58.7% reduction (68,030 vs 164,643 tokens)
• Time-series data (60 days): 59.0% reduction (9,115 vs 22,245 tokens)
• GitHub repository data: 41.7% reduction (8,937 vs 15,337 tokens)
• E-commerce orders (nested): 32.9% reduction (72,832 vs 108,611 tokens)
• Mixed structures (mixed-structure track): 32.7% reduction (264,734 vs 393,637 tokens)

In an agentic loop where the same context is re-sent on each iteration, the savings multiply. A 43% reduction in context size means 43% fewer tokens billed on every turn. Combined with prompt caching — where Anthropic offers a 90% discount on cached reads and OpenAI caches automatically above 1,024 tokens — the economics compound further. Our cost-efficient chatbot guide covers how to stack these techniques.

Keeping generateObject and Tool Schemas on JSON

generateObject in the Vercel AI SDK uses a Zod schema (or a JSON Schema equivalent) to constrain the model's output to a specific shape. This is exactly the generation use case where JSON wins. The model emits structured JSON, the SDK validates it, and your application code consumes a typed object. Switching generateObject to TOON output would require custom parsing and would reduce accuracy — the arXiv study is explicit that JSON outperforms TOON for one-shot structured generation.

The same logic applies to tool-call arguments. When the model emits a tool call, the arguments are JSON Schema-validated. The model has deep training on JSON as a generation format. Do not attempt to reformat tool args as TOON; it provides no benefit and risks parse errors.

Tool results, however, are a different matter. A result returned to the model is consumed by comprehension, not generated — so TOON is appropriate. Encode the result as TOON before inserting it into the tool result message, and the model reads it as efficiently as any other TOON context block.

For a deeper look at where TOON fits in the format decision, see our JSON vs TOON comparison.

Accuracy Across Models: Test Before You Deploy

TOON accuracy varies significantly by model. The official v4.1 benchmark results across four models were (JSON lands within about two points of TOON on each):

• Grok 4.5: 97.1% accuracy with TOON
• Gemini 3.6 Flash: 69.3%
• Claude Haiku 4.5: 65.6%
• GPT-5.4 Nano: 57.0%

If you are routing through a smaller or less capable model in your Vercel AI SDK application, TOON comprehension accuracy could be substantially lower than the headline figure. Run a representative sample of your real queries against your target model before committing to TOON in production. The free converter at json2toon.co lets you inspect the TOON output for your actual data before writing any SDK code.

Frequently Asked Questions