Using TOON with Google Gemini

TOON works exceptionally well with Google Gemini. In the official TOON benchmark, Gemini 3 Flash scored 96.7% retrieval accuracy — the highest of the four models tested — while TOON itself used 39.9% fewer tokens overall and up to 58.8% fewer on flat uniform tables. For developers using Gemini's large context window, TOON is among the most efficient input formats available.

Why Gemini Scores the Highest TOON Accuracy of Any Model

The official TOON benchmark ran 5,016 LLM calls across 209 questions, six formats, and four models: claude-haiku-4-5, gemini-3-flash-preview, gpt-5-nano, and grok-4-1-fast. The methodology used the GPT-5 o200k_base tokenizer via gpt-tokenizer and covered six real-world data shapes — flat tables, time-series, e-commerce orders, GitHub repo metadata, and mixed structures.

Gemini 3 Flash achieved 96.7% retrieval accuracy on TOON. For comparison, GPT-5 Nano came second at 90.9%, Claude Haiku scored 59.8%, and Grok 4.1 scored 58.4%. The gap between Gemini and the lower scorers is substantial — nearly 38 percentage points — which makes Gemini an exceptionally safe pairing for TOON-formatted context in production.

The benchmark also shows that TOON's overall accuracy (76.4%) edged out JSON (75.0%) while consuming 39.9% fewer tokens, yielding 27.7 accuracy-points per 1,000 tokens versus JSON's 16.4. With Gemini at the top of the accuracy ranking, that efficiency ratio improves further in practice.

See the full accuracy breakdown and methodology in our TOON benchmark deep-dive, or compare TOON against JSON token-by-token in the JSON vs TOON guide.

How TOON Extends Gemini's Large Context Window

Gemini's large context window is one of its defining strengths. But a large window does not help if your input format wastes tokens on structural glyphs — repeated keys, braces, and quotes — that carry no information. TOON eliminates that overhead for uniform arrays of objects by declaring fields once in a header and then emitting only values per row.

The token savings by data shape, from the official benchmarks, are:

• Flat / uniform tables: 58.8% fewer tokens (67,778 vs 164,452)
• Time-series (60 days): 59.0% fewer tokens (9,115 vs 22,245)
• GitHub repo data: 42.3% fewer tokens (8,744 vs 15,144)
• E-commerce orders (nested): 33.3% fewer tokens (73,126 vs 109,599)
• Mixed structures: 21.9% fewer tokens (227,830 vs 291,711)

In practical terms: if your Gemini context window holds 1M tokens, encoding context as TOON instead of JSON effectively fits the data equivalent of roughly 1.6M tokens of JSON on flat tables. That is additional retrieved rows, more few-shot examples, or longer conversation history — without a model or pricing tier change.

For a complete cost analysis of how context format affects API spend, see our guide on optimizing API costs with TOON.

Gemini API Request: TOON Context vs JSON Context

The following example shows the same product catalog data sent to the Gemini API first as JSON, then as TOON. Both carry identical information; TOON is significantly more compact.

// ── JSON version ── (approx 210 tokens for 5 rows)
const jsonPayload = {
  contents: [
    {
      role: "user",
      parts: [
        {
          text: `Here is our product catalog in JSON:
[
  {"id":1,"name":"Widget A","price":9.99,"stock":142,"category":"tools"},
  {"id":2,"name":"Widget B","price":14.49,"stock":87,"category":"tools"},
  {"id":3,"name":"Gadget X","price":49.00,"stock":23,"category":"electronics"},
  {"id":4,"name":"Gadget Y","price":79.99,"stock":11,"category":"electronics"},
  {"id":5,"name":"Part Z","price":2.50,"stock":500,"category":"supplies"}
]

Which products have fewer than 25 units in stock?`
        }
      ]
    }
  ]
};

// ── TOON version ── (approx 88 tokens for 5 rows — ~58% fewer)
const toonPayload = {
  contents: [
    {
      role: "user",
      parts: [
        {
          text: `Here is our product catalog in TOON format
(fields declared once in the header; values follow as comma-separated rows):

products[5]{id,name,price,stock,category}:
  1, Widget A, 9.99, 142, tools
  2, Widget B, 14.49, 87, tools
  3, Gadget X, 49.00, 23, electronics
  4, Gadget Y, 79.99, 11, electronics
  5, Part Z, 2.50, 500, supplies

Which products have fewer than 25 units in stock?`
        }
      ]
    }
  ]
};

// Send to Gemini API
const response = await fetch(
  `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent?key=${API_KEY}`,
  {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify(toonPayload),
  }
);

The TOON version declares the schema once in the header (products[5]{id,name,price,stock,category}:) and then lists values without repeating keys. For five rows this saves a modest number of tokens, but the savings scale linearly: 500 rows of the same schema would save roughly 5,800 tokens versus JSON.

If you need to convert an existing JSON file to TOON before sending it to Gemini, the free json2toon.co converter handles that client-side with no data leaving your browser. For a primer on TOON syntax, see What is TOON?

Gemini Use Cases: When to Use TOON vs JSON

Not every Gemini prompt benefits from TOON. The format pays off on large, uniform arrays; it adds overhead (the "prompt tax" of format instructions) on small or non-uniform payloads. A 2026 arXiv paper (2603.03306) confirmed TOON's efficiency advantage for comprehension and retrieval while noting that plain JSON outperforms TOON on generation tasks where the model must produce structured output.

TOON and Gemini Context Caching: A Multiplicative Effect

Prompt caching discounts the price per token. TOON reduces the number of tokens. These two effects multiply rather than add.

When you cache a stable TOON context block in Gemini — a product catalog, a user table, a metrics dataset — you write a smaller block to the cache and then re-read a smaller block on every subsequent turn. The caching discount applies to that already-reduced token count. At scale across hundreds of requests, the combined effect can reduce context costs by 70–80% compared to uncached JSON, depending on cache hit rates and data shape.

This pattern is particularly effective for agentic workflows where the same tabular data is referenced repeatedly across multiple tool calls or conversation turns. For a detailed cost breakdown of TOON combined with caching and batch APIs, see our API cost optimization guide.

Model Accuracy Varies: Always Test on Your Data

The 96.7% figure for Gemini 3 Flash is the benchmark average across all question types and data shapes. Within that aggregate, accuracy varies by task type. The official benchmarks break down TOON accuracy across all models:

• Field retrieval: 99.6% — essentially perfect
• Structure awareness: 89.0%
• Structural validation: 70.0%
• Aggregation: 61.9%
• Filtering: 56.8%

If your Gemini integration primarily involves aggregation (sum, count, average across rows) or filtering (find rows matching a condition), the average accuracy is lower. For those workloads, consider pre-computing aggregates or filters before the prompt, or evaluate the full benchmark data against your specific question types.

The benchmark also tested a specific model version (gemini-3-flash-preview). Newer Gemini model releases may score differently. Run your own accuracy evaluation against a representative sample of your queries before committing to TOON in production — the converter lets you generate TOON from any JSON dataset in seconds.

Frequently Asked Questions