Does TOON Save Tokens on Every Tokenizer? GPT, Claude, Gemini & Llama

TOON saves tokens on every BPE-based tokenizer — GPT, Claude, Gemini, and Llama included — because it removes structural repetition that all tokenizers charge for. But the exact savings depend on vocabulary size, and accuracy varies sharply by model. Always count tokens on your actual target model before treating the benchmark numbers as gospel.

Why Token Savings Are Universal But Not Identical

Every major LLM API uses a variant of Byte Pair Encoding (BPE) for tokenization. BPE starts from individual bytes, then iteratively merges the most frequent adjacent pair into a new token until a target vocabulary size is reached. At inference time, those merge rules are applied deterministically to split any input text into tokens.

TOON's savings come from a structural reduction, not a tokenizer quirk. In a JSON array of N objects with the same schema, every key is repeated N times — surrounded by double quotes, a colon, and commas. TOON replaces all of that with a single header line: items[N]{field1,field2,field3}:. The per-row token cost collapses to the values alone plus a delimiter. This collapse happens regardless of the tokenizer's vocabulary because the keys themselves are what disappear.

What the vocabulary does affect is the baseline token count for the JSON side of the comparison. A tokenizer with a larger vocabulary can merge multi-character sequences — including some punctuation clusters — into single tokens, which lowers the JSON baseline slightly and narrows TOON's relative advantage. The magnitude of savings varies; the direction does not.

For background on how TOON's tabular block syntax works and why it was designed that way, see our TOON introduction.

Which Tokenizer Did the TOON Benchmark Use?

The official toonformat.dev benchmarks ran 5,016 LLM calls across 209 questions, six formats, and four models. All token counts were computed using OpenAI's o200k_base tokenizer via the gpt-tokenizer npm package.

o200k_base is the tokenizer for the GPT-4o and GPT-5 model families. It has a vocabulary of approximately 200,000 tokens — roughly twice the size of cl100k_base (used by GPT-3.5-turbo and GPT-4). The larger vocabulary means more multi-character sequences, including common English words, merge into single tokens. In English prose, o200k_base averages roughly 4 characters per token.

Using o200k_base as the benchmark tokenizer is a reasonable choice for measuring token-count comparisons when targeting GPT-5-family models. For other models the absolute token counts will differ, though the percentage reduction from TOON should remain in the same general range because the structural savings are proportional to the number of repeated keys, not to how the tokenizer handles individual characters.

For a full breakdown of what the benchmarks measured and how 39.9% average savings splits across data shapes, see our TOON format comparison guide.

The Same Array Across Tokenizers: What Changes and What Stays

The following example uses a small product array to illustrate the structural reduction that applies under any BPE vocabulary, and to show where tokenizer differences show up.

// JSON — keys repeated on every row.
// Every tokenizer charges for "id", "name", "price", "qty" four times.
[
  {"id": "A1", "name": "Widget",  "price": 9.99,  "qty": 120},
  {"id": "A2", "name": "Gadget",  "price": 24.50, "qty": 45},
  {"id": "A3", "name": "Doohickey","price": 4.99, "qty": 300},
  {"id": "A4", "name": "Thingamajig","price":14.99,"qty": 80}
]
// o200k_base estimate: ~85 tokens
// cl100k_base estimate: ~90 tokens  (slightly more; smaller vocab, more splits)
// Llama-3 (32k vocab) estimate: ~95 tokens  (smallest vocab, most splits)

// TOON — keys declared once in the header; rows are pure values.
products[4]{id,name,price,qty}:
  A1, Widget,      9.99,  120
  A2, Gadget,      24.50,  45
  A3, Doohickey,   4.99,  300
  A4, Thingamajig, 14.99,  80
// o200k_base estimate: ~52 tokens  (~39% fewer)
// cl100k_base estimate: ~54 tokens  (~40% fewer)
// Llama-3 (32k vocab) estimate: ~56 tokens  (~41% fewer)
//
// The percentage reduction is similar across all three tokenizers.
// The absolute counts differ; the structural advantage does not.

Notice that the percentage reduction is stable — roughly 39–41% — while the absolute counts shift modestly. As datasets grow to hundreds or thousands of rows, the structural savings dominate further and the tokenizer-vocabulary effect becomes negligible.

For a side-by-side comparison of JSON and TOON token counts across multiple data shapes with the GPT-5 tokenizer, see our JSON vs TOON deep-dive.

Accuracy Varies Sharply by Model — Count Both Dimensions

Token savings are a tokenizer property. Accuracy is a model property. They are independent, and the gap between models on accuracy is far wider than the gap in savings.

The official benchmark tested four models. Overall TOON accuracy across all models was 76.4% at 39.9% fewer tokens — 27.7 accuracy-points per thousand tokens versus JSON's 16.4. But that overall figure masks a dramatic spread by model:

• Gemini 3 Flash: 96.7%
• GPT-5 Nano: 90.9%
• Claude Haiku: 59.8%
• Grok 4.1: 58.4%

A 38-percentage-point gap between the best and worst model on the same format is not a rounding error — it is a fundamental difference in how well each model was trained to parse TOON's tabular syntax. If you are routing to Claude Haiku or Grok 4.1, you cannot assume the 96.7% figure applies to your workload. Run your own accuracy evaluation before committing TOON to production on those models.

For the Claude-specific picture, including why smaller Claude models struggle with TOON and what that means for cost-accuracy trade-offs, see TOON efficiency with Claude models.

Tokenizer and Model Comparison Table

The following table summarizes what to expect from TOON across the major tokenizer families and the models that use them. Token-savings behavior is structural and consistent; accuracy caveats are model-specific.

If you need tokenizer-aware optimization without manual counting, TONL is multi-tokenizer aware out of the box, with explicit support for GPT-5, Claude Sonnet 4.5, Gemini 2.5/3 Pro, and Llama-4.

How to Measure TOON Savings for Your Model

The benchmark numbers are a useful starting point, but the correct way to validate TOON savings for your specific model and dataset is to measure directly. Each major provider exposes a token-counting path:

• GPT models: Use the tiktoken library with the o200k_base or cl100k_base encoding. Run len(enc.encode(text)) on both the JSON and TOON versions of your dataset.
• Claude models: Use the Anthropic token-counting API endpoint (POST /v1/messages/count_tokens) — it accepts the same message format as the main API and returns exact counts before you incur inference cost.
• Gemini models: Use the countTokens method in the Gemini API. It returns the token count for a given prompt without running inference.
• Llama models (self-hosted): Use the tokenize endpoint on your serving infrastructure, or load the tokenizer directly from the model's tokenizer.json via the Hugging Face transformers library.

Beyond counting, also measure accuracy on a representative sample of your actual queries. The benchmark used 209 questions spanning field retrieval, aggregation, filtering, and structure awareness. For most production retrieval tasks — which skew toward simple field lookups — TOON's 99.6% accuracy on field retrieval is the relevant figure, not the blended 76.4% average.

Use our free converter to generate TOON from your JSON instantly and paste both versions into your preferred token counter.

TONL: When You Need Multi-Tokenizer Guarantees

If you are routing requests across multiple model providers — for example, GPT-5 for reasoning and Gemini for speed — manually verifying token savings for each tokenizer gets tedious. TONL (Token-Optimized Notation Language) was built for exactly this scenario. At version 2.5.2, it explicitly supports GPT-5, Claude Sonnet 4.5, Gemini 2.5/3 Pro, and Llama-4, with tokenizer-aware serialization that maximizes savings on each target.

TONL also adds type hints (u32, str, bool), which cost roughly 20 additional tokens but unlock schema validation, auto TypeScript generation, and a SQL-like query API with indexed lookups under 0.1ms. Even with type hints, TONL saves 32–50% fewer tokens than JSON. For a comparison of when to use TOON versus TONL, see our format comparison guide.

Frequently Asked Questions