Does TOON Save Tokens on Every Tokenizer? GPT, Claude, Llama

TOON saves tokens on every BPE-based tokenizer — GPT, Claude, Gemini, and Llama included — because it removes structural repetition that all tokenizers charge for. The exact counts depend on the vocabulary, and Claude's tokenizer changed with Claude 4.7. Accuracy depends on the model. Count tokens on your actual target model before relying on the benchmark's 42.6% average.

Why Token Savings Are Universal But Not Identical

Every major LLM API uses a variant of Byte Pair Encoding (BPE) for tokenization. BPE starts from individual bytes, then iteratively merges the most frequent adjacent pair into a new token until a target vocabulary size is reached. At inference time, those merge rules are applied deterministically to split any input text into tokens.

TOON's savings come from a structural reduction, not a tokenizer quirk. In a JSON array of N objects with the same schema, every key is repeated N times — surrounded by double quotes, a colon, and commas. TOON replaces all of that with a single header line: items[N]{field1,field2,field3}:. The per-row token cost collapses to the values alone plus a delimiter. This collapse happens regardless of the tokenizer's vocabulary because the keys themselves are what disappear.

What the vocabulary does affect is the baseline token count for the JSON side of the comparison. A tokenizer with a larger vocabulary can merge multi-character sequences — including some punctuation clusters — into single tokens, which lowers the JSON baseline slightly and narrows TOON's relative advantage. The magnitude of savings varies; the direction does not.

For background on how TOON's tabular block syntax works and why it was designed that way, see our TOON introduction.

Which Tokenizer Did the TOON Benchmark Use?

The official toonformat.dev benchmarks (current run, v4.1) made 5,856 LLM calls: 244 questions, six formats, and four models. All token counts were computed using OpenAI's o200k_base tokenizer via the gpt-tokenizer npm package.

o200k_base is the tokenizer for the GPT-4o and GPT-5 model families. It has a vocabulary of approximately 200,000 tokens — roughly twice the size of cl100k_base (used by GPT-3.5-turbo and GPT-4). The larger vocabulary means more multi-character sequences, including common English words, merge into single tokens. In English prose, o200k_base averages roughly 4 characters per token.

Using o200k_base as the benchmark tokenizer is a reasonable choice for measuring token-count comparisons when targeting GPT-5-family models. For other models the absolute token counts will differ, though the percentage reduction from TOON should remain in the same general range because the structural savings are proportional to the number of repeated keys, not to how the tokenizer handles individual characters.

TOON averaged 42.6% fewer tokens than JSON, with 58.7% on the flat-only track and 32.7% on the mixed-structure track. For a full breakdown of what the benchmarks measured, see our TOON format comparison guide.

The Same Array Across Tokenizers: What Changes and What Stays

The following example uses a small product array to illustrate the structural reduction that applies under any BPE vocabulary, and to show where tokenizer differences show up.

// JSON: keys repeated on every row.
// Every tokenizer charges for "id", "name", "price", "qty" four times.
[
  {"id": "A1", "name": "Widget", "price": 9.99, "qty": 120},
  {"id": "A2", "name": "Gadget", "price": 24.5, "qty": 45},
  {"id": "A3", "name": "Doohickey", "price": 4.99, "qty": 300},
  {"id": "A4", "name": "Thingamajig", "price": 14.99, "qty": 80}
]

The same data encoded with @toon-format/toon 4.1.1:

products[4]{id,name,price,qty}:
  A1,Widget,9.99,120
  A2,Gadget,24.5,45
  A3,Doohickey,4.99,300
  A4,Thingamajig,14.99,80

The four repeated key names, the braces, and the quotes around every key disappear under any tokenizer. What changes between tokenizers is how many tokens the remaining values and punctuation cost. A smaller vocabulary splits words like Thingamajig into more pieces, which raises both the JSON and the TOON count. The percentage reduction stays in a similar range, while the absolute counts shift. As datasets grow to hundreds or thousands of rows, the structural savings dominate and the vocabulary effect matters less.

For a side-by-side comparison of JSON and TOON token counts across multiple data shapes with the GPT-5 tokenizer, see our JSON vs TOON deep-dive.

Does TOON Accuracy Depend on the Model?

Token savings are a tokenizer property. Accuracy is a model property. The two are independent, and the spread in accuracy between models is much wider than the spread between formats on any one model.

In the current official benchmark, TOON scored 72.2% overall against JSON's 71.4% while using 42.6% fewer tokens. That works out to 29.2 accuracy points per 1,000 tokens versus JSON's 16.6. Per model, TOON vs JSON:

• grok-4.5: 97.1% vs 96.3%
• gemini-3.6-flash: 69.3% vs 68.4%
• claude-haiku-4-5-20251001: 65.6% vs 63.5%
• gpt-5.4-nano: 57.0% vs 57.4%

Models differ by about 40 points from one another, but on each model TOON and JSON are within about two points. The benchmark reports Wilson 95% confidence intervals and treats overlapping intervals as not statistically meaningful. So the practical takeaway is that switching to TOON does not cost accuracy on these models, while picking the right model matters a lot. An earlier run (early 2026) used different models and questions and reported different per-model figures. Do not mix the two runs.

For the Claude-specific picture, including current Claude models and pricing, see TOON efficiency with Claude models.

Tokenizer and Model Comparison Table

The following table summarizes what to expect from TOON across the major tokenizer families and the models that use them. Token-savings behavior is structural and consistent; accuracy caveats are model-specific.

If you need tokenizer-aware optimization without manual counting, TONL is multi-tokenizer aware out of the box, with explicit support for GPT-5, Claude Sonnet 4.5, Gemini 2.5/3 Pro, and Llama-4.

Did Claude's Tokenizer Change?

Yes. According to Anthropic's pricing documentation (as of October 2026), Claude 4.7 and later models use a newer tokenizer that "produces approximately 30% more tokens for the same text." Claude Sonnet 4.6 and earlier use the previous tokenizer. The exact increase depends on the content.

This matters for two reasons. First, if you migrate from Sonnet 4.6 to Sonnet 5.5 or Opus 5.5, the same JSON payload is billed as more tokens even before any price change, so cost estimates built on old counts will be low. Second, TOON's structural savings still apply. Removing repeated keys cuts characters, and fewer characters mean fewer tokens under either tokenizer. The absolute number of tokens saved per request is larger on a tokenizer that produces more tokens per character. Always measure with the token-counting endpoint on the exact model ID you deploy.

How to Measure TOON Savings for Your Model

The benchmark numbers are a useful starting point, but the correct way to validate TOON savings for your specific model and dataset is to measure directly. Each major provider exposes a token-counting path:

• GPT models: Use the tiktoken library with the o200k_base or cl100k_base encoding. Run len(enc.encode(text)) on both the JSON and TOON versions of your dataset.
• Claude models: Use the Anthropic token-counting API endpoint (POST /v1/messages/count_tokens) — it accepts the same message format as the main API and returns exact counts before you incur inference cost.
• Gemini models: Use the countTokens method in the Gemini API. It returns the token count for a given prompt without running inference.
• Llama models (self-hosted): Use the tokenize endpoint on your serving infrastructure, or load the tokenizer directly from the model's tokenizer.json via the Hugging Face transformers library.

Beyond counting, also measure accuracy on a representative sample of your actual queries. The benchmark used 244 questions spanning field retrieval, aggregation, filtering, structure awareness, and structural validation. For production retrieval tasks that skew toward simple field lookups, TOON's 97.8% on field retrieval (JSON: 99.2%) is the relevant figure, not the blended 72.2% average. Filtering is much harder for every format (TOON 38.0%, JSON 41.1%).

Use our free JSON to TOON converter to generate TOON from your JSON instantly and paste both versions into your preferred token counter, or compare them side by side in the TOON token calculator.

TONL: When You Need Multi-Tokenizer Guarantees

If you are routing requests across multiple model providers — for example, GPT-5 for reasoning and Gemini for speed — manually verifying token savings for each tokenizer gets tedious. TONL (Token-Optimized Notation Language) was built for exactly this scenario. At version 2.5.2, it explicitly supports GPT-5, Claude Sonnet 4.5, Gemini 2.5/3 Pro, and Llama-4, with tokenizer-aware serialization that maximizes savings on each target.

TONL also adds type hints (u32, str, bool), which cost roughly 20 additional tokens but unlock schema validation, auto TypeScript generation, and a SQL-like query API with indexed lookups under 0.1ms. Even with type hints, TONL saves 32–45% fewer tokens than JSON. For a comparison of when to use TOON versus TONL, see our format comparison guide.

Frequently Asked Questions