How to Calculate Your LLM Token Savings from TOON

To estimate your TOON savings, multiply your monthly input token count by TOON's reduction rate — about 43% overall (42.6%), up to 58.7% on flat tables per toonformat.dev benchmarks — then multiply by your per-token price. Subtract the one-time prompt-tax overhead, and stack the result against any batch or caching discounts you qualify for.

Why Token Count Is the Right Starting Point

LLM pricing is almost always quoted per million tokens, so token count is the direct lever on your bill. When you send data to a model, that data must be tokenized before the model reads it. JSON serialization is token-inefficient for arrays of objects because every key, every brace, every quote character is repeated on every row.

TOON addresses this structurally. Its header line declares field names once — for example, users[500]{id,name,plan,created_at}: — and each subsequent row contains only values separated by commas. On a 500-row dataset that header pays for itself many times over. The official toonformat.dev benchmark (v4.1: 5,856 LLM calls, four models, six formats) measured a 42.6% overall token reduction and 58.7% on the flat uniform tables track — 68,030 tokens vs 164,643 tokens for the same data.

The savings are real, but the model is simple enough that you can verify it against your own data before any code changes. Use the TOON token calculator to paste your JSON and compare JSON, TOON and TONL token counts side by side, or count them with any tokenizer.

The Three-Step Savings Formula

The complete calculation has three components: raw token reduction, prompt-tax adjustment, and stacked pricing discounts.

Step 1 — Raw token reduction

Take your current monthly input token volume for the calls that send tabular or array data. Apply the reduction rate that matches your data shape. The toonformat.dev benchmarks give concrete figures by shape:

• Flat uniform tables: 58.7% fewer tokens
• Time-series (60 days of data): 59.0% fewer tokens
• GitHub repository data: 41.7% fewer tokens
• E-commerce orders (some nesting): 32.9% fewer tokens
• Mixed structures (track average): 32.7% fewer tokens, down to 15.0% on semi-uniform event logs

If you are unsure of your shape, 42.6% is the overall average; use 32.7% (the mixed-structure figure) for a conservative estimate.

Step 2 — Subtract the prompt tax

A February 2026 arXiv study (2603.03306) on TOON vs JSON identified what the authors call a prompt tax: the instructional overhead required to teach or constrain the model to read the TOON format. That overhead typically runs 50 to 150 tokens per system prompt. The tax is a one-time cost per conversation session, not per row, so it becomes negligible on large payloads. The study's non-linear scaling hypothesis is precise: TOON's efficiency pays off only beyond the threshold where cumulative per-row savings exceed the upfront instruction cost. For a four-field schema with a 100-token tax, that threshold is roughly 15 to 20 rows. Below it, JSON wins on total token cost.

Step 3 — Stack pricing discounts

Token volume and per-token price multiply together. Two structural discounts reduce the price side of that product and stack with TOON's volume reduction:

• Prompt caching. Anthropic charges cache reads at 10% of the standard input rate — a 90% discount — with cache writes at 1.25x (5-minute TTL) or 2x (1-hour TTL). OpenAI caches stable prefixes automatically once they exceed 1,024 tokens, at roughly 50% of the input rate. A smaller TOON block is cheaper to write into cache AND cheaper to re-read on every subsequent call. Sources: Anthropic prompt caching docs, ngrok prompt caching overview.
• OpenAI Batch API. Asynchronous batch requests receive a 50% discount on both input and output tokens, with a 24-hour turnaround window (most finish in 1 to 6 hours). The discount applies independently of caching. Sources: OpenAI pricing page.

Combining all three: on GPT-5.4 (illustrative standard input rate of $2.50/M tokens), cached batch input can reach approximately $0.625/M tokens — roughly 75% off the standard rate, before TOON's token-count reduction is applied on top.

Worked Example: 500-Row Product Catalog, 1 Million Calls per Month

Below is a concrete calculation for a product catalog feed — a flat uniform table, the best-case shape for TOON. Dollar figures are illustrative and tied to the cited rates above; substitute your actual pricing tier.

-- Scenario --
Dataset:    500 product rows, 6 fields each (id, sku, name, price, stock, category)
Volume:     1,000,000 API calls per month
Task:       Input only (feeding product data into a retrieval pipeline)
Assumed price: $2.50 / 1M input tokens (illustrative standard rate)

-- Token counts per call --
JSON serialization:   ~4,100 tokens per call   (keys repeated 500x each)
TOON serialization:   ~1,690 tokens per call   (keys declared once in header)
Reduction:            ~58.8% (in this example; in line with the 58.7% flat-table track from toonformat.dev)

Prompt tax (one-time per session, not per call):
  ~100 tokens for the TOON format instruction in the system prompt
  Amortized across 500 rows: negligible per call

-- Monthly token volume --
JSON:   4,100 tokens x 1,000,000 calls = 4,100,000,000 tokens = 4,100 M tokens
TOON:   1,690 tokens x 1,000,000 calls = 1,690,000,000 tokens = 1,690 M tokens
Saved:  2,410 M tokens / month

-- Monthly cost at standard rate ($2.50 / 1M) --
JSON standard:           $10,250 / month
TOON standard:           $ 4,225 / month
Savings (TOON only):     $ 6,025 / month

-- Stack: TOON + OpenAI Batch (50% off input) --
TOON + Batch:            $ 2,113 / month
Savings vs JSON standard: $ 8,137 / month

-- Stack: TOON + Batch + Prompt Caching (~50% off cached prefix) --
If system prompt with TOON instruction is cached:
  Prompt-tax tokens are re-read at ~50% rate on OpenAI
  For 1M calls with a 100-token cached prefix:
    Cache savings: 100 tokens x 1M calls x $1.25/M (batch+cached) = ~$125 / month
TOON + Batch + Caching:  ~$ 1,988 / month
Savings vs JSON standard: ~$ 8,262 / month  (roughly 81% cost reduction)

The numbers above use a flat-table shape, which is the best case. For a mixed-structure dataset, substitute 32.7% for 58.8% and the monthly TOON savings drop to roughly $3,350 at standard rates — still meaningful at this volume, but not transformative without the pricing stacks.

For a deeper treatment of how these optimizations apply to a full chatbot pipeline, see Building a Cost-Efficient Chatbot with TOON. For the mechanics of what makes JSON expensive token-by-token, see JSON vs TOON: a token-by-token comparison.

Scenario Comparison: Standard JSON vs TOON vs TOON with Stacked Discounts

The table below summarizes three common deployment scenarios at 1 million calls per month using illustrative per-token pricing. Actual rates vary by model and tier — consult your provider's pricing page.

The Anthropic caching row assumes the entire TOON data block is served from cache on repeated calls — the ideal case for static datasets like product catalogs or knowledge-base snapshots. For dynamic data that changes every call, only the system prompt prefix is cached, and the savings are smaller. The full API cost optimization guide covers when to structure prompts for maximum cache hit rate.

When the Math Does Not Favor TOON

The formula above has a clear failure mode: small payloads. If your typical call sends fewer than 15 to 20 rows, the prompt tax of 50 to 150 tokens can exceed the per-row savings TOON provides. The arXiv study (2603.03306) frames this as a non-linear scaling threshold: the cost curve only bends in TOON's favor once cumulative row savings exceed the upfront instruction overhead.

A second failure mode is mixed or deeply nested data. On mixed structures, TOON saves only 32.7% of tokens (15.0% on semi-uniform event logs), compared to 58.7% on flat tables. The formula still applies, but the denominator is much smaller, and the prompt tax looms larger in proportion. In those cases, JSON at standard pricing is often cheaper overall once you account for the zero-instruction overhead.

For a full decision framework covering which data shapes benefit from TOON and which do not, see our companion post Stop Using JSON for LLMs (And When to Keep Using It).

Putting the Model Into Practice

The cleanest way to apply this methodology to your own system is to export a representative sample of one day's data, convert it with the free converter, and count the tokens in both formats. Multiply the per-token difference by your monthly call volume and your pricing tier. That gives the raw savings figure. Then check whether the Batch API turnaround window fits your latency requirements, and whether your system prompt is stable enough to benefit from caching.

If your pipeline feeds retrieved rows into a model — a RAG system, an agent tool result, a database query response — the format of those rows is almost certainly your largest controllable token cost. The context window management guide shows how TOON's token reduction translates into fitting more evidence into a fixed-size window, which compounds the cost and quality benefits.

Frequently Asked Questions