Is TOON Safe for AI Agents? What the 2026 Research Says

Not as a default, according to the best 2026 evidence. The arXiv study "Notation Matters" found that TOON saves up to 18% of tokens in agent workloads but costs about 9 accuracy points on open-weight models. In multi-turn loops, parse failures cascade into extra reasoning steps. TOON still works well as an input format for tool results, as long as the model writes its tool calls in JSON.

What did the 2026 TOON agent study test?

"Notation Matters: A Benchmark Study of Token-Optimized Formats in Agentic AI Systems" by Kutschka and Geiger (v1 May 28, v2 June 17, 2026) tests TOON in tool-calling agents rather than in retrieval questions. It compared JSON, TOON and TRON across four benchmarks:

• BFCL (Berkeley Function Calling Leaderboard): 2,040 single-turn function-calling tasks, scored by AST matching.
• MCPToolBenchPP: 1,509 single-turn MCP tasks in six domains.
• MCP-Universe: 231 multi-turn MCP tasks verified against live APIs.
• StableToolBench: 330 solvable multi-turn queries against cached real APIs.

The models were five open-weight configurations: Mistral-Small-24B, Qwen3-32B with thinking on and off, DeepSeek-R1-Distill-Qwen-32B, and Llama-4-Scout. The design separates two conditions. In input-only compression, the model reads schemas and tool results in the compact format but still writes JSON tool calls. In full compression, the model also writes its output in the compact format. That split is what makes the study useful in practice.

What were the headline results?

The abstract summarizes it: "TRON reduces tokens by up to 27% with accuracy within 14 pp of the JSON baseline. TOON achieves up to 18% reduction at a similar 9 pp accuracy cost, but additionally cascades on multi-turn parsing failures and collapses parallel tool-call output for most models." The full paper adds the detail:

The final verdict quotes directly: "TOON is not safe as a default in multi-turn agentic systems, because its stronger per-call compression is offset by parsing-cascade effects and by structural failures on parallel tool-call output."

Why does TOON break in multi-turn agent loops?

The paper describes two failure modes, and both come from the model writing TOON, not reading it.

Parsing cascades. In an agent loop, a tool call that fails to parse does not simply fail. It triggers another reasoning iteration with its own thought, action and observation. For reasoning models, that iteration includes the full chain of thought. The authors found that two or three of these extra iterations can use up all the per-call savings gathered over a whole trajectory. Under full compression, Qwen3-32B with thinking ended up using 8% more tokens with TOON than with JSON.

Parallel tool calls. BFCL's parallel and parallel_multiple categories "collapse to near zero accuracy across most (model, format) combinations" under full compression. The paper's explanation is that models trained mostly on JSON tool calls do not reliably reproduce parallel-call structure in another format. During pilot runs, the models also "defaulted back to JSON-shaped output regardless of the surrounding format instruction."

This matches an earlier result. A 2026 study of TOON generation, arXiv 2603.03306, found that the instructions needed to make a model write TOON add a "prompt tax" that can cancel out the savings. Our guide on when not to use TOON covers that paper.

Where did TOON actually help agents?

The negative headline hides some useful positive results. With input-only compression, TOON produced its largest saving, about 30%, on MCPToolBenchPP, where tool results are verbose tabular data. On that benchmark Mistral-Small-24B gained 13 accuracy points while using 29% fewer tokens. The authors' hypothesis is that compression "reduces noise on the model's attention surface". Dropping JSON's structural characters frees attention for the actual task.

Reasoning also helped. Qwen3-32B with thinking enabled was "the only configuration whose TOON accuracy matches its JSON baseline under input-only compression". Turning thinking off cost 8 points with TOON. Explicit reasoning seems to help a model handle an unfamiliar format.

In short, TOON helped when it carried large, table-shaped tool results into the prompt. It hurt when the model had to write tool calls in it, or when the payloads were small schemas with little repetition.

How does this fit with the official TOON benchmark?

The official TOON benchmark (spec v4.1, 5,856 LLM calls) reports 72.2% accuracy for TOON against 71.4% for JSON, using 42.6% fewer tokens. TOON also scored 100% on structural validation questions, where JSON scored 50%. The two studies do not contradict each other because they measure different things:

• The official benchmark asks models to read data and answer questions. The agent study also measures writing tool calls and multi-step trajectories.
• The official benchmark used hosted models: Claude Haiku 4.5, Gemini 3.6 Flash, GPT-5.4 Nano and Grok 4.5. The agent study used open-weight models from 17B to 32B parameters and states that frontier closed models were excluded for budget reasons.
• The agent study's payloads were mostly tool schemas and results, which are often small or irregular. The official benchmark is weighted toward datasets where tables apply.

Both studies support the same conclusion: TOON is a strong input format for tabular data and a risky output format.

How should I use TOON in an agent today?

Use TOON for what goes in and JSON for what comes out. Keep the tool schemas and the model's tool-call output in JSON, which is the format the model was trained on. Convert only large, uniform tool results to TOON before you add them to the context. A ticket search that returns this compact JSON:

{"results":[{"ticket":"T-1042","status":"open","priority":"high","assignee":"mlee"},{"ticket":"T-1043","status":"closed","priority":"low","assignee":"jdoe"},{"ticket":"T-1044","status":"open","priority":"medium","assignee":"mlee"}]}

goes into the next turn as this (real @toon-format/toon 4.1.1 output):

results[3]{ticket,status,priority,assignee}:
  T-1042,open,high,mlee
  T-1043,closed,low,jdoe
  T-1044,open,medium,mlee

A safe rollout checklist:

• Gate by shape. Convert only results that encode as a table, such as search hits, rows and logs. Pass small or irregular results through as JSON.
• Never ask for TOON output in tool calls, especially when the agent makes parallel calls.
• Measure trajectories, not single calls. Count total tokens per completed task, including retries, because cascades only appear at that level.
• Test your actual model. The study found results varied a lot between models. Small non-reasoning models were the most fragile.

For implementation patterns, see TOON for function calling and building an MCP server with TOON. To compare TOON with TRON and JTON, see TOON vs TRON vs JTON. To check how your own tool results encode, paste them into the JSON to TOON converter, or let your agent call the TOON MCP server directly.

Frequently Asked Questions