Feeding SQL Query Results to LLMs: TOON vs JSON Result Sets

TOON is the best format to send SQL query results to an LLM. A result set is a uniform table — the exact data shape TOON is built for — and the official benchmark shows up to 58.8% fewer tokens versus a JSON array of row objects with 99.6% field retrieval accuracy. Do your aggregation and filtering in SQL; send only the computed result set as TOON.

Why SQL Result Sets Are TOON's Best Case

Every SQL SELECT produces the same structural pattern: a fixed set of column names repeated across every row. In a JSON array of objects, those column names are serialized once per row — pure redundancy that a database would never store but that LLM tokenizers have to process token by token.

TOON was designed to eliminate exactly this overhead. The header line declares the array length and field names once, and each subsequent row lists only the bare values. According to the official toonformat.dev benchmarks — 5,016 LLM calls across 209 questions, six formats, and four models — flat uniform tables in TOON use 58.8% fewer tokens than their JSON equivalents (67,778 vs 164,452 tokens). The overall figure across all data shapes is 39.9% fewer tokens at 76.4% retrieval accuracy, delivering 27.7 accuracy-points per 1,000 tokens versus JSON's 16.4.

A SQL result set sitting in memory is already in the uniform-table shape. You are not transforming data into a contrived format; you are choosing the serialization that matches what you already have.

JSON Array of Rows vs. TOON: A Concrete Example

Consider a simple query result: five orders with four columns. Here is the same data serialized as a JSON array of objects and as a TOON table block.

-- SQL query
SELECT order_id, customer, total_usd, status
FROM orders
WHERE shipped_at >= '2026-01-01'
LIMIT 5;

-- JSON result (array of row objects) — column names repeat on every row
[
  {"order_id": 1001, "customer": "Alice",   "total_usd": 149.99, "status": "shipped"},
  {"order_id": 1002, "customer": "Bob",     "total_usd":  49.00, "status": "pending"},
  {"order_id": 1003, "customer": "Carol",   "total_usd": 320.50, "status": "shipped"},
  {"order_id": 1004, "customer": "David",   "total_usd":  89.95, "status": "cancelled"},
  {"order_id": 1005, "customer": "Eve",     "total_usd": 210.00, "status": "shipped"}
]

-- TOON result — column names declared once; rows list bare values
orders[5]{order_id,customer,total_usd,status}:
  1001, Alice,  149.99, shipped
  1002, Bob,     49.00, pending
  1003, Carol,  320.50, shipped
  1004, David,   89.95, cancelled
  1005, Eve,    210.00, shipped

The JSON version costs roughly 130 tokens. The TOON version costs roughly 55 tokens — a 58% reduction that matches the benchmark exactly. At scale, a 1,000-row result set that would consume 26,000 tokens as JSON drops to around 10,800 tokens in TOON.

The header line orders[5]{order_id,customer,total_usd,status}: also gives the model an explicit row count and field schema to validate against — something a raw JSON array does not provide.

Do Aggregation and Filtering in SQL, Not in the LLM

It is tempting to send a large raw result set and ask the LLM to compute aggregates or apply filters. The accuracy data argues strongly against this.

The same toonformat.dev benchmark breaks down TOON accuracy by question type:

• Field retrieval: 99.6% — essentially perfect
• Structure awareness: 89.0%
• Structural validation: 70.0%
• Aggregation (COUNT, SUM, AVG, etc.): 61.9%
• Filtering (WHERE-style conditions): 56.8%

A 56–62% accuracy rate on tasks your database can handle with 100% reliability is not a trade-off worth making. Write the WHERE, GROUP BY, and HAVING clauses in SQL, send the database the query, and hand the LLM only the computed result. The LLM's job is to read and reason about what the query returned, not to re-implement database operations in-context.

This is the correct division of labor: SQL for computation, TOON for communication. For a deeper look at how TOON handles different accuracy tasks compared to other formats, see the CSV vs TOON comparison.

JSON Result Set vs. CSV vs. TOON: Which Format to Use When

All three formats can carry SQL result sets. The right choice depends on your nesting requirements, whether you want an explicit row count, and whether your pipeline already handles CSV or JSON parsing.

CSV has zero structural overhead on purely flat data and every model understands it without instructions. TOON wins when you need an explicit row count and field header — giving the model something to validate against — or when any column contains a non-string value that CSV would silently coerce. For a thorough breakdown of the CSV versus TOON boundary, read the CSV vs TOON guide.

How to Convert a SQL Result Set to TOON

Most application code serializes query results to JSON by default. Converting that to TOON before injecting it into a prompt takes one extra step. The free json2toon.co converter handles JSON arrays of row objects and outputs valid TOON table blocks. Paste your JSON result, select TOON as the target, and copy the output directly into your prompt.

For programmatic use, the @toon-format/toon npm package serializes JavaScript arrays of objects directly. In Python, a list of dicts from a database cursor maps naturally to TOON's tabular block. The key step in both cases is to run any computed fields in SQL first, so the object you serialize is already the final answer rather than raw data the LLM must process further.

For large result sets that need streaming or a built-in query API, see our guide on optimizing API costs with TOON.

The "Prompt Tax" Limit: When JSON Still Wins

TOON requires brief format instructions the first time a model encounters it. A February 2026 arXiv paper (2603.03306) on TOON versus JSON found this prompt tax can outweigh savings on very small payloads. The break-even point sits around 10–20 rows for a four-column schema assuming roughly 100 tokens of format instructions.

For SQL result sets this threshold is easy to reason about: if your query routinely returns fewer than 10 rows, stay with JSON or CSV. If it returns 50 rows or more, TOON's token reduction pays for the instruction overhead many times over. Result sets in the 10–50 row range are the gray zone — measure both and pick the smaller output.

The same arXiv paper also confirmed that for generation tasks — asking the LLM to output structured data — plain JSON remains superior. Use TOON for feeding results in; use JSON or a constrained output schema for structured responses back. See the JSON vs TOON deep-dive for a full treatment of the input/output split.

What Format Should I Use for SQL Results When Building AI Agents?

In agent architectures, a tool call to a database typically returns a result set that flows directly into the next LLM call as context. Each row costs tokens on every inference step, so format choice compounds across multi-turn conversations. TOON's 58.8% reduction on flat tables translates directly to a smaller context window usage per agent step, which lowers cost and reduces the risk of hitting context limits on long-running agents.

The practical pattern is: wrap your database tool to return TOON instead of JSON, include a one-time format instruction in the system prompt, and let the LLM read TOON natively throughout the session. The format instruction is paid once; the token savings accrue on every database tool call.

For an introduction to TOON's syntax and design, see what is TOON.

Frequently Asked Questions