json2toon.co
12 min read

Build a Cost-Efficient Chatbot with Next.js & Vercel AI SDK

A step-by-step tutorial on building a chatbot using Next.js, Vercel AI SDK, and TOON for efficient tool calling and data passing.

By json2toon.co

Building an AI chatbot today is easier than ever thanks to tools like the Vercel AI SDK and Next.js. However, once you move past the "Hello World" phase and start retrieving data, calling tools, or handling complex context, you face a new problem: Cost. Today, we're going to build a chatbot that is not just functional, but fiscally responsible.

The Goal

We will build a simple chatbot that can "search" a mock product catalog. Crucially, we will use TOON (Token-Oriented Object Notation) for all internal data passing between the tool execution and the LLM context. Using TOON best practices ensures that even if our catalog grows, our token usage stays minimal.

Tech Stack

  • Framework: Next.js (App Router)
  • AI Integration: Vercel AI SDK 7 (ai, @ai-sdk/react, @ai-sdk/openai)
  • Data Format: TOON (@toon-format/toon)
  • Model: OpenAI GPT-5.6 Luna (gpt-5.6-luna, $0.20 input / $1.20 output per 1M tokens as of October 2026 per the OpenAI pricing page), or any model the AI SDK supports

Step 1: Setup and Installation

First, create a new Next.js project if you haven't already:

npx create-next-app@latest my-toon-bot
cd my-toon-bot
npm install ai @ai-sdk/openai @ai-sdk/react @toon-format/toon zod

Step 2: The Data Layer (Mock)

lib/products.ts
export const products = [
  { id: 1, name: "Eco Tumbler", price: 25.00, inStock: true },
  { id: 2, name: "Wool Beanie", price: 18.50, inStock: true },
  { id: 3, name: "Graphic Tee", price: 30.00, inStock: false },
];

export async function searchProducts(query: string) {
  return products.filter(p => p.name.toLowerCase().includes(query.toLowerCase()));
}

Step 3: The API Route (Server Side)

This is where the magic happens. We will use streamText from the Vercel AI SDK. When the model invokes the getProducts tool, we won't return JSON. We will convert the data to TOON before sending it back to the model.

app/api/chat/route.ts
import { openai } from "@ai-sdk/openai";
import {
  streamText,
  tool,
  convertToModelMessages,
  isStepCount,
  createUIMessageStreamResponse,
  toUIMessageStream,
  type UIMessage,
} from "ai";
import { z } from "zod";
import { encode } from "@toon-format/toon";
import { searchProducts } from "@/lib/products";

export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages }: { messages: UIMessage[] } = await req.json();

  const result = streamText({
    model: openai("gpt-5.6-luna"),
    messages: await convertToModelMessages(messages),
    stopWhen: isStepCount(5),
    system: `You are a helpful assistant.
Tools return data in TOON format (Token-Oriented Object Notation).
Lists of objects use a header like products[N]{id,name,price}: followed by one comma-separated row per item.
Use the data to answer user questions.`,
    tools: {
      getProducts: tool({
        description: "Search for products by name",
        inputSchema: z.object({
          query: z.string().describe("The search term"),
        }),
        execute: async ({ query }) => {
          const items = await searchProducts(query);
          
          // HERE IS THE OPTIMIZATION:
          // Instead of returning the raw objects (serialized as JSON),
          // return a TOON string. Uniform arrays become one header plus rows.
          return encode({ products: items });
        },
      }),
    },
  });

  return createUIMessageStreamResponse({
    stream: toUIMessageStream({ stream: result.stream }),
  });
}

Why do this?

By transforming the tool output to TOON, the text injected back into the conversation history (context window) is compressed. For the three mock products, the model receives:

products[3]{id,name,price,inStock}:
  1,Eco Tumbler,25,true
  2,Wool Beanie,18.5,true
  3,Graphic Tee,30,false

Field names appear once, and numbers are written in canonical form (25.00 becomes 25). On flat, uniform data like this, the current official TOON benchmark (v4.1, 5,856 LLM calls) measured 58.7% fewer tokens than JSON, and 42.6% fewer across all of its datasets, with retrieval accuracy at 72.2% for TOON vs 71.4% for JSON. If getProducts returns 50 items, a 1,500-token JSON result shrinks to roughly 620 tokens, and you save that on this turn and on every later turn that carries the history.

Step 4: The Frontend (Client Side)

The frontend remains standard. The Vercel AI SDK handles the streaming and state management. The user never sees the TOON format; they just see the chatbot's natural language response.

app/layout.tsx
import { Inter } from "next/font/google";
import "./globals.css";

const inter = Inter({ subsets: ["latin"] });

export default function RootLayout({
  children,
}: {
  children: React.ReactNode;
}) {
  return (
    <html lang="en">
      <body className={inter.className}>
        <div className="min-h-screen bg-slate-50">
          {children}
        </div>
      </body>
    </html>
  );
}
app/page.tsx
"use client";

import { useState } from "react";
import { useChat } from "@ai-sdk/react";
import { Send, Bot, User } from "lucide-react";

export default function Chat() {
  const [input, setInput] = useState("");
  const { messages, sendMessage } = useChat();

  return (
    <div className="flex flex-col w-full max-w-2xl mx-auto min-h-screen pb-32">
      <header className="sticky top-0 z-10 bg-white/80 backdrop-blur-md border-b p-4 text-center">
        <h1 className="text-xl font-bold text-link">
          TOON-Powered Assistant
        </h1>
      </header>
      
      <div className="flex-1 p-4 space-y-6 mt-4">
        {messages.length === 0 && (
          <div className="text-center text-slate-400 mt-20">
            <Bot className="w-12 h-12 mx-auto mb-4 opacity-20" />
            <p>Ask me about our product catalog!</p>
          </div>
        )}
        
        {messages.map((m) => (
          <div
            key={m.id}
            className={`flex ${m.role === "user" ? "justify-end" : "justify-start"}`}
          >
            <div className={`flex max-w-[80%] gap-3 ${m.role === "user" ? "flex-row-reverse" : "flex-row"}`}>
              <div className={`w-8 h-8 rounded-full flex items-center justify-center shrink-0 ${
                m.role === "user" ? "bg-primary text-primary-foreground" : "bg-slate-200 text-slate-600"
              }`}>
                {m.role === "user" ? <User size={16} /> : <Bot size={16} />}
              </div>
              <div className={`rounded-2xl px-4 py-2.5 shadow-sm ${
                m.role === "user" 
                  ? "bg-primary text-primary-foreground rounded-tr-none" 
                  : "bg-white border rounded-tl-none text-slate-800"
              }`}>
                {m.parts.map((part, i) =>
                  part.type === "text" ? (
                    <p key={i} className="text-sm leading-relaxed whitespace-pre-wrap">{part.text}</p>
                  ) : null
                )}
              </div>
            </div>
          </div>
        ))}
      </div>

      <div className="fixed bottom-0 left-0 right-0 p-4 bg-gradient-to-t from-slate-50 via-slate-50 to-transparent">
        <form 
          onSubmit={(e) => {
            e.preventDefault();
            sendMessage({ text: input });
            setInput("");
          }}
          className="max-w-2xl mx-auto relative group"
        >
          <input
            className="w-full pl-5 pr-12 py-4 bg-white border border-slate-200 rounded-2xl shadow-xl focus:outline-none focus:ring-2 focus:ring-primary/20 focus:border-primary transition-all text-slate-900"
            value={input}
            placeholder="Type your message..."
            onChange={(e) => setInput(e.currentTarget.value)}
          />
          <button 
            type="submit"
            className="absolute right-2 top-2 p-2.5 bg-primary text-primary-foreground rounded-xl hover:bg-primary/90 transition-colors disabled:opacity-50"
            disabled={!input.trim()}
          >
            <Send size={18} />
          </button>
        </form>
        <p className="text-[10px] text-center text-slate-400 mt-3">
          Powered by Vercel AI SDK & TOON
        </p>
      </div>
    </div>
  );
}

Conclusion & Next Steps

Congratulations! You have just built a "Chat with your Data" application that is optimized for the token economy. Here is what we accomplished:

  1. Seamless Integration: We slotted encode() right into the tool's execute function. No complex re-architecture required.
  2. Transparent Optimization: The LLM understands the format naturally (with a tiny system prompt nudge), and the user is none the wiser.
  3. Scalable Savings: As your product catalog grows or your user base expands, your token savings scale linearly with them.

This pattern applies to any structured data: user profiles, transaction histories, analytics reports, or documentation chunks. Wherever you have lists of objects, TOON is your wallet's best friend.

Get the SDKBack to Home

Recommended Reading

Token-Efficient Apps with TOON and the Vercel AI SDK

How to format context as TOON inside Vercel AI SDK apps to cut input tokens—while keeping generateObject and tool calls on JSON for reliable structured output.

•8 min read

Migrating to TOON v4: Breaking Changes and Upgrade Checklist

Upgrading from TOON v2/v3 to v4? Key folding is gone, # lines are now comments, and decoders must upgrade first. A step-by-step checklist with fixes.

•8 min read

Few-Shot Prompting with TOON: Cheaper, Clearer Examples

Few-shot examples repeat in every prompt, so their format compounds. TOON-encoded example tables cut tokens; output examples should match your target format.

•8 min read
TutorialNext.jsVercel AI SDKChatbotTOONCost Efficiency