Tokens

Why token counts appear at all

A tool costs an LLM's context window twice, in two unrelated ways.

First, just by existing. Completion APIs are stateless, so a client resends the full list of available tools on every single request. Each tool's name, description, and JSON Schema ride along whether the model calls it or not. A verbose schema is a standing charge against every request you make.

Second, when it actually runs. The arguments go up, the result comes back, and both occupy context for the rest of the conversation. That cost is paid once per call.

foro reports both, separately, so a tool with a huge schema and no traffic reads differently from a small tool called constantly. You can see them side by side in the Playground.

A token is not a word, and not a character

A tokenizer maps text to integers drawn from a fixed vocabulary. The method almost every current model uses is byte pair encoding: start from raw bytes, repeatedly merge whichever adjacent pair occurs most often across a large corpus, and freeze the resulting merge table. Common whole words survive as a single token. Rare words get split into pieces. Punctuation, runs of whitespace, and identifiers like user_id usually split into several.

That has a direct consequence for JSON. Braces, quotes, colons, and long snake_case parameter names are all real tokens. A schema is not free structure around your description, it is itself a meaningful part of the count.

Two properties are worth holding on to:

The count depends on the exact bytes, whitespace included. Reformatting JSON changes it. And the same text produces different counts under different vocabularies, so a token count only means something once you say which tokenizer produced it.

This is the shape foro tokenizes for a tool's schema cost, shown here with indentation for readability. On the wire it is compact, with no spaces or newlines:

{
  "type": "function",
  "function": {
    "name": "spending_by_category",
    "description": "Total spending per category for a given month.",
    "parameters": {
      "type": "object",
      "properties": { "month": { "type": "string" } },
      "required": ["month"]
    }
  }
}

BPE tokenizers and their encoding models

foro counts with a BPE tokenizer built from OpenAI's published encodings. An encoding is one frozen vocabulary plus its merge table. Each model family is tied to an encoding, so counting tokens for a model means picking the right one.

The list is open at the top: every new proprietary OpenAI model has used o200k_base since GPT-4o, so newer names not mentioned above almost certainly land there too. o200k_harmony is that same vocabulary with extra control tokens for the harmony response format, so ordinary text counts identically under both.

Every row above describes an OpenAI model. Claude, Gemini, and Llama use their own tokenizers entirely, with their own vocabularies, so none of these encodings describes what they charge you.

Why foro pins o200k_base

Because foro cannot know which tokenizer your requests will actually use.

The dashboard chat is bring your own provider: you supply a base URL, an API key, and a model string in your account settings, and any OpenAI-compatible endpoint works. There is no allowlist and no fixed model.

Your deployed servers are the same story one step removed. foro does see which client called: every MCP client announces a name and version during the handshake, and those show up as Request sources on the Metrics tab. But a client name is not a model. Knowing a call came from Claude Code or a custom agent says nothing about which model was driving it, and therefore nothing about which tokenizer priced it.

So there are two options. Guess per request, and produce a number whose meaning changes per tenant and per week. Or fix one encoding and be clear about what it is. foro fixes one. o200k_base is the encoding of OpenAI's current generation, which makes it a reasonable modern yardstick, and both of foro's counters use it so the two figures on your screen are directly comparable.

Treat these numbers as a ruler, not an invoice. They are built for comparing one tool against another, and this week's schema against last week's. They are not built to reconcile against a provider bill.

The two figures, side by side

Schema cost is a pure function of the schema, so it needs no traffic to exist: a tool nobody has ever called still has one, and it is available the moment your server is live. foro derives it by serializing a single tool to the OpenAI-compatible function shape and encoding that string.

Tokens per call are counted inside your own container, so only the numbers ever leave it. The arguments and the result text are tokenized in place and the counts are emitted; the payloads themselves are never shipped out for counting. The tokenizer and its merge data are baked into the image at build time, so counting needs no network access at runtime. If the tokenizer is unavailable the count is simply absent rather than wrong, and the dashboard shows a dash instead of a zero.

Where the estimate is loose

Both numbers are honest approximations. Four things move the real figure away from what foro shows.

Your client may not send every tool. Clients that filter, page, or expose a subset of a server's tools carry less schema than foro assumes.

Providers use different wire shapes. foro counts the OpenAI-compatible function wrapper. Anthropic's input_schema shape is roughly ten tokens of wrapper lighter for the same tool.

The two paths do not serialize identically. The schema figure is produced in JavaScript and the per-call figure in Python, and the two disagree about whitespace in JSON. Some SDKs also add fields, such as a strict-mode flag, that foro never emits.

Special tokens count as text. A payload containing something like an end-of-text marker is counted as ordinary characters rather than rejected, so a hostile or unusual payload never breaks metrics collection.

Trimming a schema

If the schema cost of a server is higher than you want, the levers are predictable, in rough order of payoff.

Descriptions are the biggest one. They are prose, they tokenize like prose, and they are resent on every request. Write them for a model, not as documentation for a human reader.

Then parameter surface: long names, deeply nested objects, and large enums all add up, and an enum of forty values costs forty values on every request. Finally, tool count itself. Splitting one flexible tool into five near-duplicates multiplies the standing cost by five while the useful work stays the same.

The Playground shows each tool's share of its server's total, so you can see which one is actually worth the trim before you start.