Token
The unit of text a language model reads and writes. Model prices are usually quoted per million tokens, with input and output priced separately.
A token is the unit of text that a language model reads and writes. It is usually a fragment of a word, so a short word may be one token and a long or unusual word may be several. Spaces and punctuation count too. Model providers measure input and output in tokens and charge for each, often at different rates.
Tokens are the basis of cost and of limits. Every call has a ceiling on how many tokens it can contain, and every call has a price that scales with the tokens it uses.
In practice
I estimate the token use of a workflow before I choose a model. For each step I count the instructions, the retrieved material, the conversation history and the expected output. Multiplying that by the number of steps and the volume of tasks gives an estimate that is usually better than a guess from the price list. Prices change, so the estimate must cite the price table used and its date.
Token counts also explain latency and quality. Longer inputs are slower and can dilute the parts of the input the model should attend to.
Example
[Composite] A document workflow sent the full contract to the model at every step. Most of the cost came from repeating the same contract text. Sending only the relevant clause for each step, and caching the stable instructions, reduced the token count per task without a drop in the evaluation score.
Common mistakes
- Estimating cost from one call instead of the whole workflow.
- Ignoring output tokens. Long answers cost more than short ones.
- Quoting a price without its date. Model prices change.
Related terms
Cite this definition
Token. Mayur Patil, mayurpatil.ai, updated 2026-11-01. https://mayurpatil.ai/glossary/token/