Tokens
A token is the atomic unit of text a model reads and writes. Before a model can process your input, the text is broken into tokens; the model then generates its reply one token at a time. Tokens are the currency of everything a model does: limits, cost and speed are all measured in them.
A token is roughly word-sized, but not exactly. Common words are often a single token, while longer or rarer words are split into pieces. As a rough guide, one token is about four characters of English, so 1,000 tokens is roughly 750 words. Punctuation, spaces and line breaks count too.
Splitting text into tokens (called tokenization) is language- and model-specific. The same sentence can become a different number of tokens depending on the model, and non-English text often uses more tokens per word.
Why tokens matter
Almost every practical limit you meet comes back to tokens:
- Context limits: the context window is measured in tokens, and it is shared between the text you send in and the answer the model writes back.
- Cost: providers bill per token, and usually charge more for the tokens a model generates (output) than for the tokens you send (input).
- Speed: a model produces output token by token, so longer answers take longer to appear.
Tokens in Rational AI
Wherever the platform talks about how much text a model can handle, or how much knowledge to retrieve for a request, it is ultimately budgeting tokens. Retrieving only the most relevant resources through RAG, rather than sending everything, keeps each request within the model's token budget and controls cost.