Every score is worked out from the vendor's own pages · How we score →Disclosure
SAASINSPECTOR
AI concepts

Token

The small chunk of text an AI model reads and writes in, and the unit most AI usage is priced by.

01In plain English

A token is a small piece of text, often part of a word, that a language model reads and writes in. A short English word is usually one token, and a longer one can be several. Models count their input and output in tokens.

02How it works

Before a model can read your text, a tokenizer cuts it into pieces called tokens. Common short words such as the or cat are usually one token each, while a long or rare word is split into several parts. Punctuation and spaces count too, and so do the instructions you do not see. Every request is measured in two ways: the tokens you send in, and the tokens the model writes back. The provider adds them up and bills for the total, or counts them against your plan's limit. As a rough guide, a page of ordinary English is a few hundred tokens. A tool priced in credits is usually converting tokens into a simpler number for you.

03Why it matters when you are choosing

AI providers price usage and set limits in tokens, so they decide both what a tool costs to run and how much text it can handle at once.

04What to check

Ask the vendor

When a tool prices by credits or usage, check how that maps to tokens or words, and what happens when you hit a limit.

05Where you will meet it

06Common questions

How many words is a token?

In English, a token is about three quarters of a word, so a hundred tokens is roughly seventy five words. The ratio changes with the language and the content. Code and languages that use other alphabets often take more tokens for the same meaning, which makes them more expensive.

Why does the AI cut off in the middle of a long answer?

Usually it has reached a limit, either the maximum reply length or the total the model can hold at one time. Asking it to continue often works. For long jobs, split the work into parts so the full text, including your earlier messages, fits inside the limit.

Do I pay for tokens in a flat-rate chatbot subscription?

Not directly. A monthly plan hides the tokens behind a usage allowance, which may be a message cap or a fair-use limit. Tokens appear on the bill when you use an interface for developers, where you pay per amount of text processed.

← Back to the glossary