What is a token limit?
The hard ceiling on how much text a model can read and write in a single request.
A token limit is the maximum number of tokens a model can process in one request, and everything shares the budget: your question, the standing instructions, any retrieved passages, and the answer being written. The limit exists because the model reads the whole input at once, through a window fixed when the model was built. Exceeding it means truncation, a squeezed answer or a refusal, which is why grounded tools select what goes in.
One number, many budgets.
Every piece of text in a request is first broken into tokens, and all of them draw on the same account. The instructions at the front spend some, the retrieved passages spend more, your question spends some, and whatever room is left is what the answer can occupy. A request that looks short in words can be heavy in tokens once a few documents ride along with it. This is why a tool that works over your documents thinks in tokens from the start, and why the size of what it retrieves is a real design decision rather than a cosmetic one.
Where the limit comes from.
The ceiling is set by the model's architecture and its context window, both fixed when training ends. A model that reads 8,000 tokens cannot read 9,000 no matter how you phrase the request; no prompt changes it, and the file of weights carries the number inside it. Bigger windows exist, but they raise memory and speed costs on the hardware that runs the model, which is why local tools often work with modest windows and lean on retrieval to choose what earns a place inside one.
What happens when you hit it.
Long documents do not fit whole, so systems that handle them cut documents into passages and send only the ones that matter, which is exactly how answers over long PDFs stay inside the budget. The alternative, pushing everything in and hoping, produces the failure modes above: uneven attention near the edges, summaries done in sections, or text that never arrives. The limit is not a flaw to defeat. It is the reason retrieval exists as a discipline, and a tool that respects it will tell you what it sent.