Why does AI forget the beginning of long documents?
The detail was in the text. The model read all of it, but it did not read all of it equally.
A model reads everything in its context window at once, and its attention is not uniform across that text: in very long inputs, the start and the middle get less reliable attention than the end. The details were present; they just got less weight. The fix is not a bigger window but retrieval, which places the passages that matter where the model reads best, with citations back to their source.
Everything has to fit in one reading.
A model has no memory between the pages of a document, only the context window, the single span of text it can attend to while writing its answer. Everything the question needs must sit inside that span: the document, the question, whatever instructions apply. When a document fits comfortably, the model reads it whole and this problem never appears. When the text stretches toward the window's edge, the reading degrades quietly, and the first symptom is exactly the one people notice: facts from early pages go missing from the answer.
Attention is not uniform.
Attending to context is a weighting, not a recording. Every token gets some weight, but the weights are not equal, and in long inputs the middle of the text tends to be read least reliably, with the end strongest because it sits nearest the answer being written. This is sometimes called the lost-in-the-middle pattern, and it is a property of how attention distributes over long spans, not a bug a setting removes. It is also why a model can quote a document's conclusion accurately while filling an early chapter's detail with a confident guess: the material was in the window, and still effectively unread.
Why retrieval is the fix.
If reading everything makes parts of it unread, read less and read it well. That is what retrieval-augmented generation does: the document is split into passages at natural boundaries, and for each question only the handful that bear on it are placed in front of the model, where attention is sharpest. The passage boundaries decide whether those pieces make sense cold, and the citations let you check the answer against the exact text it came from. A question answered from a handful of well-chosen passages beats a question answered from everything, half-attended.