How much RAM does a local AI need?
Less than the cloud era taught you to assume. The model sets the number, and quantization has been quietly shrinking it for years.
It depends on the model size, and the answer is gentler than most people expect. A 7B to 8B model quantized to 4 bits fits in about 5 gigabytes, so 16 gigabytes of system memory is a comfortable working setup, and 8 gigabytes handles the smaller end. The documents you index barely register. What limits most people is GPU memory, not RAM.
The model sets the number.
A model's appetite is roughly its parameter count times the bits you spend storing each one. That is why quantization is the whole story here: a 7B model at full precision wants around 14 gigabytes, and the same model stored at 4 bits wants about 5, which is a machine most people already own. Models ship as single files with the bit depth in the name, so choosing a GGUF and choosing your memory budget are the same decision. Add a few gigabytes for the running application and the context, and 16 GB of system memory carries the class of models that does honest research work. 8 GB carries the smaller end; 32 GB buys headroom for the big ones, and nothing about the basic loop requires it.
Your library barely counts.
The instinct from cloud tools is that more documents means more memory, because remote services meter everything. Locally the arithmetic is different. The text of a thousand PDFs is a few gigabytes at most, the embedding vectors that make it searchable are smaller still, and retrieval runs on the CPU while the model does the heavy lifting. A library that would cost a monthly subscription on someone else's servers is a folder on yours, and growing it does not move the hardware requirement. The model is the consumer; the library is a rounding error.
Check the GPU before the RAM.
On most machines the binding constraint is not system memory but GPU memory, which is smaller and faster and fills first. A 4 GB card cannot hold a whole quantized model, so the tool splits the work: the layers that fit go to the GPU, the rest run on the CPU with system RAM as their workspace. That split is where RAM earns its keep, and it is also where the penalty for too little shows up. If the machine starts swapping to disk, generation slows to a crawl, which is the real sign you have crossed the line. A GTX 1650 class rig with 16 GB of RAM sits comfortably on the right side of it.