What is VRAM?
The graphics card's own memory, separate from system RAM, and the resource a local model is really trying to fit in.
VRAM is the memory built into a graphics card, separate from the system's RAM, and for local AI it is where the model wants to live. It is smaller than system memory, much faster, and directly wired to the processor that runs the model's math. When a model fits in VRAM, answers come back in seconds; when it does not, the tool splits the work with your CPU and everything slows.
A second, faster memory.
VRAM, video random access memory, is the memory mounted on a graphics card, beside the processor it feeds. Its defining property is bandwidth: the path between the card's processor and its memory is wide and short, which is why generation speed tracks VRAM so closely. System RAM holds more but feeds the CPU through a slower path. This is why a GPU speeds long answers even when the same model technically runs on a CPU: the model's numbers stream from VRAM thousands of times per second during generation.
Why the model wants to live there.
Every token the model writes requires reading the model's weights. If those weights sit in VRAM, the GPU reads them at full speed; if they sit in system RAM, the CPU reads them at a fraction of it. This is why quantization and VRAM are two halves of one decision: fewer bits per weight squeezes a larger model into the same 4 GB. A GTX 1650 class card holds a useful stack of quantized models, which is the whole basis of running research AI on a GTX 1650.
When it does not fit.
Tools handle an oversized model by splitting it: the layers that fit go to the GPU, the rest run on the CPU with system RAM as their workspace. The answer still arrives and its quality is identical; only the speed changes, because the split adds a trip through the slower memory on every token. So the practical question is never whether a model can run but how fast, and the RAM arithmetic is what decides where the split lands.
What else lives in VRAM.
The model's weights are the biggest tenant, not the only one. The working state that carries a long conversation, the text of a document currently in context, and the scratch buffers used while generating all draw on the same 4 GB. This is why a small model fed a long document can still slow down: the document changes nothing about answer quality, but it competes for the same memory as everything else. A context-length cap is, in hardware terms, a decision about how much VRAM the conversation itself may occupy.
Reading your own numbers.
Windows Task Manager's GPU page shows two memory figures, and only one of them matters here. Dedicated GPU memory is the real VRAM, the budget a local model is trying to fit inside. Shared GPU memory is an accounting label for system RAM the GPU may borrow; it looks like extra headroom and behaves like the slow path. When a model runs mostly out of shared memory, generation speed collapses even though everything technically works. The command-line tool nvidia-smi tells the same story with per-process detail, which is useful when you want to know whether a model genuinely fit or quietly split.
When there is no card at all.
A laptop with integrated graphics has no VRAM of its own; its graphics processor draws from system RAM over the same path the CPU uses. Local AI still runs there, and a small quantized model is the right shape for that machine, but the honest expectation is CPU-class speed rather than card-class speed. The rule of thumb holds in every case: the model wants to live in dedicated VRAM, and each step it takes away from that memory, the meter slows.