What fits in 8 GiB of VRAM?

Keep the first configuration small

An 8 GiB budget makes the gap between download size and runtime allocation visible. Qwen2.5-7B and Qwen3-8B are the smaller checkpoints in this comparison, but they have different cache footprints. Start with one short-context sequence. A 14B weight file using aggressive quantization is not evidence that the full workload fits.

If the result is tight, reduce context before accepting a configuration with no spare memory. Monitor display use, close GPU-accelerated applications, and compare the actual runtime allocation with the assumed 1.5 GiB allowance. This catalog does not cover smaller 1B–4B models, which are also worth evaluating when 7B–8B is too constrained.

Before committing to a configuration

1. Confirm the exact checkpoint and quantization. Model-family names are not interchangeable.
2. Measure free GPU memory after the desktop and applications are running.
3. Allow for the full conversation, including generated output, rather than the prompt alone.
4. Run the intended workload and record peak allocation. Adjust the calculator assumptions to match.

“Exceeds budget” means the full-GPU configuration exceeds this example. CPU offloading can change that result. For Apple unified memory, reserve space for the operating system and application workload; this discrete-GPU budget table is not a Metal allocation prediction.

How the estimate works

Start with the stored weights, add the conversation cache, then allow space for the runtime. The display and other applications consume a separate reserve. The result assumes the whole model and its cache stay on one GPU.

Weights (GiB) ≈ parameters × assumed bits per weight ÷ 8 ÷ 2³⁰
KV cache (GiB) = 2 × layers × KV heads × head dimension × bytes per cached value × context tokens × parallel sequences ÷ 2³⁰
Total allocation = weights + KV cache + runtime allowance

Weight presets use 4.8, 5.7, 6.6 and 8.5 effective bits per parameter for Q4_K_M, Q5_K_M, Q6_K and Q8_0. These are planning assumptions, not exact file sizes. Tensor mixtures, unquantized tensors and container metadata vary by conversion. Use the advanced weight override when you have measured the checkpoint allocation.

The cache calculation models a full dense GQA cache at the selected capacity. FP16 uses 2 bytes per value. The Q8_0 and Q4_0 estimates include block overhead (34 and 18 bytes per 32 values). Backend padding and allocation strategies can increase usage. Your runtime must support the chosen cache type and kernels; smaller caches can affect output quality.

What the fit labels mean

Usable memory is GPU capacity minus the display/other-app reserve. “Room in estimate” leaves at least 10% of that usable budget unused. “Tight budget” means the estimate fits but has less headroom. “Exceeds budget” means this full-GPU configuration exceeds the selected allocation. Partial CPU offloading may still work, with different performance.

Check it on your machine

Use the exact quantized file, runtime version and context settings you plan to deploy. Load one sequence first, inspect the runtime's memory report and GPU monitor, and then test a conversation near the intended context length. Record peak use during both prompt processing and generation before increasing concurrency.

In Ollama, ollama ps reports the processor split and allocated context. A model partially offloaded to CPU can run even when it exceeds the GPU budget. Do not interpret successful loading alone as proof of full GPU residency.

Where this tool stops

This is inference planning for the six named dense text checkpoints. It does not estimate training, fine-tuning, image generation, MLA, sliding-window caches, prefix sharing, multi-GPU placement, or Apple unified-memory limits. No tokens-per-second claims are made. Context is bounded to the presets listed in the source notes; fitting memory does not validate model quality at that length.