Find where the failure starts
Save the first CUDA error, the few lines before it, and the exact command you ran. Run nvidia-smi to see available memory on the target device. A GPU sold with 12 GiB may have less available when the desktop or other processes are using it.
| When it fails | What to try first |
|---|---|
| While loading weights or moving layers to CUDA | Lower --gpu-layers or try a smaller GGUF. Moving layers to the CPU trades GPU memory for system RAM and may reduce speed. |
| While initializing context or KV cache | Reduce --ctx-size and test with one server slot. Compare the old and new context settings in the KV-cache calculator. |
| While setting up compute buffers or processing a long prompt | For a text-generation model, try lowering --ubatch-size while keeping it no larger than --batch-size. Retest the same prompt and check whether prompt processing still completes. |
| Only when multiple requests overlap | Run one request at a time with --parallel 1. Increase concurrency only after measuring headroom at the context length you need. |
Treat those messages as clues. A build can reserve more than one kind of buffer during startup. Use the last successful allocation and the first failure together to pick your next test.
Start with one model and one short request
This example uses a CUDA-enabled llama-server build and a local text GGUF. On Windows PowerShell, run .\llama-server.exe if the binary is in your current folder. Use your actual model path. Check llama-server --help first because flag names and automatic fit behavior vary by build.
llama-server -m ./model.gguf --host 127.0.0.1 --port 8080 --ctx-size 2048 --parallel 1 --gpu-layers 8 --batch-size 256 --ubatch-size 128Eight GPU layers is a test starting point. If it still fails while placing weights, try fewer layers or a smaller file, taking available system RAM into account. If it loads, open http://127.0.0.1:8080 in your browser and submit one short prompt. Save the actual layer count and context reported in the logs, especially if your build adjusts settings automatically.
A 12 GiB example
Imagine the GPU has 10.5 GiB free before launch. Your weights need about 8 GiB on the GPU, and your planned context and compute buffers bring the estimate to 11 GiB. The file fits on its own, but the run is already half a GiB short before any additional headroom. These are illustrative numbers you can replace with your own.
If the error happens during context setup, try 4,096 tokens in place of 8,192 and leave the GPU layer count alone. If that succeeds, repeat with a representative prompt. If you need the original context, try smaller weights or less GPU offload and measure the speed tradeoff. The VRAM calculator helps you lay out the budget in one place.
A fix has to survive your real workload
Change one setting at a time and restart the test server between runs. Record model and quantization, llama.cpp build, free device memory before launch, requested and effective context, GPU layer count, and whether a short prompt and a typical prompt both complete. Add concurrency back only if you use it. A server that opens successfully has cleared startup, and a representative request tells you whether the settings solve the problem you came here for.
The command and memory numbers are illustrative troubleshooting examples. Check the settings against your model, build and available memory.