Local LLM VRAM calculator
New to local AI? Start with the beginner GPU buying guide.
Local LLM VRAM planner
Estimate a VRAM planning target for local inference. It answers capacity, not model speed or tokens per second.
The default 7B, 4-bit values (2 GiB KV cache, 1 GiB runtime, and 20% headroom) are illustrative starting points, not representative requirements for every 7B model.
Worked 4-bit examples
Illustrations from the same formula, using a 2 GiB KV-cache allowance, 1 GiB runtime allowance and 20% headroom for every row. These allowances are examples only, not representative requirements for models of these sizes. Try a row to load its exact settings into the calculator, adjust the allowances, and copy a link to your own estimate.
| Parameters | Ideal weights (GiB) | KV cache (GiB) | Runtime (GiB) | Headroom | Planning target (GiB) | Try settings |
|---|---|---|---|---|---|---|
| 7B | 3.26 | 2 | 1 | 20% | 7.51 | Try 7B settings |
| 8B | 3.73 | 2 | 1 | 20% | 8.07 | Try 8B settings |
| 13B | 6.05 | 2 | 1 | 20% | 10.86 | Try 13B settings |
| 14B | 6.52 | 2 | 1 | 20% | 11.42 | Try 14B settings |
| 32B | 14.9 | 2 | 1 | 20% | 21.48 | Try 32B settings |
| 70B | 32.6 | 2 | 1 | 20% | 42.72 | Try 70B settings |
Reverse planning by usable VRAM budget
These are usable VRAM budgets available to this calculation, not marketing GPU capacity. With 2 GiB KV-cache allowance, 1 GiB runtime-overhead allowance and 20% headroom, the table gives an ideal arithmetic upper bound for parameter count. It excludes quantization metadata, model-specific overhead and runtime compatibility.
| Usable VRAM budget | Ideal weight room (GiB) | Ideal maximum parameters (4-bit) |
|---|---|---|
| 8 GiB usable | 3.67 | 7.87B ideal max parameters |
| 12 GiB usable | 7 | 15.03B ideal max parameters |
| 16 GiB usable | 10.33 | 22.19B ideal max parameters |
| 24 GiB usable | 17 | 36.51B ideal max parameters |
| 32 GiB usable | 23.67 | 50.82B ideal max parameters |
| 48 GiB usable | 37 | 79.46B ideal max parameters |
Planning assumptions: ideal weight memory is parameters × bits per weight ÷ 8, converted to GiB. KV cache and runtime overhead are allowances you enter; they are not measured by this page. KV-cache needs vary with architecture, context length, batch size, and parallelism, so they cannot be inferred from parameter count alone.
This estimate excludes quantization metadata, mixed precision, temporary buffers, model-specific kernels, operating-system display use, and any guarantee that a model will fit.
Background: Hugging Face quantization documentation; Hugging Face cache explanation; llama.cpp performance notes.
Choose allowances for your workload
Start with the model's actual parameter count and weight format. Use a KV-cache allowance based on its architecture, intended context length and concurrent requests; longer sequences can need more cache. Reserve runtime memory separately. Measure a representative run in your chosen runtime before treating an estimate as a purchase requirement.
Can I combine two GPUs or use system RAM?
This calculator models one usable memory budget. Adding the advertised capacities of two cards does not establish that a runtime can split your model between them. CPU offloading changes where data is stored and how it is transferred; check your runtime's support and measure performance separately.
Maintained by Triftan. The formulas are planning arithmetic, not hardware benchmarks. About GraphicCardPrices.
This is a memory estimate, not a compatibility check or a promise that a model will fit. For CUDA workflows, confirm GPU and software support in your selected runtime's documentation. Compare observed listings for 16GB+ GPUs, NVIDIA 24GB+ GPUs, 32GB+ GPUs, or review the Amazon US price-by-VRAM sample.