GraphicCardPrices

Local LLM VRAM calculator

New to local AI? Start with the beginner GPU buying guide.

Local LLM VRAM planner

Estimate a VRAM planning target for local inference. It answers capacity, not model speed or tokens per second.

Plan for 7.51 GiB total VRAM (3.26 GiB ideal weights + 2 GiB KV cache + 1 GiB runtime, then 20% headroom).

The default 7B, 4-bit values (2 GiB KV cache, 1 GiB runtime, and 20% headroom) are illustrative starting points, not representative requirements for every 7B model.

Worked 4-bit examples

Illustrations from the same formula, using a 2 GiB KV-cache allowance, 1 GiB runtime allowance and 20% headroom for every row. These allowances are examples only, not representative requirements for models of these sizes. Try a row to load its exact settings into the calculator, adjust the allowances, and copy a link to your own estimate.

Illustrative 4-bit weight and VRAM planning examples for 7B, 8B, 13B, 14B, 32B and 70B. All memory values are GiB.
ParametersIdeal weights (GiB)KV cache (GiB)Runtime (GiB)HeadroomPlanning target (GiB)Try settings
7B3.262120%7.51Try 7B settings
8B3.732120%8.07Try 8B settings
13B6.052120%10.86Try 13B settings
14B6.522120%11.42Try 14B settings
32B14.92120%21.48Try 32B settings
70B32.62120%42.72Try 70B settings

Reverse planning by usable VRAM budget

These are usable VRAM budgets available to this calculation, not marketing GPU capacity. With 2 GiB KV-cache allowance, 1 GiB runtime-overhead allowance and 20% headroom, the table gives an ideal arithmetic upper bound for parameter count. It excludes quantization metadata, model-specific overhead and runtime compatibility.

Ideal 4-bit upper-bound parameter count by usable VRAM budget. Weight memory values are GiB.
Usable VRAM budgetIdeal weight room (GiB)Ideal maximum parameters (4-bit)
8 GiB usable3.677.87B ideal max parameters
12 GiB usable715.03B ideal max parameters
16 GiB usable10.3322.19B ideal max parameters
24 GiB usable1736.51B ideal max parameters
32 GiB usable23.6750.82B ideal max parameters
48 GiB usable3779.46B ideal max parameters

Planning assumptions: ideal weight memory is parameters × bits per weight ÷ 8, converted to GiB. KV cache and runtime overhead are allowances you enter; they are not measured by this page. KV-cache needs vary with architecture, context length, batch size, and parallelism, so they cannot be inferred from parameter count alone.

This estimate excludes quantization metadata, mixed precision, temporary buffers, model-specific kernels, operating-system display use, and any guarantee that a model will fit.

Background: Hugging Face quantization documentation; Hugging Face cache explanation; llama.cpp performance notes.

Choose allowances for your workload

Start with the model's actual parameter count and weight format. Use a KV-cache allowance based on its architecture, intended context length and concurrent requests; longer sequences can need more cache. Reserve runtime memory separately. Measure a representative run in your chosen runtime before treating an estimate as a purchase requirement.

Can I combine two GPUs or use system RAM?

This calculator models one usable memory budget. Adding the advertised capacities of two cards does not establish that a runtime can split your model between them. CPU offloading changes where data is stored and how it is transferred; check your runtime's support and measure performance separately.

Maintained by Triftan. The formulas are planning arithmetic, not hardware benchmarks. About GraphicCardPrices.

This is a memory estimate, not a compatibility check or a promise that a model will fit. For CUDA workflows, confirm GPU and software support in your selected runtime's documentation. Compare observed listings for 16GB+ GPUs, NVIDIA 24GB+ GPUs, 32GB+ GPUs, or review the Amazon US price-by-VRAM sample.