01 / Model weights
Parameters × estimated bits per weight, converted to GiB. Quantized formats include an approximate allowance; actual files can differ.
Local AI · Memory estimator
Will your GPU run it?
Find out how much memory a language model may need. Compare your GPU, adjust the context, and see where the memory goes.
● Private by design · No sign-up
Build your scenario
Estimates for single-GPU text generation (inference).
Quantized sizes vary by file.
File size is a useful proxy; actual loaded weights may differ.
Optional: enter the available VRAM reported by your system. Other apps and the driver reduce what is free.
Depends on runtime support.
Advanced settings
Your memory estimate
Model weights + KV cache + runtime allowance. All values in GiB.
Estimated GPU memoryLikely to fit
— GiB
—
Model weights—
KV cache—
Runtime allowance—
GPU memory used for comparison—
Estimated remaining—
These are estimates, not a guarantee. Other apps, model files and inference software change actual memory use.
Under the hood
Know what fits. Know why.
02 / Context & cache
For supported transformer architectures: 2 × layers × KV heads × head dimension × context × sequences × cache bytes.
03 / Runtime headroom
Extra memory is reserved for the runtime and temporary buffers. It varies with software and workload, so adjust this value if needed.
The details
Questions, answered.
Why can the available GPU memory be lower?
Preset GPU capacities are approximate nominal VRAM in GiB. Drivers, the display and other applications reserve memory. For a better comparison, enter your available VRAM as reported by your system.
Does this estimate guarantee the model will run?
No. It is an estimate for text-generation inference with one GPU. Runtime buffers, model files, driver reservations, other applications and architecture-specific behavior all affect actual usage. A fit also does not predict speed.
Can I run a model larger than my GPU memory?
Some tools can offload part of a model to system RAM, usually with a speed trade-off. This calculator estimates a full single-GPU fit and does not estimate offloaded performance.
What is KV cache?
It stores attention data for previously processed tokens. Longer context and more concurrent sequences generally require more cache memory.
What models and formats are supported?
Built-in presets cover a small selection of dense Llama and Qwen models. The formula is an approximation for standard transformer KV caches, not a general estimator for image, video, training, hybrid attention or every quantization format.
Method reference: NVIDIA: LLM inference and KV caching ↗. Built-in model architecture values should be rechecked when models or software change.