What GPU Do You Need to Run an LLM Locally?
· 7 min read
The rule that answers most of the question: a model needs roughly its parameter count multiplied by the bytes per parameter, plus about 20% overhead. A 7-billion-parameter model at 16-bit precision is 7 × 2 = 14 GB of weights, so around 17 GB of VRAM in practice. Quantize it to 4-bit and the same model fits in about 5 GB.