Skip to main content

6 posts tagged with "LLMs"

Large language models: how they work, what they cost, and where they fail.

View All Tags

What GPU Do You Need to Run an LLM Locally?

· 7 min read
Hamed El Ghoul
Machine Learning Engineer @ InSkillML

The rule that answers most of the question: a model needs roughly its parameter count multiplied by the bytes per parameter, plus about 20% overhead. A 7-billion-parameter model at 16-bit precision is 7 × 2 = 14 GB of weights, so around 17 GB of VRAM in practice. Quantize it to 4-bit and the same model fits in about 5 GB.

RAG vs Fine-tuning: How to Actually Choose

· 8 min read
Karim Benali
LLM & RAG Systems Engineer @ InSkillML

The short answer, and it is worth stating before anything else: RAG changes what the model knows, fine-tuning changes how the model behaves. Most teams reach for fine-tuning when they actually have a knowledge problem, spend six weeks on it, and end up with a model that sounds right and gets facts wrong.