Module 4 — Parameter-efficient fine-tuning

Module 3 closed on an uncomfortable arithmetic: 84 GB and a hundred dollars per attempt, for a task that fundamentally does not need every weight to move. This module is the family of answers to that arithmetic, collected under the label PEFT — parameter-efficient fine-tuning. The core idea is simple enough that it can be stated in one sentence: freeze the base model, and train a small set of extra parameters that adapt it.

The idea in one picture

A pretrained language model has already learned language, reasoning, formatting and refusal behaviors. What we want to add is thin: a task-specific twist that steers the same machinery toward our meeting-minutes output. It would be surprising if that twist required moving every parameter — and it does not. The empirical finding behind PEFT is that a fine-tune's useful signal lives in a very low-dimensional subspace of the parameter space. If we can identify that subspace ahead of time, we can train only the coordinates that live in it and leave the base weights alone.

Three families of methods sit under the PEFT umbrella. They differ in where the extra parameters are injected, but they share the same skeleton.

Adapters: a bottleneck inside each block

The original adapter, introduced by Houlsby and colleagues in 2019, inserts a small two-layer network inside each transformer block. A down-projection to a low dimension, a non-linearity, an up-projection back to the model width, and a residual connection so the block behaves like the identity at initialization.

class Adapter(nn.Module):
    def __init__(self, d_model, bottleneck=64):
        super().__init__()
        self.down = nn.Linear(d_model, bottleneck)
        self.up   = nn.Linear(bottleneck, d_model)
        self.act  = nn.GELU()

    def forward(self, x):
        return x + self.up(self.act(self.down(x)))  # residual: identity at start

The base weights are frozen; only the two small linear layers per block are trained. For a 32-layer 7B model with a bottleneck of 64, that is on the order of a few million trainable parameters — a thousand times fewer than the full model.

Adapters work. Their weakness is inference latency: each block now runs one extra forward pass, and at scale that adds up. LoRA (module 5) will remove that penalty entirely by merging the adapter into the base weights.

Prefix tuning and prompt tuning: parameters as virtual tokens

A second family avoids adding new modules altogether. Instead, it prepends a small sequence of learnable virtual tokens to the input, and trains only their embeddings. The rest of the model is frozen. The model sees a prompt that starts with tokens it did not know, and those tokens are shaped by training to nudge every subsequent computation toward the target task.

Prefix tuning does this at every layer (a distinct set of prefix vectors is injected into each attention layer), prompt tuning does it only at the input embedding layer. Prompt tuning is elegant and extremely cheap — for a 7B model, a prefix of 20 tokens is around 100 000 parameters — but it is fragile on small models and has trouble with hard tasks. Prefix tuning is more robust and roughly matches adapters in quality, at the price of a shorter effective context window.

LoRA: the method that won

LoRA — Low-Rank Adaptation of Large Language Models — is the third family, and it is the one that took over the ecosystem in 2023 and has not been dethroned since. Rather than inject a new module or a virtual prompt, LoRA writes the update to certain weight matrices as a product of two low-rank matrices, and trains only those two. Module 5 is dedicated to it.

Three reasons LoRA won:

• It plugs into the existing matrices. No new modules, no latency penalty at inference — you can even fold the update back into the base weights and get the same speed as the unmodified model.
• It composes well. You can train several LoRA adapters (one per task, one per language, one per team) and swap them at inference time without reloading the base model.
• It has a cheap 4-bit cousin. QLoRA (module 6) quantizes the frozen base to 4 bits, dropping the memory footprint to fit on a 24 GB card.

What "frozen" actually means in code

The distinguishing feature of every PEFT method is what it freezes. The Hugging Face peft library encodes this consistently: you wrap a base model, declare the method, and the library sets requires_grad = False on the base parameters and requires_grad = True only on the new ones.

from transformers import AutoModelForCausalLM
from peft import LoraConfig, get_peft_model, TaskType

base = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.3")

config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=16, lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05, bias="none",
)

model = get_peft_model(base, config)
model.print_trainable_parameters()
# trainable params: 4,194,304 || all params: 7,247,552,512 || trainable%: 0.058 %

That last line — 0.058 % of the model is trainable — is the whole point of the exercise. The gradient and optimizer state now cover four million parameters instead of seven billion, and the 84 GB from module 3 collapses to under 20 GB.

When PEFT is not enough

Two situations still call for the full monty. When the target task is very far from the base model — a completely new language, a fundamentally different output modality — the low-rank subspace is not expressive enough, and the fine-tune plateaus at a mediocre loss. And when the dataset is very large (hundreds of thousands to millions of examples), the low-rank bottleneck becomes the limit. Neither describes our red-thread task, and the rest of this course commits to the PEFT path.

Read the PEFT library source once

peft is a small, readable codebase — a couple thousand lines of Python. Reading LoraLayer.forward once is more educational than any tutorial, because it makes the "freeze the base, add a small update" idea click in code, not in prose. It also demystifies the config: every parameter you set in LoraConfig maps to one line in that file.

Summary

• PEFT freezes the base model and trains a small number of extra parameters (typically 0.1 % of the model), which cuts the training memory by roughly the same factor.
• Three families: adapters (bottleneck modules inside each block), prefix and prompt tuning (learnable virtual tokens), and LoRA (low-rank updates on existing matrices).
• LoRA won because it has no inference-time latency, composes across tasks, and quantizes to 4 bits (QLoRA in module 6) to fit on a consumer GPU.
• The peft library encodes the freezing pattern uniformly: you configure the method, wrap the base model, and only the new parameters carry gradients.

Next module: the exact mathematics of LoRA, and how to pick its two hyperparameters (rank and alpha) for the meeting-minutes task.