📄️ Fine-tuning models
Specialize an open 7B language model with LoRA and QLoRA on a single 24 GB GPU, from dataset to merged weights. 8 hours, a 40-question exam and a verifiable certificate.
📄️ 1. Prompt, RAG or fine-tune
Module 1 of the Fine-tuning premium course: what a fine-tune actually teaches, what it cannot fix, and how to pick between a better prompt, a RAG index and training.
📄️ 2. Instruction dataset
Module 2 of the Fine-tuning premium course: conversational format, chat templates, deduplication, train and validation split, and the risks of synthetic data.
📄️ 3. Full fine-tune cost
Module 3 of the Fine-tuning premium course: memory for weights, gradients and optimizer states on a 7B model, catastrophic forgetting, and when full fine-tuning is still right.
📄️ 4. Parameter-efficient
Module 4 of the Fine-tuning premium course: adapters, prefix tuning, LoRA in the PEFT family, which weights stay frozen, and using the PEFT library.
📄️ 5. LoRA adapters
Module 5 of the Fine-tuning premium course: the low-rank decomposition, rank and alpha, target modules, trainable parameter count and stacking multiple adapters.
📄️ 6. QLoRA 4-bit
Module 6 of the Fine-tuning premium course: NF4 quantization, double quantization, paged optimizer, running QLoRA on a single 24 GB GPU and its measured quality gap.
📄️ 7. Hyperparameters
Module 7 of the Fine-tuning premium course: learning rate, epochs, effective batch size, sequence length, warmup and reasonable starting values for LoRA and QLoRA.
📄️ 8. Monitoring & stopping
Module 8 of the Fine-tuning premium course: training and validation loss curves, overfitting on small datasets, generated samples per epoch and disciplined checkpoints.
📄️ 9. Merge & export
Module 9 of the Fine-tuning premium course: merging LoRA into the base weights, exporting to GGUF for llama.cpp and Ollama, Hub publishing and base-model licensing.
📄️ 10. Evaluation
Module 10 of the Fine-tuning premium course: held-out test set, format and content metrics, blind human judgment and regression on general capabilities.
📄️ Recap and exam
Complete recap of the Fine-tuning premium course: dataset, PEFT, LoRA, QLoRA, hyperparameters, monitoring, merging and evaluation, then the 40-question exam.