Recap and final exam
Ten modules to go from a Keras model that runs on a GPU to a .tflite file that runs on a farmer's phone with no network. Here is the course condensed, then the decision tree that summarises the whole optimisation flow.
The course at a glance
| Module | What to retain |
|---|---|
| 1. On-device constraints | Four numbers before anything else: file size, RAM peak, latency, energy per inference |
| 2. Converting to TFLite | Convert from a SavedModel; unsupported ops mean rewrite the model, not pile on Select TF Ops |
| 3. Post-training quantization | Full-integer int8 unlocks accelerators; the representative dataset must be real validation data |
| 4. QAT | Only when post-training loses more than one point on classes that matter, at a very low learning rate |
| 5. Pruning and clustering | Raw file size does not change; the gzipped size is what pays, and it can nearly halve |
| 6. Interpreter and delegates | Load, allocate, invoke; GPU on sustained loads, NNAPI on Android 10+, Core ML on A11+, XNNPACK everywhere |
| 7. Android integration | noCompress("tflite"), Task Library reads metadata, STRATEGY_KEEP_ONLY_LATEST prevents backlog |
| 8. iOS integration | Swift Package or Pod, watch out for BGRA versus RGB, Core ML delegate for Neural Engine |
| 9. Measuring latency and energy | Benchmark tool, warm-up first, percentiles not averages, three phone tiers |
| 10. On-device project | The trade-off matrix is a decision; document what was sacrificed |
The decision tree
Every optimisation choice in the course flows from a single question — "what is off-budget?" — and the answer picks the tool.
- File size is over the budget? Start with full-integer quantization (module 3). If still over, add pruning with a gzip-friendly compression pipeline (module 5). If still over, add weight clustering.
- Latency is over the budget? Confirm you shipped an int8 model, not float32. Enable XNNPACK (default in recent versions) and set
numThreads = 4. Try the GPU delegate if inferences are sustained. Try NNAPI on Android 10+ or Core ML on iPhone A11+. Measure each step; a wrong delegate is worse than no delegate. - Energy is over the budget? The model is running too often, or it is too big. In order: drop the inference rate (frame skipping in live camera), pick a smaller architecture family (MobileNetV3-Small over MobileNetV2), and only then push int8 harder.
- Accuracy dropped more than one point after quantization? Post-training was not enough. Run QAT at a very low learning rate for a few epochs, on the same training data.
- Predictions look fine locally but drift on the phone? The bug is preprocessing. Confirm the normalisation constants match training, confirm BGR versus RGB on iOS, confirm image size and rotation. Metadata + Task Library remove most of this class of bug.
- App size is fine, then bloated after enabling one library? Select TF Ops adds several MB of native library. Rewrite the offending op in the training code and re-export the SavedModel.
The threads running through the course
On-device is not a smaller cloud. The whole discipline reverses the cloud reflex. You cannot A/B test after ship, you cannot roll back server-side, you cannot patch a preprocessing bug without a store submission. Every design choice earlier in the course — bundle the model, put preprocessing inside the graph, ship metadata alongside the file — is a consequence. This is the axis modules 1, 2 and 10 keep returning to.
The bottleneck is almost never where you look for it. The instinctive response to "the model is too slow" is a smaller model. The reality is that most latency budget goes to preprocessing, YUV conversion, memory copies and the interpreter set-up — not the arithmetic. Module 6 handles the interpreter, module 7 and 8 handle the frame pipeline, module 9 tells you where the time actually goes. Optimising the model first and measuring last is how you spend a week and change nothing.
Trade-off matrix over hero numbers. No variant is universally best. Post-training quantization wins for teams without the training pipeline. QAT wins when a percentage point matters. Pruning wins for cellular downloads. Clustering wins on the tightest budgets. Modules 3 through 5 each add a row; module 10 defends the pick. A single "best" number for latency or size is almost always misleading — the numbers only make sense next to the constraints.
Measure early, measure often, measure on real hardware. An emulator lies. A flagship lies. A benchmark on a Pixel 4a and a Redmi 9, run after every model artifact, is the smallest process that stops "we shipped a regression six weeks ago" from becoming the norm. Module 9 exists so this measurement is a five-minute habit, not a two-day investigation.
The final exam
The exam has 40 questions covering all ten modules: which constraints govern which optimisations, how to write a representative dataset, when QAT is worth its cost, why pruning does not shrink the raw file, how to pick a delegate for a workload, why BGRA on iOS is not RGB, why a P50 that looks great can hide a P95 that does not, and how to read the trade-off matrix.
Several questions present situations to diagnose: a pruned model with the same on-disk size, a full-int model that regressed on the rare classes, a Core ML delegate that fell back to CPU silently, an app that classifies the leaf correctly in Python and wrongly on the phone, a benchmark run that says one thing and a field session that says another. Judgement is what gets assessed, not memorised API signatures.
On success, your certificate of completion is issued immediately; its number can be verified by any third party on the platform.
Go back over the decision tree above and, for each branch, ask yourself "what symptom would push me to this branch?". If you can explain why a representative dataset drawn from augmented data will regress the quantized model, why enabling the GPU delegate on a mixed model can be slower than pure CPU, and why the "same" model produces different predictions on Android and iOS after a normalisation constant drift, you are ready. Good luck!
Final exam
Ready to validate this course?
40 questions drawn at random from the course bank · passing score 70% · verifiable PDF certificate issued immediately on success.
Start the examYou need to be signed in to your InSkillML account with an active subscription. You can also start the exam from My courses.