Module 7 — Android integration

Modules 1 to 6 stayed in Python. From here on, the model has to live inside an app. This module covers the Android side: adding the dependencies, loading the .tflite, converting a camera frame into the tensor shape the interpreter expects, running inference off the UI thread, and using the Task Library when the model has proper metadata. The examples are in Kotlin, which is the modern default; the Java equivalents differ only in syntax.

The dependencies

Two paths. The low-level path uses tensorflow-lite and demands you handle preprocessing by hand. The high-level path uses tensorflow-lite-task-vision and reads the metadata we attached in module 2.

// app/build.gradle
dependencies {
    // low-level runtime
    implementation("org.tensorflow:tensorflow-lite:2.16.1")
    implementation("org.tensorflow:tensorflow-lite-gpu:2.16.1")
    implementation("org.tensorflow:tensorflow-lite-gpu-delegate-plugin:0.4.4")

    // high-level Task Library
    implementation("org.tensorflow:tensorflow-lite-task-vision:0.4.4")
}

android {
    aaptOptions {
        // Do not compress the model file — the runtime maps it directly.
        noCompress("tflite")
    }
}

The noCompress line is not decorative: without it, the APK builder gzips the model, and the runtime has to unpack it into memory at load time. That eats the memory budget from module 1 twice — once as file, once as decompressed buffer.

The model itself sits in app/src/main/assets/leaf_classifier.tflite, alongside a labels.txt with one class name per line if you did not attach labels via metadata.

The Python-style path: doing it by hand

Even if you plan to use the Task Library, understanding the manual path helps you diagnose the errors when metadata is missing or wrong.

import org.tensorflow.lite.Interpreter
import java.io.FileInputStream
import java.nio.MappedByteBuffer
import java.nio.channels.FileChannel

class LeafClassifier(context: Context) {
    private val interpreter: Interpreter

    init {
        val model = loadModelFile(context, "leaf_classifier.tflite")
        val options = Interpreter.Options().apply {
            numThreads = 4
        }
        interpreter = Interpreter(model, options)
    }

    private fun loadModelFile(context: Context, name: String): MappedByteBuffer {
        val fd = context.assets.openFd(name)
        val stream = FileInputStream(fd.fileDescriptor)
        return stream.channel.map(
            FileChannel.MapMode.READ_ONLY,
            fd.startOffset,
            fd.declaredLength,
        )
    }

    fun classify(input: ByteBuffer): FloatArray {
        val output = Array(1) { FloatArray(NUM_CLASSES) }
        interpreter.run(input, output)
        return output[0]
    }
}

The MappedByteBuffer matters: the runtime reads directly from the mapped file rather than copying it into a heap buffer. Combined with noCompress, this keeps the RAM peak close to the interpreter's own arena and away from the file bytes.

Preprocessing a camera frame is the tedious part: convert YUV to RGB, rotate to match the device orientation, resize to 224x224, normalise, pack into a ByteBuffer. The ImageProcessor in the Task Library does exactly this:

import org.tensorflow.lite.support.image.ImageProcessor
import org.tensorflow.lite.support.image.TensorImage
import org.tensorflow.lite.support.image.ops.ResizeOp
import org.tensorflow.lite.support.common.ops.NormalizeOp

val imageProcessor = ImageProcessor.Builder()
    .add(ResizeOp(224, 224, ResizeOp.ResizeMethod.BILINEAR))
    .add(NormalizeOp(127.5f, 127.5f))     // (pixel - 127.5) / 127.5 → [-1, 1]
    .build()

val tensorImage = imageProcessor.process(TensorImage.fromBitmap(bitmap))

The normalisation constants must match what MobileNetV2 was trained with. Using ImageNet mean and standard deviation on a MobileNetV2 that expects [-1, 1] will not throw an error; it will silently degrade predictions. This is the training-serving skew from course 08, module 10, resurfacing on mobile.

The high-level path: Task Library

When metadata is attached (which we did in module 2), the whole class collapses:

import org.tensorflow.lite.task.vision.classifier.ImageClassifier
import org.tensorflow.lite.task.vision.classifier.ImageClassifier.ImageClassifierOptions
import org.tensorflow.lite.task.core.BaseOptions

val baseOptions = BaseOptions.builder()
    .setNumThreads(4)
    .useGpu()                     // fallback to CPU if unavailable
    .build()

val options = ImageClassifierOptions.builder()
    .setBaseOptions(baseOptions)
    .setMaxResults(3)
    .setScoreThreshold(0.30f)
    .build()

val classifier = ImageClassifier.createFromFileAndOptions(
    context,
    "leaf_classifier.tflite",
    options,
)

// Inference from a bitmap or ImageProxy:
val results = classifier.classify(TensorImage.fromBitmap(bitmap))
results.first().categories.forEach {
    Log.d("Leaf", "${it.label} — ${it.score}")
}

The Task Library reads the normalisation constants and the labels from the metadata, applies the correct preprocessing, runs the interpreter, decodes the output. useGpu() requests the GPU delegate and silently falls back to CPU if the device does not support it — the "catch delegate failures" rule from module 6, made trivial.

Camera and background thread

The single most common Android bug in this course is running inference on the UI thread. Even 22 ms is enough to skip a frame; a burst of ten inferences freezes the animation for a quarter of a second, and every user notices.

CameraX integrates cleanly with a background executor:

import androidx.camera.core.ImageAnalysis
import java.util.concurrent.Executors

val inferenceExecutor = Executors.newSingleThreadExecutor()

val analysis = ImageAnalysis.Builder()
    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
    .setTargetResolution(Size(640, 480))
    .build()
    .also {
        it.setAnalyzer(inferenceExecutor) { imageProxy ->
            val bitmap = imageProxy.toBitmap()
            val results = classifier.classify(TensorImage.fromBitmap(bitmap))
            imageProxy.close()

            runOnUiThread {
                updateLabels(results)
            }
        }
    }

Two design points here.

STRATEGY_KEEP_ONLY_LATEST drops frames while inference is running. Without it, the queue grows during a stall and the classifier ends up processing frames from ten seconds ago. Users read that as "the app is laggy"; the real cause is that inference cannot keep up.

A single background thread for inference. Four threads inside the interpreter (numThreads = 4) plus four parallel invocations across four threads is not four times faster — the phone has only so many performance cores, and they contend. One inference at a time, on one background thread, with a bounded queue.

Permissions

Camera work needs the runtime permission dance:

<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera.any" />

And in code, request Manifest.permission.CAMERA before opening CameraX. Farmers using our leaf classifier will not read a documentation page: the app has to walk them through granting the permission, and it has to keep working with a still gallery pick if they deny it.

Model updates: from bundled to downloaded

Version 1 of the app ships the model in assets/, and it fits: 2.35 MB is nothing next to a 30 MB app. But a model retrained six months from now on new disease data should not require an app update — the review cycle alone burns two weeks.

The clean pattern is a small download at first launch:

• Ship a default model in assets/ so day-one users see nothing.
• On first launch (with connectivity), check a manifest URL for a newer model version.
• Download to internal storage, checksum it, atomically replace the loaded model.
• Keep the previous version until the new one has served ten inferences without exception.

The last point is the rollback path: a corrupted or drifted model on production devices is exactly what module 1 warned about — no server-side control. A local rollback recovers without waiting on the store.

Key takeaways

• Add noCompress("tflite") and use MappedByteBuffer so the model file is memory-mapped, not decompressed into RAM.
• Preprocessing must match training exactly; the ImageProcessor chain (resize, normalise) or the Task Library reading metadata are both correct paths — hand-rolled YUV conversion is where bugs hide.
• Never infer on the UI thread; use a singleThreadExecutor and STRATEGY_KEEP_ONLY_LATEST on CameraX to drop backlog rather than pile up frames.
• Ship the model in assets/ for day-one, then plan a remote update with checksum verification and local rollback: the store cannot save you when a bad model is already on the phone.

Next module: the iOS counterpart, with a look at Core ML as an alternative to TFLite on Apple hardware.