Module 8 — iOS integration

The iOS story is close to the Android one — same interpreter lifecycle, same preprocessing rules, same "never on the UI thread" law — with two differences that matter: the toolchain (CocoaPods or Swift Package Manager instead of Gradle), and the presence of Core ML as a serious alternative on Apple hardware. This module walks through both, so you can pick the right runtime for your app rather than defaulting to whichever one you happened to try first.

Pod or Swift Package

Both work. The Swift Package is the modern default:

// Package.swift
dependencies: [
    .package(
        url: "https://github.com/tensorflow/tensorflow.git",
        .exact("2.16.1")
    ),
],
targets: [
    .target(
        name: "LeafClassifier",
        dependencies: [
            .product(name: "TensorFlowLiteSwift", package: "tensorflow"),
            .product(name: "TensorFlowLiteTaskVision", package: "tensorflow"),
        ]
    )
]

Or via CocoaPods for older projects:

# Podfile
target 'LeafClassifier' do
  use_frameworks!
  pod 'TensorFlowLiteSwift', '~> 2.16'
  pod 'TensorFlowLiteTaskVision', '~> 0.4'
end

Once integrated, drop leaf_classifier.tflite into the app bundle (Xcode: drag into the project, tick "Copy items if needed", tick the target). The Android noCompress concern does not apply here — iOS ships the file as-is.

Loading and running the interpreter

import TensorFlowLite

final class LeafClassifier {
    private var interpreter: Interpreter

    init() throws {
        guard let modelPath = Bundle.main.path(
            forResource: "leaf_classifier",
            ofType: "tflite"
        ) else {
            throw NSError(domain: "LeafClassifier", code: 1)
        }

        var options = Interpreter.Options()
        options.threadCount = 4

        interpreter = try Interpreter(modelPath: modelPath, options: options)
        try interpreter.allocateTensors()
    }

    func classify(rgbBuffer: Data) throws -> [Float] {
        try interpreter.copy(rgbBuffer, toInputAt: 0)
        try interpreter.invoke()

        let outputTensor = try interpreter.output(at: 0)
        let count = outputTensor.data.count / MemoryLayout<Float32>.stride
        return outputTensor.data.withUnsafeBytes {
            Array(UnsafeBufferPointer<Float32>(
                start: $0.baseAddress?.assumingMemoryBound(to: Float32.self),
                count: count,
            ))
        }
    }
}

The pattern is the same six steps from module 6: load, allocate, discover, set, invoke, read. Swift's error handling makes each step explicit, which is a good thing for a code path that can fail in many ways on old devices.

Preprocessing: the pixel format trap

An AVCaptureVideoDataOutput frame arrives as a CVPixelBuffer in the format you asked for — most commonly kCVPixelFormatType_32BGRA. Note the order: B, G, R, A. Your model expects RGB. Feeding BGR to a network that trained on RGB is a silent bug that costs about ten accuracy points on our classifier, with no exception, no warning.

The safe pattern is to convert in one place:

private func rgbData(from pixelBuffer: CVPixelBuffer, size: CGSize) -> Data? {
    CVPixelBufferLockBaseAddress(pixelBuffer, .readOnly)
    defer { CVPixelBufferUnlockBaseAddress(pixelBuffer, .readOnly) }

    guard let source = CVPixelBufferGetBaseAddress(pixelBuffer) else { return nil }
    let sourceRowBytes = CVPixelBufferGetBytesPerRow(pixelBuffer)
    let sourceHeight = CVPixelBufferGetHeight(pixelBuffer)

    // Resize to model input using vImage or Accelerate, then swizzle BGRA -> RGB.
    // Normalise to [-1, 1] to match MobileNetV2.
    // Return the packed float32 buffer.
    // ...
    return packed
}

The vImage and Accelerate frameworks do the heavy lifting: hardware-accelerated resize and colour conversion in a few lines. Do not roll your own byte loop — it is both slow and error-prone.

The TensorFlowLiteTaskVision Task Library shortcuts this the same way it did on Android, if metadata is attached:

import TensorFlowLiteTaskVision

let options = ImageClassifierOptions(modelPath: modelPath)
options.classificationOptions.maxResults = 3
options.classificationOptions.scoreThreshold = 0.30
options.baseOptions.computeSettings.cpuSettings.numThreads = 4

let classifier = try ImageClassifier.classifier(options: options)

let mlImage = MLImage(pixelBuffer: pixelBuffer)!
let results = try classifier.classify(mlImage: mlImage)

The Task Library reads the normalisation constants from the metadata and applies them. When metadata is correct, the entire preprocessing story becomes "hand the pixel buffer to the classifier".

Delegates on iOS

Two delegates matter.

Metal delegate turns the CPU inference into a GPU one. Setup is one call:

let metalDelegate = MetalDelegate()
interpreter = try Interpreter(modelPath: modelPath, delegates: [metalDelegate])

Core ML delegate goes further: it converts supported subgraphs into Core ML ops that run on the Neural Engine of an A11 chip or newer. On an iPhone 12, our leaf classifier drops from 22 ms (CPU) to about 6 ms (Neural Engine) and uses almost no energy — the Neural Engine was built for this kind of workload.

let coreMLDelegate = CoreMLDelegate()
interpreter = try Interpreter(modelPath: modelPath, delegates: [coreMLDelegate])

Unsupported ops stay on the TFLite CPU or Metal path. Always fall back gracefully: if let coreMLDelegate = CoreMLDelegate() { ... } else { /* CPU */ }.

Core ML as an alternative to TFLite

On iOS, coremltools can convert a TensorFlow model directly to a native .mlmodel or .mlpackage. That model then uses Core ML end to end — no TFLite runtime in the app.

Criterion	TFLite	Core ML
Cross-platform	yes — same file on Android and iOS	no — iOS only
Neural Engine access	via delegate	native
Runtime size	~1.5 MB added	zero (part of the OS)
Debugging on device	Xcode + custom	Xcode Instruments, native profiling
Metadata and Task-Library-style API	yes	yes (with MLModelDescription)

The decision comes down to this: if the model has to run on Android and iOS, use TFLite on both. Maintaining two model artifacts (one .tflite, one .mlmodel) with converters in the middle is where drift creeps in — the two versions produce slightly different predictions on the same input, and after six months no one can explain why. If the app is iOS-only, Core ML is a strong native fit and it saves the runtime binary weight.

Our leaf classifier is meant to run on both, so it stays on TFLite. Modules 7 and 8 give it the same behaviour on both platforms, and the trade-off matrix is one table, not two.

Preprocessing must be identical on Android and iOS

The Android team wrote NormalizeOp(127.5f, 127.5f). The iOS team must do the same maths. When those two constants drift by one number, the two apps predict differently on identical inputs, and the training team cannot reproduce either. Attaching normalisation constants via metadata (module 2) is what removes this class of bug.

Camera pipeline

AVCaptureSession is the iOS counterpart of CameraX:

let session = AVCaptureSession()
session.sessionPreset = .vga640x480

let device = AVCaptureDevice.default(for: .video)!
let input = try AVCaptureDeviceInput(device: device)
session.addInput(input)

let output = AVCaptureVideoDataOutput()
output.setSampleBufferDelegate(self, queue: DispatchQueue(label: "inference"))
output.alwaysDiscardsLateVideoFrames = true    // drop backlog under load
session.addOutput(output)

session.startRunning()

The alwaysDiscardsLateVideoFrames = true line is the iOS equivalent of STRATEGY_KEEP_ONLY_LATEST from module 7. Same reasoning: if inference cannot keep up, drop frames rather than accumulate them. The user sees a lower effective frame rate, not a growing lag.

The dedicated inference queue keeps the main queue free for UI updates. When results are ready, dispatch back to the main queue:

DispatchQueue.main.async { [weak self] in
    self?.updateLabels(with: results)
}

Camera permissions

iOS requires an explicit NSCameraUsageDescription in Info.plist, and the value is shown to the user in the permission prompt. Write it in the user's language and be specific — "This app uses the camera to identify plant diseases on your crop" is far better than "Camera access needed", and materially increases the grant rate.

Key takeaways

• Use the Swift Package integration for new projects, CocoaPods for legacy — both give you TensorFlowLiteSwift and the Task Library.
• BGRA is not RGB: convert with vImage/Accelerate, or let the Task Library do it via metadata; hand-rolled byte swizzles are where silent accuracy drops start.
• The Core ML delegate unlocks the Neural Engine on A11+ chips for dramatic latency and energy wins; use it and fall back to Metal or CPU when it is unavailable.
• Ship one model artifact for both platforms when the app runs on both; convert to native Core ML only if the app is iOS-exclusive and the drift from two artifacts would cause more harm than the extra runtime.

Next module: measuring latency and energy on real devices, so the trade-off matrix stops relying on guesswork.