Module 1 — The ONNX format: graph, operators, versions
Every deep learning framework — PyTorch, TensorFlow, JAX, MXNet, CNTK back in the day — stores its models in a private format. Loading one requires the framework that produced it, at approximately the same version. Serving a PyTorch checkpoint from a Java service is impossible without a PyTorch-Java bridge; loading a TensorFlow SavedModel from a Node.js runtime forces you to install TensorFlow.js and hope the ops line up. This friction is the reason ONNX exists. The Open Neural Network Exchange defines a shared intermediate representation: one file, one operator vocabulary, and a growing list of runtimes that speak it. This module describes that representation before the rest of the course exports, optimizes, quantizes and serves it.
The two running models of the course — a ResNet18 fine-tuned on Fashion-MNIST in the PyTorch course, and a small text encoder for sentence classification — will both end up in this format. Understanding what is inside a .onnx file, and what its version numbers mean, is what turns "the export failed" from a wall into a diagnostic.
An ONNX file is a computation graph, not a program
A neural network in PyTorch is a Python object whose forward method describes the computation. Under the hood, that method runs a sequence of tensor operations. ONNX serialises the sequence itself: it captures which operations, in which order, with which parameters produce the outputs from the inputs. The Python code, the class hierarchy, the training loop — all of it stays behind. What travels is the graph.
Concretely, an ONNX model is a Protocol Buffers file — a compact binary format designed by Google — with a strict schema. At the top sits a ModelProto. Inside it, a GraphProto holds three lists that describe the entire computation:
- initializers: the trained tensors — weights and biases — stored as raw byte blobs.
- inputs and outputs: the tensors the caller provides and the ones the model produces, each with a name, a dtype and a shape.
- nodes: the operations, in topological order. Each node names an operator (
Conv,Relu,MatMul,Softmax), the names of its input tensors, the names of its output tensors, and any attributes (kernel size, stride, epsilon).
There are no loops, no if statements, no Python bytecode. The graph is static and deterministic; two ONNX Runtime processes running the same file on the same inputs return the same outputs. This is exactly what makes the format easy to run outside the training framework — and exactly the reason exporting a model with control flow needs care, which is the whole subject of module 9.
Operators and opsets
Where PyTorch offers torch.nn.Conv2d and TensorFlow offers tf.keras.layers.Conv2D, ONNX defines a single operator called Conv, with a documented signature: it takes an input tensor, a weight tensor, an optional bias, and attributes for kernel size, stride, padding and dilations. Every runtime that claims to support Conv must produce numerically identical outputs for the same inputs. The exporter's job is to translate framework-specific layers into this shared vocabulary.
An opset — operator set — is a numbered version of that vocabulary. Opset 11 defines the operators as they existed at a point in time; opset 17 adds new ones and modifies signatures. When you export a model, you pick an opset:
import torch
torch.onnx.export(model, example, "resnet.onnx", opset_version=17)
Two rules govern this choice, and both bite in practice. First, the source framework must know how to translate its layers into the selected opset. A torch.nn.LayerNorm did not exist as a native ONNX operator before opset 17; exporting to opset 11 would decompose it into a chain of primitive ops. Second, the target runtime must support that opset. ONNX Runtime supports up to opset 21 as of writing; older mobile runtimes may stop at opset 13. Exporting to an opset the runtime cannot execute yields a [E:onnxruntime] : unsupported opset at load time, not at export time.
The safe habit is to pick the newest opset that both the exporter and the target runtime support, and to write it explicitly in the export call. Implicit defaults change with each framework release and silently break reproducibility.
Two teams exporting a nn.LayerNorm with the same PyTorch version can produce different ONNX graphs if they pick different opsets. The receiving team cannot debug the difference by looking at PyTorch — the source is gone. Always log the opset alongside the model file.
Reading a model with Netron
The single most useful tool of this course is Netron, a viewer that opens .onnx files in the browser and shows the graph visually. Nodes appear as boxes, tensors as arrows, and clicking a node reveals its attributes. Before optimizing, quantizing or debugging anything, opening the file in Netron and reading it end-to-end saves hours of terminal work.
The command-line alternative is onnx.load in Python:
import onnx
model = onnx.load("resnet.onnx")
print(f"opset: {model.opset_import[0].version}")
print(f"producer: {model.producer_name} {model.producer_version}")
print(f"inputs: {[(i.name, [d.dim_value or d.dim_param for d in i.type.tensor_type.shape.dim]) for i in model.graph.input]}")
print(f"outputs: {[(o.name, [d.dim_value or d.dim_param for d in o.type.tensor_type.shape.dim]) for o in model.graph.output]}")
print(f"nodes: {len(model.graph.node)}")
opset_import names the operator vocabulary. producer_name identifies which exporter emitted the file — pytorch or tf2onnx — a detail that matters when a bug depends on the exporter. The input and output listing shows dimensions: a dim_value is a fixed size, a dim_param (e.g. "batch") is a symbolic dimension that will be filled at runtime. Recognising this distinction is the entire point of module 2's section on dynamic axes.
Why the format is worth its friction
Every course on ONNX must justify itself: why not stick with the training framework? The answer is not one benefit but three, and each carries weight in a real production stack.
Portability. An ONNX model runs in C++, C#, Java, Python, JavaScript, Rust, Go and Swift, on Windows, Linux, macOS, Android, iOS, and even in the browser through ONNX Runtime Web. The training environment installs 3 GB of PyTorch dependencies; the serving environment loads a 50 MB file into a 30 MB runtime.
Optimization. Because the graph is explicit and pure, a runtime can fuse consecutive operators, fold constant subgraphs, and rewrite common patterns before executing. Module 5 measures the gain — typically 1.3x to 2x on CPU, more on GPU with TensorRT.
Hardware routing. The same file dispatches to CPU, CUDA, ROCm, DirectML, TensorRT, OpenVINO or CoreML depending on the runtime configuration. A single artifact becomes the source of truth across the entire serving fleet, which is worth more than any raw performance number: it is a contract between the team that trains models and the team that serves them.
Before diagnosing "the exported model is slow" or "the exported model produces wrong outputs", open the file in Netron. Nine out of ten export bugs are visible in the graph: a Cast inserted around every operation because dtypes were inconsistent, an Identity chain from a training-time hook, a Loop where a static shape was expected.
In summary
- An ONNX file is a computation graph serialised as Protocol Buffers: initializers (weights), inputs, outputs and a topologically ordered list of operator nodes; there is no Python bytecode inside.
- An opset is a versioned operator vocabulary; picking one determines which layers can be exported and which runtimes can load the file. Always write the opset explicitly.
- Netron is the fastest way to understand what actually landed in the file;
onnx.loadis the scriptable equivalent for CI checks. - The format is worth its friction because it delivers portability, optimization and hardware routing from a single artifact — a contract between training and serving.
Next module: torch.onnx.export, dynamic axes, and putting the ResNet18 into an ONNX file.