Lesson 5 — Reading the notation
Most people who avoid papers are not defeated by the ideas. They are defeated by the symbols, which are unfamiliar, undefined, and inconsistent between authors. This lesson is a decoder. The ideas behind these symbols are ones you already met in the previous lessons.
The convention nobody tells you
There is an informal convention that most papers follow, and knowing it removes a surprising amount of confusion:
- Lowercase italic letters are scalars: a single number.
n,i,t - Lowercase bold letters are vectors: a list of numbers.
x,w - Uppercase bold letters are matrices: a grid.
W,X - Uppercase calligraphic letters are sets or datasets.
Dfor a dataset,Lfor a loss
So x is one example, X is the whole dataset as a matrix, and w is the parameter vector. This is not universal, and it holds often enough to be worth assuming until contradicted.
The symbols you will actually meet
| Symbol | Read it as | What it means |
|---|---|---|
| Σ (sigma) | "the sum over" | add up a series of terms |
| Π (pi) | "the product over" | multiply a series of terms |
| θ (theta) | "the parameters" | all the model's learnable numbers, collectively |
| α (alpha) | "the learning rate" | how big a step training takes |
| η (eta) | "the learning rate" | the same thing, different author |
| λ (lambda) | "the regularisation strength" | how hard you penalise complexity |
| σ (sigma) | "sigmoid" or "standard deviation" | context decides; a squashing function, or a spread |
| μ (mu) | "the mean" | the average |
| ∇ (nabla) | "the gradient of" | the vector of slopes from lesson 3 |
| ∂ | "the partial derivative" | slope with respect to one variable, others held fixed |
| ŷ ("y hat") | "the prediction" | what the model output |
| y | "the truth" | the correct answer |
| ∈ | "is in" | membership in a set |
| ≈ | "is approximately" | equal enough |
| argmin | "the input that minimises" | not the smallest value, the input achieving it |
| argmax | "the input that maximises" | which class had the highest score |
| E[·] | "the expected value of" | the average over a distribution |
| ‖x‖ | "the norm of x" | the length of a vector |
| x ~ D | "x is drawn from D" | sampled from a distribution |
Two of those deserve emphasis because they cause the most confusion.
argmin and argmax return an input, not a value. If a function is smallest at x = 3 with value 7, then min is 7 and argmin is 3. Training is written as finding the argmin of the loss over the parameters, meaning: find the parameter values, not the error value.
∇ (nabla) is just the gradient. When you see ∇θ L, read it as "how the loss changes with respect to each parameter" — precisely the vector of slopes from lesson 3. Once you read the symbol as a word, the equations become sentences.
Reading a loss function
Here is a formula you will meet constantly, mean squared error:
L(θ) = (1/n) Σ (y_i − ŷ_i)²
Translated, term by term:
L(θ)— the loss, as a function of the parameters(1/n) Σ— take the average over all n examplesy_i − ŷ_i— for each example, the truth minus the prediction: the error( )²— squared, so that negative and positive errors both count as bad, and large errors count much more than small ones
Read as a sentence: the average squared gap between prediction and truth. That is the whole content. Every loss function you meet can be unpacked the same way, and doing so a few times deliberately is what removes the intimidation permanently.
The other one you will see everywhere is cross-entropy, used for classification. Its formula looks worse and its meaning is equally simple: penalise the model according to how confidently wrong it was. Being 99% sure and wrong is punished enormously; being 51% sure and wrong is punished mildly. That asymmetry is exactly what you want from a classifier, because it teaches the model to be uncertain when it should be.
Reading a paper you do not fully understand
You will never understand a paper completely on a first pass, and that is normal — experienced researchers do not either. What they have is a strategy.
Practical habits that help more than they should:
- Read the figures before the text. A well-made paper explains its contribution in one diagram, and that diagram is usually clearer than the paragraph describing it.
- Skip every equation on the first pass. If the prose stops making sense without them, come back. Usually it does not.
- Look for the reference implementation. Reading twenty lines of PyTorch is very often faster than reading the equation it implements, because code cannot be vague.
- Note the symbol definitions as you go. Authors define notation once, early, and then use it for fifteen pages. A three-line glossary on scrap paper saves constant scrolling.
- Read the limitations section. It is the most honest part of most papers and tells you immediately whether the result applies to your situation.
Rather than the newest paper on a topic, find the one everyone cites. Foundational papers are usually better written, more thoroughly explained by others, and cover ideas that will still matter in five years. The transformer paper from 2017 is a better use of an afternoon than most of what was published last month.
In three sentences
Papers use a loose but real convention — lowercase for scalars, bold for vectors, uppercase bold for matrices — and roughly twenty symbols cover almost everything you will meet. Read each symbol as a word and equations become sentences: mean squared error is simply the average squared gap between prediction and truth. Read papers in passes, starting with figures and skipping equations, because understanding a paper completely on the first pass is not something anyone does.
Next — Lesson 6: recap and FAQ →