#neuron — Neural networks

Let us break a neuron apart: inputs, weights, sum, activation.

What you'll play with

Welcome to the #neuron channel. On the left, three inputs x1, x2, x3. In the center, one neuron. On the right, the output y. Each tube carries a weight: blue = positive, red = negative, thickness = magnitude.
Change the first weight to see the effect live. Type /weight 1 0.8 or click the button.
Modify one input. Type /input 2 -0.5. The sphere changes size, and so does the sum.
The weighted sum plus the bias gives Σ. Without an activation, you would only have a linear regression — no way to learn XOR. Switch to ReLU with /activation relu.
Try /activation sigmoid then /activation tanh. Each function bends the output differently. A neuron is nothing more than y = f(Σ w·x + b). Stack 128 of them and you have a mini brain.

Artificial neuron: A compute unit that takes the weighted sum of its inputs, adds a bias and applies an activation function. Stacked by the thousands, these neurons make up a network.
Weight: A coefficient w multiplying an input. This is what learning tunes: a large weight makes the neuron sensitive to that input, a negative one flips it.
Bias: A constant b added to the weighted sum. It shifts the decision boundary without changing its direction, like the intercept of a line.
Weighted sum: z = Σ wᵢ·xᵢ + b, the "pre-activation". Geometrically it is the signed distance from the point to the decision boundary.
Perceptron: The historical neuron (Rosenblatt, 1958): weighted sum then threshold. It learns any linear boundary but fails on XOR, which motivated hidden-layer networks.
Decision boundary: The set of points where the neuron hesitates (z = 0). For a single neuron it is a line in 2D, a plane in 3D, a hyperplane beyond.
Linear separability: A dataset is linearly separable if one line (or hyperplane) is enough to split the classes. XOR is not: you need at least one hidden layer.
Hidden layer: A layer of neurons between the input and the output. Each draws its own boundary; their combination produces curved or piecewise boundaries.

#neuron — Let us break a neuron apart: inputs, weights, sum, activation.
#dropout — Dropout regularization: the best friend of deep networks.
#activation — Why an activation? ReLU, sigmoid, tanh, Leaky ReLU, GELU and the vanishing gradient.
#gradient-descent — Loss is a landscape. The gradient says which way is up, so we step the other way.
#backpropagation — The computation graph replayed backwards: each node receives ∂L/∂(itself) and the chain rule does the rest.
#overfitting — A big model on few data points: the decision boundary twists until it memorizes the noise.
#cnn-filters — An image is a grid of numbers. A 3×3 filter slides over it, multiplies, adds up: that is a convolution.
#is-it-einstein — Is it Einstein? Two faces go through the scanner of a network that learned only Einstein: Haythem → NO, Einstein → YES. Watching is free; touching is Premium.
#embeddings-3d — A word becomes a vector: close in space = close in meaning, and you can do math on them.