Neuron
A tiny decision-maker. It adds up its inputs and decides how loudly to fire.
y = activation(Σ wᵢxᵢ + b)
Glossary
Every term in plain kid-friendly language, with the real math version underneath.
Layer
A row of neurons that all look at the same signal together.
h⁽ˡ⁾ = activation(W⁽ˡ⁾h⁽ˡ⁻¹⁾ + b⁽ˡ⁾)
Weight
A volume knob that says how much one input matters.
wᵢ — the multiplier applied to input xᵢ before summing
Bias
A head start added to a neuron so it can fire even when its inputs are quiet.
sum = Σ wᵢxᵢ + b; b shifts the threshold
Activation
The rule that turns a neuron's total into its final 'boom'.
e.g. ReLU: max(0, x), sigmoid: 1/(1+e⁻ˣ), tanh: (eˣ−e⁻ˣ)/(eˣ+e⁻ˣ)
Gain
A stretch knob. Higher gain makes the activation react more steeply.
y = activation(gain · x) — gain scales the input before the curve
Learning rate
How big a step the network takes when it fixes a mistake.
w ← w − α · ∂Loss/∂w, where α is the learning rate
Loss (error)
A score for how wrong the network's answer was — smaller is better.
Loss = (guess − correct)², averaged over examples
Class
A label we want the network to pick, like 'cat' or 'dog'.
y ∈ {A, B} — the category the output is matched against
Decision boundary
The dividing line where the network switches from saying 'this one' to 'that one'.
the set of points where Σ wᵢxᵢ + b = 0
Supervised learning
Learning with an answer key: every example comes with the correct label.
minimize Σ L(f(x), y) over labelled pairs (x, y)
Unsupervised learning
Learning with no answer key — the network hunts for patterns and groups in the data.
find structure in {x}; e.g. cluster centroids that minimize within-cluster distance
Reinforcement learning
Learning by trial, error and reward: good moves get points, bad moves lose them.
maximize E[Σ γᵗ rₜ] by acting in an environment
Neuroevolution
Breed a population of networks: mutate them and let the best-scoring ones survive.
evolve weights by selection + mutation; no gradient required
Epoch
One full pass through all the practice examples.
one sweep over the training set: for each batch, w ← w − α∇Loss
Gradient
A slope that says which way is downhill and how steep it is.
∇Loss — the vector of partial derivatives ∂Loss/∂wᵢ
Backpropagation
The network's blame game: errors flow backward so every weight learns its share.
reverse-mode differentiation of Loss via the chain rule
Chain rule
To find how a change travels through a chain, multiply the little effects along the way.
dL/dw = dL/dy · dy/dz · dz/dw
Exploration
Trying something new instead of always taking the move you think is best.
with probability ε pick a random action (ε-greedy)
Mutation
A small random tweak to a child network so it differs from its parent.
w′ = w + N(0, σ), where σ is set by the mutation rate
Population
A whole group of candidate networks that compete and breed each generation.
P = {θ₁, …, θₙ}, evolved by selection
Agent
The learner that takes actions in a world to earn rewards.
policy π(a|s) mapping states to actions
Reward
A number the agent gets for doing something: positive for good, negative for bad.
rₜ — scalar feedback used to score actions
Self-supervised learning
The data labels itself: hide part of it and learn to predict the missing bit.
maximize P(next token | previous tokens)