Building Brains That Remember

The Playground agent looks at the world one instant at a time. Ten sensors in, a move out. No past, no future — just now.

That works surprisingly often. But watch a trained agent walk into a corridor and it hits the same wall again and again. It is not slow, and it is not badly trained. It simply has no memory of the half-second before.

This lesson is about fixing that: what memory is, why it changes everything, and how the simplest version of it works.

A reactive agent maps its current sensors straight to an action: sensors now → move. There is no note to itself and no idea of where it just was.

Picture a corridor with a single open side. Every snapshot at the dead end looks identical to the snapshot at the entrance: wall ahead, wall left, wall right, open behind. A reactive brain sees the same inputs in both places, so it makes the same choice — and shuffles back and forth forever.

Lesson 6 called some maps impossible for a reactive agent, and this is exactly why. The task does not need a bigger brain. It needs a brain that can remember.

The easiest memory is just the previous step, fed back in. Take the usual sensor inputs and glue on one or two extra numbers: where the agent was last turn, or which action it chose.

Now the dead end is no longer identical to the entrance. At the entrance the 'last action' input says *I just arrived*; at the dead end it says *I just walked forward*. The network can finally tell the two apart.

Think of it as a short note passed forward: at each moment, the agent tucks a tiny note about the past into its next set of inputs.

The previous step is a start, but it fades after a single moment. Recurrence keeps the note going: the network's own previous output becomes an input to itself.

That circling signal is called the hidden state — a small bundle of numbers that acts like the agent's scratchpad. It updates every step from two things: the old state and the new sensor reading.

Running-tally analogy. If you count cars passing on a street, you do not re-derive the total from scratch each time. You keep a tally in your head and add one whenever a car goes by. The tally is your hidden state: cheap, compact, and it grows with the whole history. A recurrent network does the same thing with numbers.

This loop is the heart of a family of networks called recurrent neural networks (RNNs). Instead of treating every input as a fresh, unrelated moment, they carry a state from one step to the next.

They are behind tasks where order matters: predicting the next word in a sentence, reading a line of handwriting, or listening to a stream of speech.

Plain RNNs can forget over long stretches — the same shrinking-gradient idea from Lesson 5. A refinement called the LSTM adds little gates that decide what to keep and what to drop, which lets the state survive much longer. This course stays conceptual here: you do not need the gate math to understand the big idea.

The big idea is simply this: let the past flow into the present as an input.

Put the whole course's tools together and you get a recipe for harder tasks:

1. A good goal (Lesson 6) — a shaped reward that gives the agent something to learn from before it ever wins.

2. Enough capacity — hidden layers that can combine the sensors it has.

3. Memory — a state that carries the past forward, so the agent can act on *where it has been*, not just *what is in front of it*.

Change the goal and the brain can learn. Add memory and it can learn tasks that need history at all.