Train an Agent with the Playground

You have built networks by hand and looked inside their training. Lessons 4 and 5 showed you how a network learns — rewards, gradients and evolution.

This lesson is about the other half: setting a goal that learning can actually reach. Get the goal right and even a tiny brain solves the map. Get it wrong and the smartest brain spins in circles.

Your lab is the Playground — a small grid world where you draw goals, hazards and zones, then train an agent to survive it.

Training needs a number that says how well the agent is doing. In evolution that number is the fitness; in reinforcement learning the same idea is called the reward. Either way it answers one question: what does success mean?

A good goal has three properties:

1. It rewards the outcome you actually want — reaching the flag, not merely moving fast.

2. It gives feedback often enough to learn from — a score after every episode beats a score once in a blue moon.

3. It cannot be gamed — a clever agent should never earn a high score by doing something silly.

The Playground speaks a simple vocabulary:

- Goal ◆ — touch it to finish that objective.

- Hazard ✕ — step on it and the episode ends in death.

- Dwell zone ◎ — stay on it for a required number of seconds.

The trainer scores every episode with one fitness formula, so each object type becomes a term in it.

Reading left to right: winning is worth a huge +1000; touching some of the goals pays 400 scaled by the fraction you touched; meeting part of a zone's dwell pays 300; every step costs a small 0.5, so dawdling is discouraged; and dying costs a serious 200.

Notice the shaping: even before the agent ever wins, partial progress (goal and zone) earns reward. That is what pulls evolution toward the flag instead of waiting for a lucky random walk.

Sometimes training simply will not converge, and it is usually not the brain's fault alone. Three failure modes show up again and again:

- Sparse rewards. With only a +1 at the goal, a random population may never reach it, so every candidate scores the same and learning has nothing to climb.

- Local traps. The agent finds a safe corner, or a spot that earns partial credit, and every mutation that leaves it scores worse. It is stuck on a small hill while the real peak sits across a valley.

- Tiny reactive brains. The Playground controller only sees the current sensor readings — it has no memory. Some maps need the agent to remember where it has been or to stand still on a zone, and no amount of training fixes a network that cannot represent the answer.

A bigger hidden layer gives the controller more room to combine sensor readings. In this project a 1×8 shape (one hidden layer of eight neurons) sometimes converged faster than the smaller 1×4 shape, because the extra neurons could represent a few more situations.

But more is not automatically better. Extra neurons mean more weights to search, which can slow training down or make lucky runs less repeatable. The honest rule is: give the brain just enough capacity for the task, and shape the reward so it has something to learn from.

And remember the hard limit — some maps are impossible for a reactive agent. If winning needs memory ("remember that this zone was already visited"), no hidden-layer size can rescue an agent that only sees the present.

Training is easy to watch once you know the three numbers on screen:

- Fitness / reward curve — the score per generation (evolution) or episode (reinforcement). It should trend upward; a flat line means the goal is too sparse or the population too small.

- Generation / episode count — how many rounds of breeding or attempts have happened. Early noise is normal; look for a trend over ten or more.

- Success rate — the fraction of recent episodes that actually win. This is the number that matters for a real task, because one lucky episode can spike the best fitness without the agent being reliable.

A healthy run shows the fitness climbing, then the success rate catching up behind it.