Overview
Training a single agent says little about how stable an algorithm is. This trainer runs up to twelve DQN agents at once, each in its own thread with its own dashboard panel, so the differences between runs are visible as they happen.
Runs can start fresh, resume from checkpoints or fine-tune saved weights with reduced exploration. When training ends, the best agent plays five showcase episodes, which can be recorded to MP4.
How it works
State
A 14-value observation: danger straight ahead, left and right at one and two steps, the current direction as one-hot, and which way the apple is.
Reward
+10 for an apple, −10 for dying, and +1 or −1 for moving towards or away from the apple. An episode ends on a collision or after too long without eating.
Learn
DQN with two 256-unit hidden layers, a 100k replay buffer, batch size 128, γ = 0.95 and a target network refreshed every 1,000 steps, with ε decaying from 1.0 to 0.01.
Watch
Every agent's panel shows score, best score, episodes, ε, average score and progress, while checkpoints save every 10k steps.