Overview
A line follower needs only a few sensor readings in and two wheel speeds out, so a deep network would be overkill. Tabular Q-learning learns a policy that is literally a lookup table, which can be ported to a microcontroller as a switch statement.
The first repository is a lightweight simulator with a Gym-style interface. The second ports the robot into Webots, where downward-facing cameras act as line sensors and the table trained in the simulator serves as a sim-to-sim starting point.
How it works
Sense
Five IR sensors, modelled on real sensor bars like the TCRT5000 and QTR-8A, each report how much of their footprint is dark, from 0 to 1.
Discretise
Each reading falls into one of three bins, giving up to 243 states, of which only about 50 to 60 are ever visited.
Act
Five commands, forward plus a soft and a hard turn each way, each mapped to a pair of wheel speeds on differential-drive kinematics.
Train and evaluate
Around 1,000 episodes train the table in about a minute on a CPU, then evaluation runs on random tracks the agent has never seen, with a manual WASD mode for comparison.
Port to Webots
A Python controller drives a custom differential-drive robot through a Webots world whose track texture is generated from a fixed seed.
What's next
- Measuring a real chassis and calibrating sensor thresholds for hardware
- Training directly against Webots physics
- Sensor and wheel noise during training