A demonstration of Reinforcement Learning where a bipedal walker learns to walk from scratch.
The walker is a Ragdoll simulated with Verlet Integration.
Verlet Integration
It has mass points connected by constraints.
Muscles are active constraints that can expand or contract, generating force to move the legs.
Each walker has a unique Neural Network.
Inputs (10): Torso angle, height, and relative positions of feet/knees.
Hidden Layers: A deep structure [16, 12] processes these inputs.
[16, 12]
Outputs (6): Signals to contract or expand each muscle.
We start with random brains. Most fail immediately.
Selection: Walkers that travel the furthest are selected as parents.
Crossover & Mutation: Offspring inherit traits from parents, with slight random mutations to discover better strategies.
Red Muscles: Contracting (Pulling).
Blue Muscles: Expanding (Pushing).
Top-Left corner shows the Brain Activity of the best walker.