A robot learns the kullawada
A dance fitted to a robot is only a list of angles. To dance it the robot has to practise: thousands of copies of it try the kullawada at once in simulation, fall, and try again. This note is about that practice, and about the check that caught what the practice had missed.
The same eight seconds, three times
On the left, Lucía.
In the middle, the robot copies her. It looks right, and on a real floor it would fall at once: nothing in an angle keeps a body on its feet.
On the right, the robot has practised. This note is about the distance between the middle and the right.
How a robot practises
4,096 copies of the robot try the dance at the same time. Each copy is rewarded for staying close to the dance and for staying up. A copy that falls starts again.
One round of practice takes about a second. A dance gets 30,000 rounds. For 32 seconds of kullawada that is more than nine hours on one rented graphics card.
What comes out is a policy: a small neural network. 50 times a second it reads 154 numbers about its own joints, its tilt and the dance, and answers with 29 numbers, one for each motor.
A policy knows one dance. Another dance is another policy and another nine hours. Eight dances are eight policies, and with every time we trained one again, we spent more than 250 dollars in all.
Passing its own practice is not proof
A policy that stands in the simulator it practised in has only shown that it knows that simulator.
So each policy is run in a second simulator, built on the robot's official model. It runs 20 times, and each time the floor, the motors and the sensors are a little different.
The first time we did this, it went badly. In practice the eight policies had fallen between 0 and 5 times in 256 tries. In the second simulator they fell in almost every run.
What the check caught
First we blamed ourselves. We had changed the training recipe so that the robot would follow the dance more closely and cope with a rougher floor. We trained again without our changes. The kullawada then stood 17 times of 20. Better, and still not right.
Then we stopped changing things and compared. We ran the same policy in both simulators and put what it reads side by side, number by number. Four kinds of numbers matched exactly. One did not: the joint positions were off by 22 degrees.
A policy does not read its joints from zero. It reads them from a resting pose, and the two simulators were using two different ones. In practice the robot rested with its knees bent 38 degrees. In the second simulator, 17.

With the wrong resting pose, the fast dances stood 0 times of 20. With the right one, all eight dances stood 20 times of 20.
The rule we kept: when a policy stands in one simulator and falls in another, compare what it reads before touching anything else.
What we ask the robot to do
In practice we ask one thing 4,096 times at once: hold the 29 angles of the dance, 50 times a second, and do not fall. For the kullawada Lucía danced that is 46,313 angles in 32 seconds, with the hips turning 1,911 degrees in all and one foot rising 25 centimetres.
In the second simulator we ask for the same dance 20 more times, and each time we change the floor, the motors and the sensors a little.
Thanks
Paqarina is a Mozilla Fellowship project.
Thanks to Dr Pan Hui, Professor Xin Tong and Jie Yu for allowing us to test our policies on the robot.
What the record shows
| Measure | Value |
|---|---|
| Copies of the robot practising at once | 4,096 |
| Rounds of practice for each dance | 30,000 |
| One round | about a second |
| Practice for 32 seconds of kullawada | more than nine hours |
| What a policy reads | 154 numbers, 50 times a second |
| What it answers | 29 numbers, one for each motor |
| Falls in practice, the first eight policies | 0 to 5 in 256 tries |
| The same policies in the second simulator | fell in almost every run |
| Trained again without our changes | the kullawada stood 17 times of 20 |
| Joint positions, difference between the two simulators | 22 degrees |
| Knees in the two resting poses | 38 and 17 degrees |
| With the resting pose corrected | 8 dances of 8, 20 runs of 20 each |
What remains open
Whether one policy can learn more than one dance. Whether practice can include the pose the real robot arrives in. Whether a check on the real floor, with a hoist, can become as routine as the second simulator. Eight policies stand in two simulators. The next practice begins from the pose the robot stands in.