Humanoid robot mastered sprinting and roundhouse kicks with the help of AI

Humanoid robots could help people with physical work in a variety of situations. True, so far most of these machines confidently perform only a limited set of movements.

Researchers at the University of California, Berkeley (UC Berkeley) and Stanford University have created BeyondMimic, an AI system that expands the repertoire of humanoid movements and reduces the need for customization for specific tasks.

Why was it difficult

Teaching a humanoid a variety of dexterous, natural movements is a complex and time-consuming task. One way is to train controllers (algorithms that control the robot's movements) on a large array of human demonstrations.

This approach allows you to reproduce some movements, but sometimes gives unnatural results or requires adjustment to each individual action. In addition, many previous methods worked through learning or fine-tuning control policies for specific movements and goals - and did not transfer well to new tasks.

The new framework is based on reinforcement learning plus a model that builds movement plans by refining noise within a compressed motor representation.

How BeyondMimic works

To train the system, the researchers took about 2.5 hours of data on people's movements: walking, running, dancing, martial arts, jumping, cartwheels and other dexterous elements. These movements were then adapted to the proportions and joints of the G1 humanoid from Unitree Robotics.

The team trained control policies using reinforcement learning to replicate the positions, orientations, and velocities of body parts in each reference movement. Reinforcement learning is a widely used method of training computer algorithms through rewards and penalties. In this case, the model received a reward for accurately tracking movements, and penalties discouraged the desire to make sudden jerks, take dangerous joint positions, and allow unwanted contacts between parts of the robot’s body.

“We present BeyondMimic, a framework that scales to a variety of movements and has the flexibility to seamlessly combine them to solve never-before-seen problems,” Qiyuan Liao, Takara E. Truong and their colleagues wrote in their paper. “The compact formulation of motion tracking enables the mastery of a wide range of highly maneuverable behaviors, including aerials, spinning kicks, flip kicks and sprints, all with a single setup and common hyperparameters, while achieving human-like performance.”

To compress the actions the robot had learned, the team used a variational autoencoder, an algorithm that translates action data into a compact and structured representation (that is, a lower-dimensional latent space). The decoder then converts these compressed representations into commands for the robot.

The researchers then trained an algorithm known as a diffusion model to recognize sequences containing both the robot's predicted physical states and compressed representations of movements. During training, they added noise to these sequences, and the model learned to reconstruct the original movements.

“Moving beyond simple imitation of existing movements, we propose a unified latent diffusion model that enables flexible goal setting, seamless task switching, and dynamic composition of these dexterous behaviors,” Liao, Truong and their colleagues wrote. “Using classifier guidance - a diffusion-specific optimization technique during testing against unseen targets - our model expanded its capabilities to solve downstream tasks never encountered during training, including motion recovery, joystick telecontrol, and obstacle avoidance, and transferred these zero-shot skills to real-world hardware.”

First demonstrations and future opportunities

The researchers first tested the movements in the simulation and then deployed 30 representative clips of the movements on the Unitree G1 robot. Their results were promising, with the robot successfully performing a variety of movements including balancing poses, dancing, crawling, sprinting, kicking, spinning jumps and aerial cartwheels.

During these experiments, the researchers also asked 77 participants to compare the walking and running motions generated by BeyondMimic with those generated by the standard Unitree controller. In 70.8% of comparisons, participants rated the BeyondMimic's movements as more human-like and natural than those provided by a standard controller.

In the future, the team's framework can be further developed and tested on an even wider range of movements. It could also potentially be adapted for other humanoid robotic platforms.