A method has been developed for teaching robots to effectively collaborate with people

People are difficult to model. We think quickly and complexly, and our exact actions are impossible to predict. How then can we train robots to work side by side with us?

Typically, in simulation-based learning environments, the human is the fixed input parameter. Simply put, he performs consistent and predictable actions that bear little resemblance to actual human behavior.

Hao Zhang, a postdoctoral researcher in Ding Zhao's Safe AI Lab, is working on improving human-robot interaction using HALO - Lyapunov Policy Optimization for Heterogeneous Agents.

What is HALO

HALO is a framework that uses multi-agent reinforcement learning to enable robots to independently learn to interact and cooperate with humans.

During the simulation, the technique imagines the human partner as a learning robot. This is how both sides learn to interact with each other from scratch - without fixed input and without prescribed scripts.

“Effective human-robot collaboration should reduce, rather than create, additional training requirements for humans,” says Zhao, an assistant professor of mechanical engineering.

"Human partners naturally differ in the way they move, communicate and make decisions, so forcing everyone to follow a prescribed interaction pattern will limit the robot's usefulness."

Gradually they learn to cooperate with each other. And so, when a person behaves unusually or unexpectedly, the robot already knows how to adapt and respond to unfamiliar movements.

Interaction between a humanoid robot and a person. Photo: College of Engineering, Carnegie Mellon University

Training without scripted human behavior

“Instead of teaching a person to behave a certain way for the sake of the robot, we train the robot in an open interaction environment where different behavior patterns of the partner can emerge,” says Zhang, a postdoctoral researcher in the Department of Mechanical Engineering.

In a rescue operation or hospital ward, it is critical that transfers and other manual labor are performed safely and securely. If robots are transporting a patient in a hospital, they must respond accurately to each other and to all other participants to provide safe and effective care.

"When you transport something, you are physically connected to a partner through the object," Zhang says. "Even small discrepancies in timing or movement can cause agents' learning processes to interfere with each other, making effective cooperation difficult to learn. Stability is critical in the learning process."

Where coordination breaks down

Researchers have shown that with the help of HALO, these robots can walk around furniture and walls, while dragging large and inconvenient objects. They quickly and accurately respond to weight shifts or unexpected movements of a person - changing their stance to better distribute the load, or turning stronger or weaker than expected, adjusting to the trajectory of their partner.

But during the learning process, a problem emerges - a gap in rationality. Both have the same goal, but they study separately. And so everyone develops their own idea of ​​how best to interact. A strategy that makes sense from the point of view of one participant may not be suitable for the team as a whole.

Take, for example, a sofa that needs to be dragged around the corner. One will try to turn the sofa onto its side so that it stands upright. At this time, the other one tries to hold the sofa horizontally and move it forward. Both end up wanting to move the same object in the same direction - and the task is delayed because the methods interfere with each other.

A stability check for teamwork

To deal with this, Zhang applies Lyapunov stability theory. This is a guarantee of safety: it ensures the convergence of learning processes for each participant and redirection to a trajectory that is optimal for the entire team. Simply put, each observes the other's movements, analyzes them, and adjusts their behavior to reach agreement.

In the laboratory, everything started with computers and two robots - as in most other simulations. This eliminated the problem of implausible, programmed human behavior. When the robots learned to work smoothly with each other, one was replaced with a living person.

“In HALO, human variability becomes part of the robot's learning process - the burden of adaptation is shifted from the human to the machine,” explains Zhao.

Now they continue to prove the scalability of their method. The plans are as follows: gradually introduce more and more robots and people who interact with each other. By September, up to four robots should be able to work together.