New test measures how well humanoid robots handle real-world forces

Researchers have introduced ThorArena, a simulation-based benchmark intended to test whether humanoid robots can remain accurate and balanced while handling the physical forces involved in everyday work. The benchmark addresses a limitation of many existing evaluations, which measure robot motion in empty environments but do not adequately assess performance when robots encounter resistance, weight, or pushes and pulls.

To build ThorArena, the researchers collected demonstrations of people completing daily tasks. Human operators wore virtual-reality headsets and motion trackers while lifting and lowering a water container, pushing and pulling a chair, and carrying an object with a partner. They used specialized 3D-printed hand tools with sensors to measure the direction of forces applied by their hands.

The recorded human motions were mapped onto a virtual humanoid in a physics simulator. The simulator then replayed the measured hand forces while the robot attempted to reproduce the movements. Performance was assessed with the team’s Force-Aware Tracking Score, or FATS, which combines tracking accuracy and balance. The evaluation covers six tasks.

Four representative humanoid control systems were tested. In simulations without physical forces, every system performed well and remained upright. When the recorded forces were applied, scores declined and notable differences among the systems appeared. These gaps were largely not visible in conventional no-force evaluations.

The researchers’ Thor2 system achieved the highest average FATS score among the four systems and maintained near-perfect balance across all six tasks. The team plans to broaden the dataset with more varied tasks and force conditions, then evaluate robots in real-world physical settings. The work is described in a paper posted on the arXiv preprint server.