ASTM develops standard to evaluate robot performance in real-world environments
English translation from Unibot. Source published 2026-10-20.
ASTM International's Robotics Committee (F45) is developing a draft standard for assessing task performance of manipulators and mobile manipulators under controlled optical signal degradation conditions. The proposed method (WK100792) will measure how much the degradation of optical information affects the robot's performance of its tasks.
Why signal degradation is important
According to ASTM International member Aarati Noronha, in real-world conditions, the optical information this technology relies on can degrade - and the robot has to compensate for the degraded input.
This compensation adds system load, latency, power consumption, and computational complexity. And if compensation fails, the task fails, the risk to workplace safety increases, and human-robot interaction becomes less predictable.
“I’ve seen this myself in both warehouse palletizing and autonomous strawberry picking: many tasks that a robot can reliably perform under ideal conditions are much more difficult for it to do in the real world,” says Noronha.
Real World Stressors
“Varying light, reflective materials, interference, high temperatures - all these stress factors degrade information from optical sensors such as cameras.
A degraded signal acts as a distraction: machine learning models receive worse information as input. The proposed test method provides a repeatable way to measure performance under controlled degradation conditions - and thereby support responsible deployment in real-world environments."
Who will benefit from the standard?
Noronha says the method will be useful to robot manufacturers, perception and machine learning system developers, system integrators, end users and test labs.
By measuring how optical signal degradation affects task performance - including in edge cases that are difficult to adequately capture in training data - the proposed standard will help developers identify system limitations before field deployment.
The standard will support UN Sustainable Development Goal 8 - decent work and economic growth.