XPENG showed social skills of humanoid IRON

XPENG showed how its humanoid IRON communicates with people: it follows the speaker, switches languages ​​without a separate command, and recognizes a person even after he has changed clothes and moved.

In a video on the company’s YouTube channel, IRON conducted “sales” conversations with three testers in Chinese, English and German, turning his head, body and waist towards the one who was speaking.

Company demonstration, not verification

Filming in the studio is a demonstration of the company itself, and not an independent verification. But it shows in detail what makes up the perception, calculations and memory of the robot that XPENG is going to use to work with clients.

The company says one of IRON's first positions is as a store assistant. True, the release date for the “shift” was not announced there.

Speaker tracking

To understand where speech is coming from, XPENG uses an array of nine microphones to estimate the direction of the sound. Next, the multimodal system determines by lip movement who is speaking, and the tracking policy coordinates the robot’s head, waist and legs.

In the video, IRON turns its head slightly, more noticeably its waist, and sometimes turns its entire body when testers change position. In some cases, it takes a step towards the speaker rather than spinning around.

The main goal of the demonstration is to make the robot's posture less abrupt during a conversation. For a humanoid who works near customers or colleagues, this is a completely practical concern.

XPENG attributes this behavior to the coordinated work of acoustics, computer vision and motion control. But tracking accuracy, response latency, and behavior in background noise or in a large group are not measured in the video.

Multilingual responses and personality profiles

IRON responds in each tester's language, including switching between Chinese, English and German. According to XPENG, it works with a multilingual foundation model, retraining, and a balanced multilingual dataset. There were no additional voice commands for changing the language in the demo.

The recognition test turned out to be more difficult. After the three participants changed clothes and switched seats, IRON still associated each with their previous car preferences. And one tester, who introduced himself by a false name, was corrected by the robot.

XPENG claims that IRON creates a digital memory profile by combining facial features, voice prints and the content of conversations, and then updates it at subsequent meetings. But the company did not explain how long such profiles are stored, what is required for registration, and how they will be managed when deployed publicly.

Three Turing chips on board

According to XPENG, IRON features three locally developed Turing chips with a total effective performance of 2,250 TOPS. The system can access a local knowledge base or search for information on the network - so, as planned, the robot makes product recommendations based on past conversations.

The demo was structured around selling cars, with IRON bringing up family size, camping and parking issues, and then offering XPENG models. So the stated application becomes unusually specific.

But the clip doesn't show how this whole stack will work full time, in a crowded store and without pre-prepared participants and topics.