Figure unveils Helix 2.5 after testing in 30 unfamiliar homes

Figure introduced Helix 2.5, a humanoid control policy that the company said handled making the bed, folding towels and cleaning the living room in 30 unfamiliar homes in the San Francisco Bay Area. The overall success rate is 56%. By comparison, a similar policy trained without the Index human behavior dataset achieved just 9%.

What exactly was tested?

The result was obtained in zero-shot mode, but with an important caveat. The houses, layouts, and manipulation objects were unfamiliar, but the three tasks were represented in pretraining data collected elsewhere. Figure used a fixed checkpoint for each task across all evaluation houses - no local data collection, weight updating, or post-deployment adaptation.

Key facts

Rating: 30 Unfamiliar Homes in the San Francisco Bay Area

Tasks: making the bed, folding towels, cleaning the living room

Helix 2.5 success: 56%

Policy without Index pretraining: 9%

Total attempts: 420, according to Startup Fortune

Strict completion criteria expose unevenness

Figure did not award partial points. Successful cleaning required the robot to put all 13-15 toys into the basket. Folding the towels involved folding and stowing each towel. To make the bed - so that both pillows and corners of the blanket are at the head of the bed, and the blanket itself is smoothed.

According to Startup Fortune, the robot completed 237 out of 420 attempts. Making the bed performed best, with 67%, or 94 out of 140 attempts. Folding towels - 62%, or 87 out of 140. Cleaning toys - 40%. Such a spread suggests that the aggregated result should not be read as uniform everyday competence.

The evaluation objects were separated before testing, says Figure, and checked against the task specification data set through an AI model, followed by human verification. The initial object configurations varied between trials. Timeouts were applied to individual objects or task elements, and any run that required human intervention for safety was recorded as a failure.

The physical scope here is broader than a test of manipulating objects on a table. The behavior combines walking, postural adjustments, bimanual manipulation, and perception in furnished spaces. Figure also showed high-quality examples of the robot stepping back, changing position, and walking around the bed to correct an error. However, the company did not provide a separate indicator of recovery after failures.

Pretraining on Index gives most of the gains

The purest experiment compares two policies trained on the same task data. The architecture, optimization, hyperparameters, and evaluation remained unchanged. One policy started with random weights, the other started with Helix 2.5 weights pre-trained on Index, the Figure dataset of human behavior.

The version with random initialization achieved success in 9% of zero-shot tests versus 56% for the policy with pre-training on Index. Figure states that no single assessment task takes up more than 1.90% of the Index set - an attempt to show that the pretraining material was broad and not limited to the three cases tested.

Figure also compared Helix 2.5 with the previous Helix 02 policy on the same task. Helix 2.5 reportedly matched its success rate, using half the amount of pre-training data for a specific task, while working in unfamiliar homes. The company did not disclose the absolute amount of additional training data in the article, so the statement remains a relative comparison with one representative behavior of Helix 02.

Scaling measures prediction, not execution

Figure trained four models on nested subsets of Index with eight times the size of the pre-training data, while keeping model size and post-training constant. The loss in predicting robot actions on a held-out sample decreased with each doubling. On smaller runs, the company said it predicted losses for the largest model to within four decimal places - with an error equal to 0.54% of the spread across the entire range of data.

This is evidence of predictable human-to-humanoid transfer in the tested configuration. But the metric here is the loss in predicting actions, and not the success of autonomously performing everyday tasks. Establishing the same scaling relationship for completion rates across more behaviors and environments would be a much more severe test.

Figure says Index adds about 35 minutes of human experience every second, and the company has committed $3.5 billion in computing power to train Helix. Helix 2.5 gives this scaling strategy a solid internal benchmark. But 56% completion leaves a fair distance between generalization and reliable housework.