Ego4WAM: What Matters When Scaling Egocentric Human Data for Robot Learning?

Zhihao Sun1, Liu Liu2†, Xinjiang Wang2, Haoyi Jiang3, Wei Feng2,
Huiqiang Zhang4, Xiaosong Jia1, Zhizhong Su2, Zuxuan Wu1‡
1Institute of Trustworthy Embodied AI, Fudan University
2Horizon Robotics 3Huazhong University of Science & Technology
4Zhejiang University of Technology
† Project Leader    ‡ Corresponding Author
sunzhihao.18@gmail.com, zxwu@fudan.edu.cn
arXiv Code Model

Abstract

Egocentric human data provides a scalable source of experience for robot learning, but varies substantially in human-robot alignment, behavioral coverage, and available supervision. Existing work shows favorable scaling with increasing human data, but it remains unclear which data properties drive downstream robot gains and how to use such data throughout the training pipeline. We present a systematic study of egocentric human data with different alignment and supervision under a unified world-action model framework. With the model backbone fixed, we disentangle the effects of human-robot alignment, data duration and task diversity, action supervision, and data usage strategies. We find that aligned human demonstrations substantially improve out-of-distribution generalization and reduce target-task robot data requirements; data duration and task diversity affect downstream capabilities differently; and video-only supervision remains effective without action labels, providing a strong foundation for subsequent video-action training. We validate these findings through closed-loop policy evaluation on both real robots and RoboDojo. Rather than treating data duration as the sole scaling axis, Ego4WAM shows how alignment, task diversity, available supervision, and usage strategy jointly shape the value of egocentric human data for robot learning.

Closed-Loop Evaluation

Key Findings

01

Robot-aligned human data enables OOD adaptation and reduces robot data.

OOD Adaptation

Real-robot evaluation results for the Place into the Basket task

Real-Robot Task: Place into the Basket

Aligned human demonstrations expose the policy to deployment conditions absent from robot training. Without any object- or scene-OOD robot demonstrations, object-OOD success increases from 10% to 60%, while scene-OOD success increases from 0% to 20% with a substantially larger gain in task progress.

Reducing Robot-Data Dependence

Real-robot evaluation results for the Fold Cloth task

Real-Robot Task: Fold Cloth

With 100 robot demonstrations, aligned human data recovers 80% ID, 60% object-OOD, and 80% scene-OOD success. Human data alone is insufficient when robot supervision falls to 20 demonstrations, but broad multi-task mid-training makes the same limited downstream data effective, reaching 80%, 80%, and 60% success, respectively.

02

Scaling egocentric data does not always lead to better performance.

RoboDojo scaling results across data duration and task diversity

Simulation:

Data duration and task diversity are distinct scaling axes, and total hours alone do not characterize useful scale. Denser coverage of recurring tasks provides limited additional gains once common behaviors are sufficiently represented, while expanding aggressively into increasingly sparse long-tail tasks can introduce negative transfer and hurt precision-sensitive performance.

03

Egocentric data provides a strong dynamics prior for world-action learning.

Learning Dynamics Priors without Action Supervision

RoboDojo ablation results for video pre-training, video-action mid-training, and joint world-action learning

Simulation:

Video-only pre-training improves the average RoboDojo score from 6.39 to 14.13 and success rate from 3.15% to 9.45% without human action labels, extending usable data beyond the action-valid subset. The learned dynamics prior further supports video-action training, while joint future conditioning exploits it most effectively, reaching a 20.29 average score and 14.35% success rate.

Competitive Performance with Limited Robot Data

RoboDojo performance comparison between Ego4WAM and existing methods

Simulation:

With only approximately 90 hours of real-robot data and 15K hours of egocentric data, Ego4WAM achieves a competitive 20.29 average score and 14.35% success rate on RoboDojo. Because the model is not explicitly designed for long-term memory or VLM-based open-instruction following, Memory and Open remain its weaker categories; nevertheless, Ego4WAM achieves the best Precision score and success rate and the best Long-Horizon score among the reported methods.

Scaling Egocentric Experience

Data

Ego4WAM egocentric data pyramid from dataset sources to video-action and video-only supervision

Ego4WAM organizes heterogeneous egocentric experience according to the supervision that can be recovered reliably: approximately 4 hours of robot-aligned video-action data, 12K hours of quality-controlled multi-task video-action data, and 120K hours of multi-task video-only data. This pyramid allows the training pipeline to retain useful visual dynamics even when reliable hand-action reconstruction is unavailable.

Unified Learning Framework

Method

Ego4WAM dual-system world-action model and unified human-robot action space

We use a fixed dual-system world-action model to isolate the effects of data rather than architecture changes. A shared-attention backbone jointly models future visual states and actions, while a unified action space maps robot joint, gripper, and end-effector supervision together with the action signals available from egocentric human demonstrations.

Closed-Loop Evaluation

Real-Robot Results

Two representative rollouts are shown for each of six real-world manipulation tasks.
All clips are displayed at 2× playback speed.

Place into the Basket

Fold Cloth

Fold Towel

Stack Plates and Bowls

Tighten a Bottle Cap

Heat Food with a Microwave

BibTeX

@article{sun2026ego4wam,
  title={{Ego4WAM}: What Matters When Scaling Egocentric Human Data for Robot Learning?},
  author={Sun, Zhihao and Liu, Liu and Wang, Xinjiang and Jiang, Haoyi and Feng, Wei and Zhang, Huiqiang and Jia, Xiaosong and Su, Zhizhong and Wu, Zuxuan},
  journal={arXiv preprint arXiv:2609.40341},
  year={2026}
}