Zero‑WAM

Zero‑WAM In‑context world modeling for zero‑shot task generalization

[GitHub]

Robotic Task Generalization.

Zero-shot task generalization remains a central challenge for robotic manipulation, especially for multi-object, complex-object, and long-horizon tasks whose procedures are difficult to specify through language alone. Existing world action models rely primarily on language instructions, which often fail to capture the fine-grained temporal and procedural structure needed to execute such tasks in unseen settings. We introduce Zero-WAM, a framework for zero-shot task generalization through joint dynamics modeling over human demonstrations, robot observations, and robot actions. Instead of relying on text alone, Zero-WAM uses semantically aligned human videos as dynamic visual context, providing explicit procedural guidance for robotic tasks.

Data Pipeline.

Zero-WAM data pipeline

To support this capability, we build a scalable data curation pipeline that constructs a large-scale human-robot in-context learning (ICL) dataset with diverse task semantics. Each robot video is paired with robot actions and aligned with a semantically corresponding human demonstration, allowing Zero-WAM to learn how human procedural cues guide executable robot behaviors. This structure enables the model to infer task procedures from visual context and apply them to unseen task configurations without task-specific robot demonstrations.

Real-world Test.

After training Zero-WAM on the large-scale ICL dataset constructed by our pipeline, we evaluate its generalization ability in a complex multi-object scenario. We collect four tasks for post-training, using only 15 episodes per task. Under this low-data regime, we test in-domain performance on the four seen tasks and out-of-domain generalization on five unseen tasks.

In-domain Task ICL

Human Video
0:00
0:00
Robot Video
0:00
0:00

Out-of-domain Task ICL

Human Video
0:00
0:00
Robot Video
0:00
0:00