LAW

Unifying Language, Action, and World Modeling
for Agentic Physical Intelligence

Jiajun Cao^1,2,* Kai Tang^1,2,* Bohua Zhang^1,2,* Zezhong Qian^1,2,* Ying Li^1,2,* Xiaozhu Ju^2,* Xiaobao Wei^1,2 Jiaming Liu¹ Yang Ding^1,2 Sixiang Chen^1,2 Yiding Sun^1,2 Siyuan Qian^1,2 Tinghao Wang^1,2 Jixian Wu^1,2 Yichen Guo^1,2 Yi Zhang² Shilong Zou² Yong Dai² Jian Tang^2,‡ Shanghang Zhang^1,‡

¹ State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University ² Beijing Innovation Center of Humanoid Robotics

^* Equal Contribution ^‡ Equal Corresponding Author

Paper Code Hugging Face BibTeX

Puzzle

Domino

Organize

Long-horizon real-robot tasks

RoboTwin, 50 tasks: 94.21%LAW + Fast-WAM average success, +2.36 over Fast-WAM
RoboTwin, with OpenWAM-α: 95.56%LAW + OpenWAM-α average success, +1.96 over OpenWAM-α
Real robot, 3 long-horizon tasks: 65%Average success, vs. 28.33% for π₀.₅ and 25% for Fast-WAM

Overview

Abstract

World-action models jointly predict actions and future world states, but long-horizon manipulation further requires language reasoning to plan subtask objectives, assess execution progress, and guide action selection. We present LAW, a closed-loop VLM–WAM framework that unifies language reasoning, action generation, and world modeling through semantic subtask progress. Within the LAW agent, a hybrid-expert vision-language model plans the remaining subtasks, estimates the values of and fuses actions proposed by a world-action model, and asynchronously assesses execution status to support continued execution and subtask transitions. Within the world-action model, Mixture-of-Transformer Experts integrates modality-specific and shared computation for joint video–action modeling. Dual-stream hierarchical visual memory maintains temporal context for both models, while subtask-local value learning aligns action evaluation with semantic objectives and preserves supervision from successful prefixes of failed trajectories. LAW achieves 94.21% average success across 50 RoboTwin tasks, 95.56% with OpenWAM-α, and 65% on three long-horizon real-robot tasks.

LAW overview: semantic subtask planning, progress-aware control, and simulation and real-world results.

Method

Framework overview

LAW pairs a hybrid-expert VLM with a world-action model. The planner specifies the active objective, the WAM proposes action chunks, and the value expert selects actions. Execution feedback keeps the agent on the current subtask or advances the plan.

LAW architecture. Planner, state, and value adapters share a frozen VLM. MoTE combines private and shared video-action experts. The agent proposes actions, ranks and fuses them, executes a prefix, and monitors subtask completion asynchronously.

1Plan
2Propose
3Select & fuse
4Execute
5Monitor

Hybrid-expert VLM

Planner, state, and value experts share a frozen VLM backbone with separate LoRA adapters. Each specializes in a control function while reusing the same visual-language representations.

Mixed-frequency coordination of planner, state, value, and action generation

Joint video–action modeling with MoTE

Video and action streams exchange information through joint attention. Modality-specific experts preserve their distinct features, while a shared expert learns common physical dynamics. MoTE strengthens joint video–action modeling by learning shared temporal and physical structure beyond attention-based information exchange.

MoTE architecture with private video and action experts and a shared expert

Dual-stream hierarchical memory

The VLM and WAM retain the same observation timestamps in their own token spaces. Short-term memory and boundary memory retain full-resolution visual tokens, while long-term memory stores spatially compressed tokens.

Short-termBoundaryLong-term

Dual-stream hierarchical memory: short-term memory and boundary memory retain full-resolution tokens, while long-term memory stores spatially compressed tokens

Subtask-local value learning

A successful subtask receives a terminal reward of 1. Value targets stop at semantic boundaries, so completed stages still provide useful supervision when a later stage fails.

32candidate action chunks

8top candidates fused

A block-stacking episode completes the first two subtasks and fails the third. Subtask-local supervision preserves positive targets for the completed stages, whereas episode-level supervision gives zero return.

Real world

Real-world experiments

Success rate (%)

Task score (0–100)

Simulation

RoboTwin results

Method	Clean (%)	Randomized (%)	Average (%)
X-VLA	72.9	72.8	72.9
π₀.₅	82.7	76.8	79.8
Qwen-RobotManip	93.4	92.5	93.0
Motus	88.7	87.0	87.9
LingBot-VA	92.9	91.5	92.2
LingBot-VA 2.0	93.8	93.4	93.6
Flex-π	94.30	94.80	94.60
Fast-WAM	91.90	91.80	91.85
LAW + Fast-WAM	94.22	94.20	+2.3694.21
OpenWAM-α	93.74	93.46	93.60
LAW + OpenWAM-α	95.60	95.52	+1.9695.56

Bold: best result in each column. Green tags: gain of LAW over the backbone it builds on. Click a column header to sort.

Citation

BibTeX

@article{cao2026law,
  title   = {LAW: Unifying Language, Action, and World Modeling for Agentic Physical Intelligence},
  author  = {Cao, Jiajun and Tang, Kai and Zhang, Bohua and Qian, Zezhong and Li, Ying and
             Ju, Xiaozhu and Wei, Xiaobao and Liu, Jiaming and Ding, Yang and Chen, Sixiang and
             Sun, Yiding and Qian, Siyuan and Wang, Tinghao and Wu, Jixian and Guo, Yichen and
             Zhang, Yi and Zou, Shilong and Dai, Yong and Tang, Jian and Zhang, Shanghang},
  journal = {arXiv preprint arXiv:2610.23334},
  year    = {2026}
}