Humans routinely look first and act second—our eyes lock onto a goal, the brain runs a brief visual “simulation” of the outcome, and only then does the body move.
At every moment, our egocentric view both serves as input from the environment and reflects the intention/goal behind the next movement.
When we consider our body movements, we should consider both actions of the feet (locomotion and navigation) and the actions of the hand (manipulation), or more generally, whole-body control.
Actions of the Feet
Actions of the Hand
Whole-body Control
Same view can lead to different movements and vice versa — because humans act in complex, embodied, goal-directed environments.
Full-body motion spans 48+ DoF with hierarchical, time-dependent dynamics—not synthetic control codes.
First-person vision reflects goals, but not motion execution—models must infer consequences from invisible physical actions.
Visual feedback often comes seconds later, requiring long-horizon prediction and temporal reasoning.
@misc{bai2025wholebodyconditionedegocentricvideo,
title={Whole-Body Conditioned Egocentric Video Prediction},
author={Yutong Bai and Danny Tran and Amir Bar and Yann LeCun and Trevor Darrell and Jitendra Malik},
year={2025},
eprint={2506.21552},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2506.21552},
}