Nima Ajam Gard13 min read

What's Missing from Robot Learning (Part I)

In this series, I examine what is missing from today's dominant approaches to robot learning. My focus is pragmatic: what is the next meaningful step toward scalable, reliable physical AI systems that actually work in the real world.

Artistic representation of learning to snowboard - Physical AI robot learning analogy
Learning a physical skill like snowboarding requires more than observation—it requires embodiment.
Three components of physical AI robot learning: Imitation Learning, World Model, and Reinforcement Learning
The three components of robot learning map to different aspects of skill acquisition.
https://path-robotics-1.wistia.com/medias/i13jzgw6ot
Scott McCloud Understanding Comics illustration about self-awareness and extended identity in physical AI world models
From Scott McCloud's "Understanding Comics" [11] - our awareness of self flows outward to include objects of our extended identity.
NVIDIA physical AI simulation ecosystem diagram showing Physics Simulation, Asset Generation, and Skill Training axes
The NVIDIA ecosystem spans three axes: Physics Simulation (Newton/Warp, Isaac Sim), Skill Training (Isaac Labs), and Asset Generation (Replicator).
Lucid Sim generating diverse simulated training scenes for physical AI robot learning
Lucid Sim [14] generating simulated scenes - both approaches can generate effectively unbounded data for robot learning.
Physical AI world model training workflow: Build Scenes, Interaction, and Train in parallel
A world model training workflow: Build scenes from real-world videos, add robots for interaction, and train in parallel with various textures, scenarios, and lighting.