Article: Google DeepMind’s RT-2, Stanford-Toyota’s OpenVLA, NVIDIA’s GR00T, Hugging Face’s LeRobot and Physical Intelligence’s Pi-Zero give robots the ability to link visual input, language commands and motor actions in a single model. That capability replaces hand-coded step sequences with “see-and-do” behavior, promising faster development and broader access to robotics.
Why vision-language-action matters
For most of robotics history, engineers wrote explicit scripts that told a robot each joint movement. The new class of Vision-Language-Action (VLA) models treats a robot like a conversational agent: a camera streams an image, a user speaks or types a request, and the model spits out the motor commands needed to fulfill it.
The five models on the block
- RT-2 (Google DeepMind) – Connects raw visual streams to human language, letting a robot describe what it sees and act on spoken instructions.
- OpenVLA (Stanford + Toyota) – An open-source release that lets any developer plug the model into their own hardware.
- GR00T (NVIDIA) – Built for humanoid platforms, it reasons about the scene and generates low-level motor commands.
- LeRobot (Hugging Face) – Supplies a data repository and toolset that speeds up model training and integration for robot developers.
- Pi-Zero (Physical Intelligence) – Says a single “brain” can be reused across different robot bodies, simplifying software stacks for varied hardware.
Data, simulation and the drift problem
All five models lean on massive datasets, most of which come from simulated environments because real-world collection is costly and risky. That reliance creates a “sim-to-real” drift: behaviours learned in perfect virtual worlds may falter when faced with noisy sensors, uneven lighting or unexpected obstacles.