Article: The frontier of physical AI hits a massive bottleneck: we lack high-fidelity, real-world training data. Large Language Models grew by scraping the internet’s text; robotics needs a far costlier approach.

The Data Scarcity Problem in Robotics

For humanoid and warehouse robots to match human dexterity, they need a data scale that simply isn’t there. Vineeth Velmurugan, Encord’s head of robot learning and a veteran of OpenAI’s robot lab, says breakthrough requires a dataset about five times the size of YouTube’s entire video corpus.

Text is abundant and free to scrape. Physical data must be manufactured. Training from video alone often misses the fidelity needed for complex manipulation. The industry has therefore shifted from tweaking model architecture to solving the much harder problem of manufacturing high-quality, annotated physical data.

Beyond Video: Using Brain Waves and Muscle Signals

Startups like Encord are testing new data modalities to give robots deeper context. In a pilot with German neuroscience startup Zander Labs, Encord equips operators with brain-wave headsets. By measuring neural activity, researchers hope to infer intent, error and surprise.

Lucas Gehrke, a neuroscientist at Zander, says tracking brain activity lets model builders know exactly when a robot should fire its most powerful computational models for a tough task.

Encord also explores:

  • Electromyography (EMG): Sensors strapped to the forearm capture electrical signals in muscles, creating a 3-D map of hand movements that head-mounted cameras often miss.
  • Leader-Follower Rigs: Paired robotic arms where one mimics a human operator, capturing precise actions such as pouring liquids or stacking poker chips.
  • Dense Annotation: Labels like “right hand tightens bolt.” Velmurugan estimates this dense annotation is 100 × more valuable than raw ego video for training specific tasks.

The Economic Reality of Physical AI

Moving from digital-first AI to physical AI flips the economics of machine learning. LLM labs built models at near-zero marginal cost by pulling from the web. Producing robot training data demands hardware, human operators and painstaking annotation. Encord aims to make high-quality data 20 × more cost-effective than its delivered value, yet even that translates into multi-million-dollar projects for large robot fleets.

Companies that generate massive, richly annotated datasets first will dominate autonomous manipulation—from plugging in Ethernet cables to picking and packing items. Those that cling to video-only pipelines risk falling behind as competitors harvest richer signals from the human body and brain.

Key Takeaways

  • Data Manufacturing vs. Collection: Physical AI requires the expensive, manual manufacturing of high-fidelity datasets, unlike LLMs that scrape existing internet data.
  • Neurological Modalities: Adding brain waves and EMG lets robots grasp human intent, error and 3-D hand movements more effectively than video alone.
  • The Scale Challenge: Robotics models need a dataset far larger than the entire YouTube archive; data generation is now the primary business frontier.

Encord has begun a pilot with German neuroscience startup Zander Labs that equips operators with brain-wave headsets and forearm EMG sensors to capture high-fidelity training data for physical AI. The partnership aims to produce a dataset large enough to rival five times the total volume of YouTube videos—a scale researchers say is required for robots to reach human-level dexterity and could reshape how robotics learns.

Why robotics data is a bottleneck

Large language models grew by scraping the open web; the raw material was abundant and cheap. Physical AI, by contrast, needs data that must be manufactured. Every grasp, twist or pour must be recorded, annotated and linked to the forces and intentions that produced it. Ordinary video often misses the subtle cues needed for complex manipulation, leaving a gap between today’s models and the demands of factories, warehouses and homes.

Vineeth Velmurugan, Encord’s head of robot learning and a former OpenAI robot-lab veteran, says the field will not move forward until a dataset roughly five times the size of YouTube’s video corpus exists. The problem is no longer model architecture; it is how to manufacture the data.

Adding brain waves and muscle signals

The Encord-Zander pilot tries to fill that gap by adding physiological signals to the visual record. Operators wear headsets that read electroencephalography (EEG) – the brain’s electrical activity – while EMG sensors on the forearm capture muscle impulses. The goal is to infer mental states such as intent, surprise or error, and to translate raw muscle activity into a three-dimensional map of hand motion that video alone cannot capture.

Lucas Gehrke, a neuroscientist at Zander, explains that knowing when a human anticipates a difficult sub-task lets a robot allocate its most powerful computational models at the right moment.

Beyond neuro-data, Encord tests a handful of complementary techniques:

  • Electromyography (EMG): Sensors on the forearm produce a live read-out of muscle activation, enabling a precise reconstruction of finger trajectories that head-mounted cameras often miss.
  • Leader-follower rigs: A pair of robotic arms work in tandem, one mirroring a human operator’s motions. This captures delicate tasks—pouring liquids, stacking chips—where millimetre-scale errors matter.
  • Dense annotation: Instead of labeling only high-level actions, Velmurugan’s team attaches fine-grained descriptors such as “right hand tightens bolt.” He estimates this detail is about 100 × more valuable for training a specific manipulation skill than raw ego-centric video.

The economics of data manufacturing

Physical AI changes the financial calculus of machine learning. LLM labs built models at near-zero marginal cost because the internet supplies endless text. Producing robot training data, however, requires hardware, human operators and painstaking annotation. Encord’s internal target is to make high-quality data 20 × more cost-effective than the value it delivers to customers, but even that aggressive efficiency still translates into multi-million-dollar projects for large-scale robot fleets.

The stakes are clear: companies that can generate massive, richly annotated datasets first will dominate the emerging market for autonomous manipulation—whether that means plugging in Ethernet cables on a data-center floor or picking and packing items in a distribution centre. Those that continue to rely on video-only pipelines risk falling behind as competitors extract richer signals from the human body and brain.

Counter-points and open questions

Not everyone is convinced that neuro-signals are the missing piece. Critics argue that the added hardware complexity could outweigh the benefits, especially when video plus force-torque sensors already deliver decent performance for many industrial tasks. Scalability is another concern: outfitting thousands of operators with EEG headsets and EMG rigs may prove prohibitive for smaller manufacturers.

The data-generation pipeline remains labor-intensive. Even with leader-follower rigs, capturing the breadth of tasks needed for a truly general robot will demand a sustained, coordinated effort across multiple labs and factories. The market will have to decide whether the incremental performance gains justify the upfront investment.

What to watch next

  • Data volume milestones: Encord’s claim of a dataset five times the size of YouTube will be a concrete benchmark. When disclosed, other players will have a clear target to match or exceed.
  • Hardware adoption rates: The speed at which EEG and EMG devices become standard in data-collection rigs will indicate whether the approach scales beyond experimental pilots.
  • Cost metrics: If Encord can demonstrably deliver the promised 20-fold cost advantage, it could spur a wave of new entrants focused on “data manufacturing” rather than model design.
  • Performance gaps