Why the robot data gold rush is on borrowed time

With additional reporting from Pranav Balakrishnan.

A new side hustle has been making its way around Southeast Asia, one that doesn’t require learning a new skill.

Workers – whether they chop vegetables, pack boxes, or assemble laptops for a living – are strapping cameras to their heads and recording themselves doing their jobs. These videos are then sold to data firms, who in turn sell them to robotics companies which use these clips to train robots to do the same task.

On paper, this seems like a good deal for the workers. Siddharth Kumar, co-founder and CEO of Jakarta-based outsourcing company MyRobin, says a worker who records five to six hours a day could earn up to 3.3 million rupiah a month (about US$187). That’s roughly 50% to 60% of Jakarta’s minimum wage, which they can earn on top of the salary from their day jobs.

“You’re not doing anything extra,” Kumar notes. “It’s just good pocket money.”

The demand is coming from a simple gap in the robotics race: Robots need to learn how humans move through physical tasks, but much of that first-person data doesn’t exist online.

This has created a scramble for egocentric data, or video recorded from a person’s point of view. Robotics labs used this data to train vision-language-action models, which help robots understand what they are seeing and decide how to move.

The buyers include companies building robot “operating systems,” robotics firms involved in both hardware and software, and manufacturers preparing robots for specific factory or warehouse environments. That’s according to Manish Agarwal, whose Bengaluru-based company Humyn Labs works on collecting and preparing physical AI training data.

But the boom may have a built-in expiration date. While egocentric video is cheap to make and missing from existing datasets, its usefulness in training is questionable. In addition, these videos only teach robots based on what people can see, and sight can teach machines only so much.

In physical work, for instance, it’s important to know how much pressure the worker uses to hold an object without crushing it.

Albert Causo, co-founder and CEO of Singapore-based Hand Plus Robotics, says that most of the time, vision only helps robots identify where an object is, how to approach it, and whether the task is complete. That means in one hour of raw egocentric footage, what’s considered usable may be only five minutes or less, he argues.

Repeating the ChatGPT playbook

Sources Tech in Asia spoke to describe the demand for egocentric data as broad and still immature. Robotics labs want to train for domestic chores such as folding clothes and cleaning dishes as well as tasks within industrial and commercial settings.

The point is not simply volume. Buyers want diversity in the data across people, environments, geographies, lighting conditions, tools, and the kinds of mistakes that are made.

The problem of easy money

The future won’t be egocentric