Hacker News Reader

Comment by etaioinshrdlu | Hacker News Reader

These are actually just the problems easier for researchers to solve, mostly due to a lot of readily available data.

Everyone shares music and art they make but nobody ever shares videos and motion capture of themselves doing laundry and vacuuming their house. Maybe we need to start sharing that instead

The UMI gripper project is working on this. they have a handheld gripper device full of sensors that they use to record doing things in the field, like picking Starbucks, which they then use as training data.

https://umi-gripper.github.io/

The other thing to note is part of the aloha project isn't just to record people folding laundry and loading the dishwasher, but to take that data and plug it into a simulator with a physics engine, and use a digital twin to get 10x the amount of data to be used in training the model than if they'd just used real world data. So yes we need that data, but not as much as we would otherwise.

https://mobile-aloha.github.io/ https://github.com/tonyzhaozh/aloha

Check out the Epic Kitchens project, there are labeled video data sets of cooking, doing dishes, etc.

https://github.com/epic-kitchens

Meta has several first person pov datasets available

https://ai.meta.com/datasets/

Even teleops are janky as hell, robots needs bodies

That's indeed the second worst issue with current model architectures. For a model to be trained to something nearing usability for an actual task it needs an amount of data that is far beyond what can be obtained. Companies like Facebook and OpenAI downloaded pirated copies of every single book humans have written to reach the current level of text generation, and even with that, it's not like those models are perfect or that intelligent.

It is going to severely limit the possibilities of building actual agentic AIs. We do not have an endless amount of data of humans performing menial chores. And normal people will probably more hostile than the kool aid drinking software developers when it comes to being spied on, who's going to agree to wear a camera while working so as to help train their own replacement? Yet it's kinda what devs are doing gleefully adopting software filled with telemetry and interacting with copilot.

There will be no difficulty equipping people in minimum wage jobs with cameras. You could probably even get companies to pay to give you training data, if you sell it as an AI-powered system for reducing shrinkage or avoiding liability. The most likely source of pushback (that companies will care about) is likely to be from customers interacting with people wearing cameras, so it might be limited to non-customer-facing roles.

Another good source of data would be exoskeletons, though I don't know that any of those have actual seen real commercial success yet.

This isn’t the reason at all and comes across a weak attempt to make researchers stealing the work to train come across as blameless and helpless to circumstance.

They’re doing it because there is a lot of value to extract in making it so anyone can do these things regardless of talent or skill.

Reply on news.ycombinator.com