Hacker News Reader

Comment by faeyanpiraat | Hacker News Reader

I believe this is a key component to true self driving cars.

Being able to imagine correctly what is behind stuff is a required skill for safe driving.

Basing life-or-death decisions on the computer vision interpretation of a synthetically produced image of what may or may not be behind an object sounds like a recipe for disaster, as if the current state of the field wasn't disaster enough.

Just thinking of the amount of adversarial "speedrunner" type of scenarios gives me a headache.

On the other hand thats what people(brain) do all the time.

Potentially explaining the disastrous results

Your brain takes decisions based on synthetically produced images? Then maybe you shouldn't drive on LSD ;)

The contents of your visual field exist precisely in your head and nowhere else. They are synthesized from limited sensory data. It so happens that your brain also generates the sensation of the content of your visual field being located in the world around you. These captured and resynthesized visual data happen to be mostly accurate but also wildly interpreted and altered, for instance the majestic and wondrous and inexplicable addition of the sensation of color.

Keyword: precisely

Your brain fills a lot of gaps, and your senses have a lot of noise, but it's still closer to reality than it isn't, particularly for the fovea.

Still, your brain doesn't make up what's behind a building violating our reliance on straight lines of light (unless you use sensorial "enhancement" drugs), or make up detailed images of what's in your peripheral vision (again, a known effect of many psychedelics), then presents it to you as truth, that's rather imagination and it uses a very different part of the brain, though connected to the visual cortex through the V1 area.

I'm sure it can have some uses (AI racing, anyone?), but most self-driving car decisions are ruled by what has been seen (signs/lights) and what can be seen, because that's how we arranged our streets and roads to be safe for us.

> your brain doesn't make up what's behind a building

The brain imagines something based on prior experience. Otherwise you'd be unable to navigate complex environments.

Everyone questioning this - if much rather a car have some idea what the things it can’t see might be. It doesn’t have to give as much weight to them as things it can see for certain. But this kind of predictive knowledge is what makes humans much better than AI in a lot of areas. It’s what we normally call ‘common sense’

There's no understanding happening here. As I understand it only provides a way to render images from novel views without any 3d/2d segmentation.

No understanding yet, but it's a step in that general direction.

This neural net could create a realistic 3D environment where AI agents could train and explore. The lack of embodiment is the main thing missing from reinforcement learning agents to close the gap to humans. The environment itself can be controllable by embeddings, spanning any conceivable situation. Of course, ideally, if this architecture could be made much more efficient.

Reply on news.ycombinator.com