Comment by HopenHeyHi | Hacker News Reader

Comment by HopenHeyHi | Hacker News Reader

3D reconstruction from a single image. They stress the examples are not curated, appears to... well, gosh darnit, it appears to work.

If it runs fast enough I wonder whether one could just drive around with a webcam and generate these 3d models on the fly and even import them into a sort of GTA type simulation/game engine in real time. (To generate a novel view, Zero-1-to-3 takes only 2 seconds on an RTX A6000 GPU)

  This research is based on work partially supported by: 
  
  - Toyota Research Institute
  - DARPA MCS program under Federal Agreement No. N660011924032
  - NSF NRI Award #1925157

Oh, huh. Interesting.

  Future Work

  From objects to scenes: 
  Generalization to scenes with complex backgrounds remains an important challenge for our method.
  
  From scenes to videos: 
  Being able to reason about geometry of dynamic scenes from a single view would open novel research directions -- 
  such as understanding occlusions and dynamic object manipulation.

  A few approaches for diffusion-based video generation have been proposed recently and extending them to 3D would be key to opening up these opportunities.

Seems like there is a bit of a gap between “runs at 0.5 fps on a $7000 workstation-grade GPU with 48GB of VRAM” and consumer applications.

With the fairly shallow slope of the GPU performance curve overtime, I don’t see them just Moores Lawing out of it either. this would need two, maybe three orders of magnitude more performance.

Computer power goes up exponentially thanks to Moore's law. Sprinkle some software optimisations on top, and it's conceivable for that to be running at interactive framerates on consumer GPUs within 5-10 years.

the processing may as well shift to the clouds. With the subscription fee, of course : )

Until we break the speed of light, I’m very bearish on cloud gaming. It just feels so bad. You’ve got like 9 layers of latency between you and the screen.

You don’t have to break the speed of light, just have the ping below human perception.

~20ms is that threshold, but even 40ms latency is barely noticeable for single player games.

A wireless controller ALONE is already over 20ms, and that’s before you touch the network, actually doing with that input, wait for the display to redraw…

At a 20ms total round trip, that only buys you about a 1500 mile radius, again completely ignoring all other latencies.

It's quite noticeable actually, and it adds up, it's not just an extra 20ms.

For casual gamers and turn based games maybe it could work, as a niche. For FPS, multiplayer, ARPG, and so on, it's a dealbreaker, anything over 100ms feels too sluggish.

We should be happy we have so much autonomy with our own hardware, I don't want some big cloud company to be able to tell me what I can play and render, unless we want the "you will own nothing and be happy" meme to become reality.

I actually, in my testing, JRPG/other turn based games were amongst the worst because there is so much “management” (inventory, loot, gear, etc) and the extra lag really throws you off

One possible definition of Edge Compute is GPU capacity at every last mile POP

I agree, though my last mile latency to the nearest POP is about 85ms. Still a bit on the high side for action games compared with playing locally.

85ms, holy crap, who is your ISP?

On Sonic fiber internet in San Francisco, I get 1.5ms to the POP. It is only 4.5ms to my VM in Hurricane Electrics Fremont DC.

> Computer power goes up exponentially thanks to Moore's law

If you look at a graph, that stopped being true well over a decade ago.

Only for single-threaded programs. Multi-threaded performance continued along the curve unabated.

Moore's law is specifically about the number (density) of transistors in an integrated circuit.

You could always get as much parallelism as you wanted by adding more chips.

Of course there is a gap. This is at the exploratory proof of concept stage. The fact that it works at all is what is interesting.

Furthermore once you've identified the make and model of the car, its relative position in 3d, any anomalies -- that ain't just a Ford pickup, it is loaded with cargo that overhangs in a particular way -- its velocity, 'etc -- I'm quite sure that extrapolating additional information from the subsequent frames will be significantly cheaper as you don't have to generate a 3d model from scratch each time.

I think this is a viable exploratory path forward.

    Make it work <- you are here
    Make it work correctly
    Make it work fast

Edit: Scotty does know ;)

I prefer:

    Make it work <- you are here
    Make it work correctly
    Make it work fast

>> fairly shallow slope of the GPU performance curve overtime

Not true.

Reply on news.ycombinator.com