What's in your hands? 3D Reconstruction of Generic Objects in Hands | Hacker News Reader

What's in your hands? 3D Reconstruction of Generic Objects in Hands

judyye.github.io

28 pointslnyan10 comments

I know even publishing the code for stuff like this is generous but it'd be so cool if the standard practice included publishing a little wasm binary blob as a library independent network with weights to play with without having to go through colab. (Many frameworks have a way of creating that automatically already, e.g. torchscript or jax jit.)

I am not as familiar with the state of the art; are there NN frameworks that export the NN model such that it can be run within WASM?

Yup. In torch you can use torchscript and in jax you can 'jit' a function and both turn your NN function into a fairly easily compliable form which could then be translated to wasm.

Here's a cool project that does something like this (though I believe it's basically superseded by torchscript): https://github.com/podgorskiy/tensor4 Here it is running stylegan in the browser (via wasm): http://podgorskiy.com/static/stylegan/stylegan.html

Note of reference for readers who may not be as familiar. One (pytorch) process is: 1) Use torch.onnx to convert the model to ONNX 2) USe `onnxruntime` built for Web Assembly (WASM) to run the NN in the browser

Well my point was that you could extract it into a wasm blob without onnx (library independent). See tensor4 as linked in the other thread.

Instead of working from a single 2D image as this work does, what is the state of the art for 3D reconstruction from multiple (non-calibrated) images? e.g. some casual shots of an object using my phone.

Photogrammetry apps, right?

It depends on the goal, but the whole family of NeRF techniques have been achieving some really interesting results.

probably Nvidia's latest Instant Neural Graphics Primitives paper (as far as I'm aware). they demonstrate near-instantaneous training of NeRF models:

https://nvlabs.github.io/instant-ngp/

  Dashcam threat sensors identified the suspect was holding a handgun-shaped object...

-A press conference in the near future

Reply on news.ycombinator.com