Hacker News Reader

Comment by AndrewKemendo | Hacker News Reader

Rapid generation of high quality 3D models of existing objects. Process should be independent of object size eg. a coke can should use the same process as a car and process time should scale with object size.

Think somewhere on the order of 10,000 models per day throughput.

There's $BNs waiting for you. It's ridiculously hard.

View on HN

This might already exist depending on your exact requirements and it's a fairly common technology in the world of metrology. I regularly work with manufacturers to reverse engineer and/or measure molds, jigs, and fixtures for which there are no drawings available. A ROMER or FARO arm with a laser scanning head outputting its point cloud to a software like the PolyWorks suite can generate an incredibly accurate CAD model of incredibly large parts in a very short amount of time (an hour or two at worst if the mesh needs a lot of cleanup).

I assume that process would be easy to speed up if the requirement for absolute accuracy was removed. The 8' ROMER arm we use is accurate to ~!2 microns over its entire volume which is absolutely overkill for something intended to produce models for visual arts applications. A quick and dirty approach to generating the mesh might increase the inaccuracy by several orders of magnitude but when coke can has dimensional tolerances to the tune of tenths of a millimeter, the quick and dirty mesh will still be representative of the end product.

Unfortunately not. FARO and other structured light systems don't export texture and are generally too precise (micron) in current form. So they take too much post processing by default.

There's $BNs waiting for you.

Who would be the primary customers? The entire 3D capturing market is currently several $B per year, including services. Where would be the customers that aren't getting served today that would double this market?

Well it would siphon everything away from the existing 3D capture industry and open it up to smaller groups and those that aren't savvy on it yet. The consumer space generally isn't doing this so anyone that sells anything would get on board at a low enough price point and simplicity.

I think Intel has been trying to do this for a while with their realsense technology and accompanying cameras.

http://www.intel.in/content/www/in/en/architecture-and-techn...

Mapillary has also been moving quickly in this space, not for objects per se, but entire environments.

https://github.com/mapillary/OpenSfM

Not really. I know some folks on the RS team and they don't really develop around applications, they are more focused on miniaturizing and making RS more available and lower power.

That said, some people have tried to use RS for this problem, but from what I've seen end up just using Kinects.

My friend's working on this, for generating 3d floor plans from point cloud data. It's a pretty complex problem.

That's a different problem set all together actually.

But yea, there are a lot of us working on that.

What are the best approaches you've found? Specifically for generating clean, useful NURBS/poly mesh models from lidar or SfM dense point cloud data

My colleagues at Creaform have something that works pretty well. Their latest handheld scanner can generate a wire mesh on the fly with 0.030mm resolution.

http://www.creaform3d.com/en

10K scans/day is way beyond their limits, and I'm not sure that's a very common use case. But I bet they could get there if they wanted.

I'm sure I speak for many of my fellow fans of physics when I say that technology capable 8.6 seconds per scanning a complex smaller object would have applications beyond just making 3D models.

What is the use case for the models themselves?

Anyone who does work with 3d content: visual effects, video games, vr, ar, etc. Being able to quickly build your scenes from a huge library or accurate models would be amazing and would save businesses lots of money.

There are multiple but mine specifically is AR. It's valuable also for VR, 3D space planning for Designers/Architects/Engineers, Assets for Game Dev, Objects for modeling and simulation, Training Deep Vision nets and on and on...

This combined with 3D printers would make fixing stuff much easier by printing the broken piece.

Product development engineer here. In the early stages of a project it can be useful to have CAD models of a competitor's product when analyzing how to improve upon them. Recently we had an intern reverse engineer a competitor's product, and we've used some of these CAD models as the basis for our new designs.

Would it be useful to capture the full appearance properties of the model (BRDF) etc to be able to accurately render the object?

What kind of structural integrity do you have in mind? Something with the density of industrial packing foam? I've seen set pieces constructed / carved from such material and it can be painted quite well. Putting aside the environmental / toxicity concerns for a moment regarding the type of material to be used, I'm genuinely curious how "rigid" such pieces might need to be.

Perhaps I should have specified that I am talking about Computer Generated 3D models, not physical models.

Oh, so you mean like a big box that could fit XYZ items inside it and capture something like 10,000 per day? I mean, to me it's kind of hard to believe nobody's tried making a "conveyor belt" like process inside a closed system (a shipping container?) with the right optics and resolution to pull it off. Fidelity plus speed plus software consistency. Considering what I saw the gaming industry doing with static models about 10 years ago I kind of thought it'd be a lot further along now, but I guess not. Sounds like a good project for a few Rensselaer Polytechnic Institute grads that otherwise would've been destined for Kodak.

I was musing another kind of 'real world capture' with videogames, because I want to race around my neighborhood in Forza. https://hackernoon.com/dashcam-google-maps-dev-kit-custom-ne...

Oh, so you mean like a big box that could fit XYZ items inside it and capture something like 10,000 per day?

Maybe but I actually think that's the wrong approach.

I mean, to me it's kind of hard to believe nobody's tried making a "conveyor belt" like process inside a closed system

Yea they have - kinda. None of it works well or fast enough though. We put up a patent for one a year ago before I thought there was a better way to do it. The manpower required to move items onto/off of a line is a big part of the problem.

Well yeah the human element is exactly what I'd want to eliminate as much as possible; I'm thinking of it more along the lines of what I've seen on How It's Made: Dream Cars in the sense that to get the various layers you're going to want - basic dimensions, surface features, coloration, reflective properties - aren't going to happen in one quick grab I don't think, and I get the feeling the process would work best in "absolute darkness" and isolated as much as possible for vibration.

Taking that 10k number - assuming disparate types of items that might be part of a series like "Bathroom" (toothbrush, hair brush, toilet brush, plunger) - in 24 hours that means cycling each item through in about 8 seconds. The only way I remotely see that possible is essentially having a robot hand pick up the item at the entry point, hold it for the capture sequence (perhaps have a custom-designed 'mount' that can allow for true 360 via a couple positions), and then drop it out the other side.

It's the scale part I'm wondering about, re: one size machine fits all doesn't seem to make sense. One machine for items under a certain dimension (e.g. "hand held") then another for items where the machine has to essentially have super-powers to pick up and rotate objects to complete the imaging process (e.g. a couch, a dresser, a motorcycle, etc). I think trying too hard to accommodate outliers ends up tainting the balance of operations a little? Just thinking out loud, really cool puzzle.

Do you know what has been tried before or references to other attempts?

Yes there have been a million attempts since the early days of 3D. Most of them are photogrammetry or structured light setups of some kind, that aren't fast enough and don't scale for sizing. Part of it is logistics of getting objects through a scanner, with the accuracy being poor or muddy at best.

IMO it should be done with a mixture image segmentation and procedural generation.

There isn't a combination of laser scanning and/or structured light projection scanning that can accomplish this? Or is it a speed/quality control issue of the output?

Triangulation laser scanning is about the closest you get in terms of accuracy. It can work on virtually any surface, including specularly reflective ones (I've worked on bespoke systems for the steel industry). It's accurate down to microns, but the usual problem is the field of view sucks - either you go further away and sacrifice resolution or you go really close and accept that you need to move the scanner (or object) around a lot. For small things, it's fine. You put your doodad on a turntable. For cars, forget it.

Stereo structured light is great, but doesn't work on specularly reflective objects. You've seen those amazing depth maps from the guys at Middlebury? Wonder how they get perfect ground truth on motorbike cowls that are essentially mirrors? Well they have to spray paint them grey so that you can see the light. The next problem is that you're limited by the resolution of the projector (so I guess if you own a cinema, yay!) and the cameras. Then you have to do all the inter-image code matching which sounds trivial in the papers, but in practice a lot harder (and since you don't get codes at all pixels you need to interpolate, etc, etc).

There are handheld scanners like the Creaform which work pretty well on small things, but I don't know what the accuracy is like.

The ultimate system would probably be a high-resolution, high-accuracy, scanned LIDAR system. Then you lose the problems with scanning ranges/depth of field, but you accept massively higher cost and possibly a much longer scan time for accurate systems.

Beyond just the 3D: Do you think that capturing the full appearance properties (BRDF etc.) of the object would be useful? This would allow users to very accurately render objects.

Definitely, though UV mapping from that data takes it to another level of difficulty.

Yeah think about it as icons but for 3d models.

That's been turned into an industry with very high throughput.

You mean better than those inferred from parallax/pictures?

And inside the object as well?

You mean better than those inferred from parallax/pictures?

I'm not sure what you're asking here. Are you asking if it should be better than what can be done with photogrammetry?

And inside the object as well?

Doing just the outside is a big enough market/problem.

Where exactly are the $BNs in this tech? Also, any datasets? I'd love to experiment.

See other responses for the markets it would serve. Use 3D modeling by hand pricing as a comp (Low end $10/model, average in the $50-100/model range, sky's the limit for super HQ stuff).

Not sure what kind of datasets you're looking for. You'll see actual products to test with.

That makes sense. I think there's opportunity for generative ML to eventually help here. An open dataset of (images, description) -> 3d model would go a long way. Check out this paper on using GANs to generate voxel-based models: http://3dgan.csail.mit.edu/

I've been studying and working with GANs for about a year now. They are still very exciting, and I'd love to try to expand my codebase to new types of data.

Additionally, there are some recent techniques that haven't been tried with voxel-based renderings.

Perhaps there is another algorithm that can help go from voxel -> polygons as well.

I think with the right tech, time, and execution this could be a matter of:

1. Take a picture

2. Generate until you get the 3d model you want

That's the exact approach that I think is going to work.

Well, not exact cause I don't like their voxel building generation method.

I think a GAN + Procedural Generator is the winner.

edit: Let me know if you want to work on this cause it's an active area of research for us. See my HN profile for contact.

Can you clarify what you mean by Procedural Generator? Isn't a generative model already a procedural generator? Its just that a model generated in case of the referred paper is voxel based. Did you mean, generate parameters of a pre-specified model e.g https://graphics.ethz.ch/~edibra/Publications/HS-Nets%20-%20... , although this paper is just learning a regression to the human body model (not using GANs).

Curious to know more about your train of thought. I am working as a researcher in the domain and thinking of experimenting with GANs for 3D model estimation using similar inputs as the one in the paper I referred to.

I think generative models could work but both 2d GAN and 3D voxel GAN are very low res

You may be interested in the COIL dataset, it contains images of objects from many angles.

http://www1.cs.columbia.edu/CAVE/software/softlib/coil-100.p...

theres a method for generating 3d point clouds from images using SIFT already

Reply on news.ycombinator.com