Colorize Lidar point clouds with camera images | Hacker News Reader

Colorize Lidar point clouds with camera images

medium.com

I did work on this as part of my thesis quite a few years back at the university. One other optimization would be to process the points in parallel.

Regarding the coloring of each 3d point, it might be feasible to not use one camera image, but a weighted sum of all camera images that can see the same point in the scene. Each pixel color is then weighted with the scalar product of the points normal and the viewing direction of the camera. This would also regard for noise and specular reflections (which can mess up the original color).

Yes, I am working on using numpy to do the projection using matrices so we dont have to loop over each point and project it individually. That should be a big boost.

The way I handle the different camera images is to simply see which one provides a lower depth and use - with the idea that if the camera is closer, it would provide better information. But what you are suggesting is pretty interestint. I'm going to try that as well.

Lidars are pretty powerful, but one big disadvantage of using point clouds for perception is that they are not colored. This makes identifying objects more difficult compared to camera images. However, by combining camera images with lidar data, we can enhance the point cloud by assigning colors to the points based on the corresponding camera image pixels. This makes visualizing and processing the point cloud much easier.

> one big disadvantage of using point clouds for perception is that they are not colored

That depends entirely on the capture device.

True. But my reference point is Lidars which mostly produce non-colored points.

I'm not sure why you've just restated the first paragraph of the article.

Likely for the engagement... any bets as to if they are a bot or not?

Their (human) reply makes me suspect it's a human using an llm without realizing how obvious it is that they're using one.

Edit: Just realized they're probably the author too of the post

Makes sense given English is likely not a strong primary language for them.

LLM's are getting heavy use from the ESL (and even English as a third or fourth language) crowd.

Shameless plug, but I work on a multispectral lidar that does produce “colored” point clouds in the SWIR [0].

It is pretty cool, we use it for detection of humidity degree or for species discrimination (e.g. plants, minerals, chemicals…).

[0]: https://www.iridesense.tech/

Lidars are expensive, if you want spare point clouds, that are not quite real time you might want to check out colmap https://colmap.github.io/

Thanks, been trying to look into AI tools to generate point clouds from photos for a hobby robot. Crazy that a mediocre LIDAR costs more than every other part of the robot combined, maybe times 10.