Comment by midjji | Hacker News Reader

Comment by midjji | Hacker News Reader

Parent Full thread

Opencv is fine for basic models like the most popular one. This one is also sufficient in most cases, I have used it for subpixel 400m stereo with a 40cm base, visual odometry, high end 3d reconstruction etc, it also has fully functional variants for wide and fisheye. You just have to select the right model, calibrate without observability issues, verify the calibration, understand the camera, etc. The better tools help a little bit, but they do not eliminate the need to understand what you are doing in the slightest. The better tools are easier to use, but generally not more accurate. The spline field variants are more accurate if you know what you are doing and the right conditions apply, but much less robust if you dont.

Opencv calibration is hard to get right if you dont know what you are doing, and at first glance it will look right despite being irrecoverably bad. I had to edit the docs and tutorials because the examples they had were of FAILED calibrations. Visualizing the resulting distortion field is critical to understand if the calibration succeeded, and that s what the better tools provide. That said, if you dont know you are looking at, they likely dont help, and the golden rule is if someone calibrated something and said it was easy, they didnt. If someone has a pdh in camera calibration said they calibratied them but that they havent used them for sfm but they plan to soon, odds are 20% its right.

Mrcal was written and verified for more difficult stereo tracking than that, approximately by a 10-15x factor (in, say, fisher information sense). It's a shame it doesn't have a better data set for verification but I expect there are reasons for that as well. Not to say that other tools are completely inappropriate for this domain, obviously, or denigrate anything or anyone.

Sounds like a paper, link?

For you to say that you would have needed the image res 2MPixel, but ignoring that, it should have been a paper I knew about if true. So I checked. mrcal is providing the same lens models as opencv, and it looks like its using opencv to do the estimate. For higher perf calibration checkout https://arxiv.org/abs/1912.02908

I thought the thread here was re: Interrogating the model and improving its calibration, as well as using different models. I know from working down the hall from Dima for a few years that mrcal has done both, and in domains beyond and more challenging than those described in this thread. So I mentioned it, that's all. Nothing to say beyond that, but I wish I had more to contribute.

That paper describes a rich splined model, very similar to the one used in mrcal. mrcal models projection (instead of unprojection like the paper does), which is better in a practical sense. Both work well to fit every lens. https://mrcal.secretsauce.net/splined-models.html

if you don't mind, I have two questions about your comment. Why is sfm relevant to assess the quality of your calibration? What does the distortion field tell you besides using the wrong distortion model or having an unbalanced data set? (e.g. not having enough samples at the borders of the image)

If you have say a 1Mpixel image which has been rectified, then a human will generally say that it looks correct if the calibration errors are less than 10 pixels. Perhaps as little as 5 if they know what they should look for and are careful. Most camera calibrations are this bad and no one notices. If showing a person the image rectified is the goal thats good enough. But, if you want to do post processing of any kind...

Sfm tests the calibration to the extreme and even single pixel calibration error will be highlighted as correlated reprojection error vector in most images. And that is assuming the bad calibration does not cause the system to fail outright. There is also the inbetween case where the system is only able to use small parts of the image.

The distortion field often visibly tells you if the estimation failed. It should be smooth, and monotonic, and you can draw it not just for where there are pixels but further and in higher resolution, and for regular lenses almost always highly symmetric. Looking at the distortion field you can see alot of problems that you could not otherwise. The most common problem is the monotonicity, since that constraint is very difficult to add to general optimizers. Since this means the distortion goes backwards, they are visible as sharp edges in the distortion field.

Reply on news.ycombinator.com