Opencv calibration is hard to get right if you dont know what you are doing, and at first glance it will look right despite being irrecoverably bad. I had to edit the docs and tutorials because the examples they had were of FAILED calibrations. Visualizing the resulting distortion field is critical to understand if the calibration succeeded, and that s what the better tools provide. That said, if you dont know you are looking at, they likely dont help, and the golden rule is if someone calibrated something and said it was easy, they didnt. If someone has a pdh in camera calibration said they calibratied them but that they havent used them for sfm but they plan to soon, odds are 20% its right.
Sfm tests the calibration to the extreme and even single pixel calibration error will be highlighted as correlated reprojection error vector in most images. And that is assuming the bad calibration does not cause the system to fail outright. There is also the inbetween case where the system is only able to use small parts of the image.
The distortion field often visibly tells you if the estimation failed. It should be smooth, and monotonic, and you can draw it not just for where there are pixels but further and in higher resolution, and for regular lenses almost always highly symmetric. Looking at the distortion field you can see alot of problems that you could not otherwise. The most common problem is the monotonicity, since that constraint is very difficult to add to general optimizers. Since this means the distortion goes backwards, they are visible as sharp edges in the distortion field.
There are applications that require pulling almost magical levels of signal from very small numbers of pixels, and for those extremely accurate results you really need to drill down on the camera model in particular. Even modelling the environment or atmosphere to get accurate results.
For those types of applications opencv is absolutely not the right choice. It just so happens that dima has an accessible software kit that is more for the latter case. But all the better because it's perfectly fine to use this tool to calibrate any camera and then go use it with any other CV pipeline. His pain your gain.
Such algorithms always create a graph over the images, but if you mean the graphslam graph filter methods, those are substantially subpar compared to classic feature based methods as well as the more modern, dense and semidense methods.
For localization work (indoor or outdoor), in which you mostly want to close loops and track 3D pose of features using measurements from, say 100s of m or less, (or perhaps scene recognition using effectively flat "very far" images), calibration using opencv is probably highly performant. It's standard, and I've certainly seen much success using it before feeding into regular gtsam etc. There are some unspoken assumptions about localization that don't translate to, say, tracking necessarily. Generally they are: Many observations, close-ish range (relative to stereo offset), mostly-correctly-pointed camera rigs (e.g., forward on a car), or perhaps assumptions about density of features, correlation across images, existence of dense prior maps, etc.
I believe the use case for something like mrcal is to improve calibration for cameras and applications that don't fit this model well. In particular, you may need to track a target at extreme range, using a pixel or two in the corner of your image, with a particularly wide FOV. These specific use cases, mentioned in top level comment, do require additional care, especially in calibration. Thus mrcal.
It just so happens that out of a domain where the very particular details of calibration matter, you may find a tool that helps with all calibration and I think that's the point in bringing up mrcal on a thread discussing calibration in general.
I think that's as precise and non-controversial as I can be.
Uncertainty propagation is very difficult to use for vision, and largely just modelling errors in vision as the error distributions, e.g. for anything observed or reconstructed from images are either subpixel accurate, or too non linear.
Another problem I run into is that the transforms are ill-defined just outside of the screen. That means that if I want to draw e.g. a line in world coordinates onto the image from a camera, then I often get garbage if the line starts or ends outside the image (even if I divide the line into many segments).
Can you be more specific on the second point? Once you have the intrinsics, it's trivial to project them to an (undistorted) image.
Yeah thats a common problem if the calibration failed. It could also be that you are not cropping to what is in front of the camera, but if its really weird, its most likely the former.
So the default, and probably most of the camera models in opencv requires a monotonic change lenses and bijective imaging. The former is common unless the lense has defects on the surface, and the latter is practically a physical constraint. The problem is that these constraints are difficult to add to the estimator, so they didnt. Meaning it will find a solution where they are not satisfied. If say the bijectiveness is not satisfied a bit outside the image, but stil valid accounting for infront and float accuracy, then that would absolutely account for the problem you describe. Is pretty obvious if you consider the function what the problem is, just hard to add the constraint in opencvs estimator.
The solution is 1, verify the result is satisfied after estimation, 2 make sure you make the parameters are as observable as possible during calibration. This means spread out in the image, evenly distributed, and all the way to the edges. Also make verify it has not rotated one or more of the detected chessboards upside down, or 90 deg sideways. Finally, because it becomes harder and harder to avoid this problem with more parameters, always start with 1, then try 2, then the two variations of 3, and so on. More parameters always fit better, so use an appropriate test.