565 karma · joined March 11, 2017
Or in other words, you can approximate exp(x) as a set of first order taylor approximations that each covers a small window to arbitrary precision, but the combination of them is still has well defined higher derivatives that are not 0.
Many of the libraries are not well maintained. I needed a basic homography estimation recently and asked a minion to try opencv for it before anything more advanced. He got it working, but the best inlier ratio for every feature he tried with default parameters was 3%. The images were offset by 30 pixels left right, and less than a pixel in warp and taken with different exposure time. So he used sift...
He argued he got it working and that was as good as keypoint matching got... If I hadnt happened to be the guy who needed it one step up, it could have just propagated, someone adding a shitty ekf to make it smooth, then buried in layers of heuristics and api.
Sfm tests the calibration to the extreme and even single pixel calibration error will be highlighted as correlated reprojection error vector in most images. And that is assuming the bad calibration does not cause the system to fail outright. There is also the inbetween case where the system is only able to use small parts of the image.
The distortion field often visibly tells you if the estimation failed. It should be smooth, and monotonic, and you can draw it not just for where there are pixels but further and in higher resolution, and for regular lenses almost always highly symmetric. Looking at the distortion field you can see alot of problems that you could not otherwise. The most common problem is the monotonicity, since that constraint is very difficult to add to general optimizers. Since this means the distortion goes backwards, they are visible as sharp edges in the distortion field.
In general a good starting point for such a problem would be to implement https://grail.cs.washington.edu/rome/ on your own.
Its not particularily hard, and there is no difference between reconstructing from uncalibrated cameras and reconstructing from many images from one camera with unknown calibration. Then you can check if the calibration is unchanged over time, and if it is, add that as a constraint. So far it will still take half an hour to run, but now you can add in tracking exploiting motion prediction and imu, which in turn will give you the real time speed.
The lense model is the parametrized approximation of the lense function. Picking the right one matters for ideal accuracy.
The estimator takes observations which constrain the parameters of the lens function, and find the parameters.
The framework helps you create those observations.
Such algorithms always create a graph over the images, but if you mean the graphslam graph filter methods, those are substantially subpar compared to classic feature based methods as well as the more modern, dense and semidense methods.
Yeah thats a common problem if the calibration failed. It could also be that you are not cropping to what is in front of the camera, but if its really weird, its most likely the former.
So the default, and probably most of the camera models in opencv requires a monotonic change lenses and bijective imaging. The former is common unless the lense has defects on the surface, and the latter is practically a physical constraint. The problem is that these constraints are difficult to add to the estimator, so they didnt. Meaning it will find a solution where they are not satisfied. If say the bijectiveness is not satisfied a bit outside the image, but stil valid accounting for infront and float accuracy, then that would absolutely account for the problem you describe. Is pretty obvious if you consider the function what the problem is, just hard to add the constraint in opencvs estimator.
The solution is 1, verify the result is satisfied after estimation, 2 make sure you make the parameters are as observable as possible during calibration. This means spread out in the image, evenly distributed, and all the way to the edges. Also make verify it has not rotated one or more of the detected chessboards upside down, or 90 deg sideways. Finally, because it becomes harder and harder to avoid this problem with more parameters, always start with 1, then try 2, then the two variations of 3, and so on. More parameters always fit better, so use an appropriate test.