I once messed up and input the RGB values in the wrong order. This lost me 2mIoU in evaluation. If anything it was a pain in the ass to figure out, because the error was so subtle that I thought maybe I was imagining it. So I don't believe that camera calibration is important anymore. What you need is to have a dataset taken by a variety of cameras.