looked at using airsim a few months ago to replace a product we'd been using for some time, problem was the actual vehicle dynamics simulation capabilities were subpar
Airsim also has poor facilities for stepped simulation needed for many RL algorithms (e.g. insert control, advance simulator by 30ms, observe state, repeat). It looks like this is still being worked on: https://github.com/microsoft/AirSim/issues/600
Weird, I'm surprised that isn't the default mode of operations for something that's designed to plug in your own control system.
Did not do a code review but I would not be surprised if they were using Unreal's default vehicle simulation; which, as you noted, is indeed sub-par. It's honestly a very challenging task to get vehicle physics to accurately replicate real cars, and the systems which are out there (commonly used by manufacturers to make advertisement renderings) are proprietary and expensive.