Airsim also has poor facilities for stepped simulation needed for many RL algorithms (e.g. insert control, advance simulator by 30ms, observe state, repeat). It looks like this is still being worked on: https://github.com/microsoft/AirSim/issues/600