There is OpenAI gym interface for Webots: https://github.com/aidudezzz/deepbots

The interface simplifies usage of Reinforcement Learning algorithms with Webots.