I'm currently working on an RL project based on an OpenAI Gym environment and have been reviewing the different frameworks available. So far I’ve come across:

- OpenAI Baselines (more a collection of algorithms than a framework)

- Keras-RL (looked ideal but has been abandoned)

- Tensorflow Agents (An 'official'? Tensorflow library, but very basic- only one algo at present)

- rllab (Developed by OpenAI people but seems to be abandoned)

- OpenAI Lab (?)

- TensorForce

My main concerns are: 1. Soundness of the algo implementations. 2. Modularity, ease-of-use, compatibility.

I first looked at Baselines as it seemed to best address the first concern but ran into frustrations when for example the DeepQ implementation didn’t work if my Gym’s action_space was a Tuple space. I am working with a team unfamiliar with RL so want something that is as plug-n-play as possible, like Keras. So far TensorForce looks promising. Can anyone add anything more? Thanks