@peerherholz @neuroecology Google sent me this https://github.com/clvrai/awesome-rl-envs Regarding a tool that would track popularity I don't know but I'd be interested in a resource if it exists. In my lab we have used mujoco and gym extensively, and I would guess that they are some of the most popular frameworks, but I really don't follow RL research enough to tell for sure.