yep, we were using Amazon ECS-Optimized Amazon Linux AMI, docker and agent should've matched there.
It was strange behavior, either all agents were connected or most of them were disconnected at the same time...
I'm wondering if Elastic Beanstalk is hiding this from us in some way. Or maybe just has a workaround built in?