We left Heroku last spring with a project consuming around 5k$/mo worth of Heroku services.

Main reason for the move was pretty simple, we just needed more control over our own infrastructure and Heroku wasn't able to offer that. Things like HTTP2 support, access to load balancer configurations, timeout settings, different auto-scaling options and better monitoring are first things that come to mind.

We are now using AWS directly (EKS, RDS, Lambda etc) and even though the move itself did cost a bit, I wouldn't say monthly costs went up too much (but it's bit hard to compare as we're using more services at AWS and scaled up right after migration).

Basically, we just grew out of Heroku.

And personally I wouldn't choose them again even if opportunity appeared.