If you are spending less than $25k/mo on Heroku or equivalent, you're not ready to move off it yet.
A lot of times when I talk to people with a high Heroku bill looking to move to bare metal, I end up being able to optimize it by 1/3rd or more just by picking dyno types, scaling appropriately, and consolidating workers. You can't really do that when you're hiring headcount.