IMHO you should go with Heroku and ignore anyone who's suggesting dedicated servers, VPSes, AWS, Docker or containers. Every moment you spend maintaining a server or doing devops is wasted because it's time not spent building or marketing your _app_.
New startups' biggest expense is founders' time. It seems a lot of people, even on HN, don't realise it because it's a hidden cost. But if you think that a decent developer is worth at least $50/hr and it might take you an 1hr to set up and 1hr/yr to manage a server (e.g. apply patches, update security) then you're better off not spending those 2hrs and paying an extra $100 hosting your app.
Worrying about the future infrastructure cost is also wrong-headed. It's a kind of premature optimisation. To get the real, expected future cost, the projected infrastructure cost has to be multiplied by the probability that you'll actually get big, which for start-ups, is very low.