Using Heroku puts two levels of bloat between you and the hardware, and locks you into a hosting environment that becomes very expensive quickly. Eschew.
"Bloat" here is a virtue. Heroku removes two levels of drudgery and administration between you and the hardware. Who wants to be racking hardware? Who wants to be patching the OS? Who wants to be dealing with package vulnerabilities? None of this helps actually build the business
Lock-in is minimal, particularly if it's a self contained app and you're not using a lot of services. There are countless examples of companies who've moved off of Heroku when they've gotten big. Plus there's also lock-in using EC2, S3, AWS. There's lock-in of some level regardless where you run your stuff.
Future cost is also really, very very low because P(success) is, sadly, super low.
If you're working in a niche market, costs are important and keeping them low can be the difference between having an independent livelihood and failure.
If you're building in a niche then (almost by definition) you're not going to need a lot of infrastructure. There won't be millions of customers and they (usually) won't need tons of infrastructure.
I think the only case is where you're building a business like Pinboard :) Where the CLTV is relatively low but the the bandwidth, storage & compute is relatively high.
Regardless of the market you're in though, if you want to build an independent livelihood or a lifestyle business typically infrastructure costs still shouldn't be a factor.
A good livelihood might be $50k/yr say or $4k/mo roughly. The difference between $5/mo and $500/mo in hosting costs isn't what determines success or failure. It's whether or not you've built, marketed and sold an app to 5 or 500 customers.
A niche market cuts both ways. Sometimes you don't need to worry about scale, but you also can't bet on large numbers of customers coming to cover up inefficient choices.
Choose OpenBSD (and don't install stupid stuff like bash) and you won't have to deal with any of that.
Besides, not every app needs big computing power, and many businesses can thrive on Heroku for a long time. An employee of ProductHunt mentioned on a podcast that they ran their website on 3 Performance L dynos at in 2015. That's $1500 plus some extra for the database and other services they may be using (say $3k in total). That's one of the popular websites out there, and that's a pretty good indication of an upper bound of the infrastructure cost on Heroku for a popular product. Most will never exceed the $200-300 a month.
Is $3k a big sum? Yes, it's noticeable. Will you end up spending more on hosting plus the ops work that needs to be done over time for a similar app to run and grow? Likely.
EDIT: spelling