I used to say "but we could just build that", but I agree with you.
Some of the things I find expensive are te SSL line items and more peculiar than expensive really, a $25 image carousel.
If letsencrypt could be a drop in replacement for te SSL spend it would save them some money.
Other than that, I don't use Heroku, but not wanted to manage a db is pretty reasonable and the cost offset is likely there.
Sometimes these startups are 2 people. So the tech cycle would be hard enough, not to mention business and customer support.