Sometimes it's about the pennies. But other times, especially early, it won't be these types chunks of money that kill you. I used to go on CL and do user-studies for $50 a pop and resell the gift cards for cash or amazon credit. So it's hard for me to believe that what you are building can't spare an hour of work to fund baseline communication systems that are pretty well supported.
That is why heroku exists right? Otherwise only linode or ec2 could make business. Now if you have free time (you shouldn't) then, it is a whole different story. I agree with you.
To put it in perspective, Netflix costs ~$108 a year. People have no issues spending $4 a day on a latte, but just 13 lattes costs more than a year of Google Apps. A single latte or a big mac meal at mcdonalds costs more than an entire month of google apps for you.
And then there's coffee, which exceeds them all :)
For that matter, so is beer!
Has anyone found a good solution for this? I could host it myself but that is a lot of hassle, free email accounts have ads and rarely let you use your own domain. The best I've found so far is email through a shared hosting provider, but it has limitations of its own (no IPv6, no 2 factor auth, self signed TLS certificates, etc). The cost is similar to a Google Apps account but without the linear price increase per mailbox.
In the case of email, it's obviously valuable. While there are other options out there, ultimately you have to determine the cost of researching, testing, setting up, and switching over you and your employees. All this time is spent away from your money making activities.