Your typical blog is probably going to suffer even more from enabling https globally when compared to major sites because not as much effort has been put into combining js, css, and images into sprites.
Seriously, https performance sucks from start to finish. If the average user doesn't care about https, but does care about performance, who are they going to go with - you or your faster competitor?
I agree that anonymous or non-logged-in activity has no need of HTTPS, but anything that's transferring cookies, passwords, or other important user or session information should be HTTPS.
You can't cache something that's encrypted!
Edit: It looks like the best way might be to do SSL between the client and nginx on my side acting as a reverse proxy, and then non-SSL internally on my side? Not sure how that setup compares to Varnish in general, but it's probably fine for my purposes.
One special case though, if you have multiple servers that serve your content load balanced, and if these servers are in different colocations, then you probably need to run any sync between them over SSL. Even if you do control the link between the two boxes, there's that off chance that your link goes down and the IP layer automatically routes traffic through a different set of routers.
Which you can't really do in shared-IP virtual hosting, since SNI support is still a bit spotty.
Can't the argument be made that if you are a startup and have launched a product to test the market, it would take up too much developer time to think about https? Depending on the nature of the service, shouldn't you defer the extra effort until only after you've validated the product/market fit?
I actually did it last night, from yum install httpd to (self-signed) SSL in less than 15 minutes. There are some webapp considerations, but they are negligible when compared to other security efforts like XSS diligence.
OTOH, I am writing this comment on an open wireless router.
On the gripping hand, nothing I put here is private, and if someone "pranks" me, I can just login again and delete offensive content. Karma isn't actually money...
This might not bother you individually, today. But maybe it will cause problems for you in the future if laws change? Maybe it is causing problems for a lot of people who aren't you today? Maybe it is causing problems for citizens in countries other than yours?
The World would be better off if https was used everywhere.