- Scalability Rules: 50 Principles for Scaling Web Sites by Martin L. Abbott & Michael T. Fisher
- Building Scalable Web Sites: Building, Scaling, and Optimizing the Next Generation of Web Applications [Paperback] by Cal Henderson
I think they are the best resources I have found about the topic. If you are using .Net, like me this is book has good tips as well:
- Ultra-Fast ASP.NET: Build Ultra-Fast and Ultra-Scalable web sites using ASP.NET and SQL Server by Richard Kiessig
To me scalability is all about architecture, and the #1 item that you can do to architect a scalable system is statelessness.
You can also run a ton of other languages on heroku now too.
Many articles focus on bigger, high-profile companies with unique challenges but there are tons of great articles and tips on scaling normal web apps. Check the archives.