I don't have doubts about Django scaling.
The caching framework is really cool. I did a test a couple of weeks ago, and I got a 10x performance boost, simply by sprinkling 3-4 lines of caching code to my site. That was justing the local memory caching: http://docs.djangoproject.com/en/1.0/topics/cache/#local-mem...
I'd imagine that using a caching back end like memcache would probably give another huge performance boost. It's really pretty straight-forward and usable out of the box.