Won't Docker be able to take this type of efficiency to a whole new level?
Docker --> hardware
Orchestration software --> Docker/LXC/VM/etc --> hardware
An additional example would be dotCloud (makers of Docker), they have orchestration software sitting atop Docker, which knows about users, machines, etc and then provision these docker instances on AWS hardware. dotCloud (orchestration software) --> Docker --> AWS EC2 (hardware)
There is a great wired article (linked to below [1] and in the OPs story), which outline how Google uses this orchestration software in its day-to-day operations. There is this great diagram [2], which shows Omega (Google's orchestration software), and how it deploys containers for images, search, gmail, etc onto the same physical hardware. There is an amazing talk by John Wilkes (Google Cluster Management, Mountain View) about Omega at Google Faculty Summit 2011 [3], I would highly recommend watching it!ps. Orchestration software has a global view of resources across all machines, so it knows how to get the best utilization out of all these machine (think jigsaw puzzle). You submit a container profile to the orchestration software, things like instance lifetime, cpu, storage, memory, redudance, and the software will figure out where to place your instances.
[1] http://www.wired.com/wiredenterprise/2013/03/google-borg-twi...
[2] http://www.wired.com/wiredenterprise/wp-content/uploads/2013...
P.S. - Not to Justin but others. Funny, you ask a honest question on here and people downvote you...sigh.