Think of servers with ARM processors in the pipeline which will come with 64+ cores, available for cellphones. The biggest problem in a datacenter is not space or processing power, it's energy consumption and heat dissipation. Walking into a datacenter gives you the feeling that the place looks empty with plenty space to fit 6x more servers. Today this can't be done because there's no capacity for more air conditioning to cool more servers in the building.
Also, the way of processing has changed in the last years, for example with map reduce, which makes having many cores way more useful than a single server with a massive 5 ghz core. Actually today, many servers are IO bound, not CPU bound. There's exceeding cpu capacity.
Think of having a server, with 64 ARM cores and and array of SSD's. This won't heat up as much as mechanical disks or today's cpus, with very small IO constraints due to SSDs speeds and far more parallel processing power.