> Do you really know this? What's the limiting factor?
The big issue is I/O bandwidth. All of the meaningful I/O in a server setup goes through one USB 2.1 hub. The second issue is that all of the memory accesses have to traverse the (useless) GPU to get to the CPU.
As designed for a phone, having the CPU be a ride along to the GPU makes perfect sense. The "thing" that is important in a smartphone is the screen, its updates and I/O to and from it. But as a general purpose server that isn't balanced.
We put RasPi servers on the Internet and local network at Blekko for a variety of tasks, and I've used them at home as well for servers. Very successful having them be single purpose servers (a DNS cache, a video streamer, a data collector) and very unsuccessful when they are multitasking multiple 'kinds' of network clients. Tried various configs with EXIM and Dovecot as a mail server, it processes very slowly. (an SSH session on the machine was 10s of seconds between the key press and the appearance of the character on my end, took me back to the worst of times on a timeshared system :-).
But seriously, everyone should try this. Set up a minecraft server, set up a web server, set up what ever you want and watch it. It is easy to set up multiple "stressor" clients with Python. Trace the flow of data from client to network to app to disk to app to network to client etc.
Then do the same thing on a machine with the same numbers and twice the I/O bandwidth (Odroid has a couple of boards with similar specs but better I/O bandwidth as do a bunch of boards in the 96boards.org spec)
It will be time well spent if you're ever asked to get the most out of a system.