Which is more repeatable but less representative. IMO a non-shared EC2 instance is probably the way to benchmark (if you can drive enough load to saturate it).
(I know it's then close to 'dedicated' hosting, but people running benchmarks also want the quick setup and discard of virtualized instances. This would be a hybrid offering that ensures their cloud services is always chosen for such benchmarking comparisons. Of course this offering would not be good for cross-cloud comparisons, because it's not representative of their usual offerings.)
That's called EC2 cluster compute.
- Using VPC you can specify that your instance should run on dedicated hardware [1]
- Cluster Compute instances [2] most likely (see [3]) run on dedicated hardware too
[1] http://aws.amazon.com/dedicated-instances/
[2] http://aws.amazon.com/ec2/#instance
[3] https://forums.aws.amazon.com/message.jspa?messageID=238197#...