They have dedicated processors for offloading IO operations, which has a big impact on reducing pipeline stalls, and also part of why they are so much better at handling a multitude of VMs.
They have 4 levels of cache, with gigabytes of memory at level 4, data is kept close to the processors.
The hardware costs a lot, but if your software is licensed per core, there is a good chance you could save millions compared to an Intel solution. It depends on your usage scenario though. If everything is open source and you aren't paying per core fees, then there is probably no advantage for you.