Intel has a CRC instruction built in to the x86 instruction set on newer CPUs. It turns out that using this instruction is a substantial performance win.
The problem with the "sufficiently smart compiler" argument, as always, is that the compiler may have a lot of optimizations, but it's still not an artificial intelligence. It can't tell that what you are trying to do with your set of operations is actually perform CRC, and there is hardware support for that.
Check it out at:
http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-comm...
We have written custom memory allocators, our own network stack and protocols, measured and optimized every operation in the years before the accelerator went live.
I would actually argue that you don't need to do these things in most cases. SCTP and DCCP are alternatives to TCP/IP that have been in the Linux kernel for a while now. If you want to trash TCP/IP and go full custom, you can do so without writing a line of protocol code. However, a better approach is to use TCP/IP with tweaks like Fast Open (what Google uses internally.) You can then continue to buy commodity hardware.
Similarly, memory allocators have been done to death. Just use tcmalloc or jemalloc rather than rolling yet another malloc(). The exception is if you want to create something like a slab allocator where you just hand out lots of mostly identically-sized buffers, or a database-like application where you manage your own writeback to disk.