I agree. If you're on Mac, the tool Seil will allow you to map Caps to Esc. And I use SharpKeys when on Windows.

The downside however is when you're stuck using someone else's computer or don't have root on Mac and Windows. I don't find myself on someone else's machine often though, so for me it's worth it to not have that half second delay.