I switched from emulator tools like valgrind to tools with hardware support like perf, pmu-tools, and VTune. I generally found them sufficient, but sometimes buggy and difficult to use.

Cachegrind is occasionally inaccurate due to an inaccurate model, but the greater problem was that cache hit percentages only tell a fraction of the story. To be able to predict performance I often needed to be able to accurately measure things like the number of memory requests in flight.

Searching now for an example, I hit on a comment I made here a few years ago where this new tool probably would have been helpful: https://news.ycombinator.com/item?id=18442131

In general I have much greater faith in the on chip performance registers. That said, other than glancing at news stories like this I haven't been keeping up with recent advances. I guess it's possible that cachegrind and friends have improved since I was using them.