I'd say it's more like benchmarks don't tell the whole story. They should still be good for catching obvious regressions before any code hits a real user's machine, but having real world data should help to identify a whole bunch of bottlenecks that were unknown before, and determine if performance tweaks are actually helping.
It's a shame browsers focus so much on JavaScript performance. I tried FF4 a few weeks ago and the user interface itself just feels slow whereas Chrome's interface is much snappier. It'd be great if all of the browser vendors realised that JavaScript doesn't tell the whole story!