You have to distinguish between fatal crashes and all crashes. Here [1] are the data for the US if you can tolerate that site. What is the typical condition for a fatal crash? It's extremely counter intuitive. It is driving straight, on a major roadway, in good weather, during the day time, while sober, in a 4 door hardtop sedan. Seriously.

I'm reluctant to go social science and start handwaving too much causality there, but on the other hand I don't think it's entirely unreasonable to observe that those 'easy' driving conditions also tend to be some of the highest speed driving conditions. Creates a nasty combination of of a situation where we might be more subject to lapses of attention, due to comfort, and exponentially increased consequences when that lapse results in a crash.

But the really cool thing here is that these are also the conditions that right now the self driving systems are rocking. In other words there's actually a chance you're right that we're not comparing apples to apples, but instead are actually oversampling conditions where fatal accidents are more likely to happen!

[1] - https://www-fars.nhtsa.dot.gov/Main/index.aspx