You've got it backwards. Supervising an ML model in a lab under perfect conditions is testing the times tables. Putting you in extremely variable situations where you have to solve different problems is learning calculus. No two driving trips are exactly the same, otherwise you could just record the steering and pedal inputs and be done with the whole thing. Autopilot has been in more situations and processed more roads than any other self driving vehicle (simply because Autopilot vehicles have been mass produced for years and the others aren't yet).

> the vast majority of data Tesla is receiving is reinforcement of what already works.

Absolutely! But is this not still valuable? This tells the model that, in times when it doesn't know what to do, it can pick the most reliable strategies that have worked 100% every other time. This applies geo-spatially too: if Autopilot drives too hard around a corner and you hit the brakes to disconnect, the model will remember this and send the incident to the cloud. As more Teslas continue to disconnect around this corner, the model will learn to adjust its speed for a smoother ride until nobody is disconnecting. At that point you, someone who has never driven a Tesla before, can go out and buy one and it will handle that same corner perfectly without you knowing that it was once a pain point. This is the benefit of collecting as much data as possible.