They had been doing that, and called it "shadow mode" [1]. I suspect it's no longer being done, perhaps they reached the limit of what they can learn from that sort of training.

[1] https://www.theverge.com/2016/10/19/13341194/tesla-autopilot...