In strictly rational terms It has to be better than 1 fatality for every 100,000,000 miles driven, but public opinion isn't exactly rational so it probably needs to be even better than that.
It'll be that way at first, yeah, but once it becomes obvious how much better the AI drivers are, it will start to become irresponsible to not use it