I am not convinced you understand this problem space well. The very notion of defining two such states is extremely elusive.
Might want to look into Toyota Guardian before dismissing the approach.
As it's exactly what they are planning to roll out in the short to medium term.
what's elusive about it? there's some pretty obvious situations where you expect driverless cars to have trouble: bad weather, unusual traffic or pedestrian patterns, emergency vehicles, rarely travelled roadways...
Outside of a few limited areas, remote control driving isn't a viable option due to the unreliability of cellular data networks.
making a safety-grade data network would be an interesting challenge. I do think it is a possible thing to do over limited areas, and I think that if we had defined areas that had safety-rated networks, that's a thing that self-driving cars could plan around... e.g. if you request a trip outside of those areas, the self driving car would take you as far as it could and then park, allowing you to prepare to drive manually.