You would need a sensor to measure relative wind. Angle of attack is the angle between the wing and the relative wind.
Obviously this system will drift over time, but it could more than adequate to detect a failed AoA sensor and revert to the second one.
Your suggestion would work under the assumption that there is never any vertical motion of air, but that's not the case in real life. E.g. in a downdraft your sensor would fail to detect a stall.
What are you basing this assumption on?
I also assume the AoA sensor failure was evident over time, through several phases of the flight and not momentarily. A sensor fusion algorithm would notice the divergence over time between the IMU and the two AoA sensors, then select the one AoA sensor that was closer.
Even in the case of a momentary failure, it would take an extreme coincidence to have the readout pattern of the failed sensor seem closer to the IMU based AoA model than the working sensor.
Keep in mind I’m only talking about cross checking the AoA sensors here; the flight control algorithms are only considering the aerodynamic sensors for AoA calculation as before, but they have an independent external reference to gauge the data quality of each sensor.
What would be wrong with using an accelerometer to measure/calculate velocity with respect to pitch give you a better reading of AoA? In conjunction with a second velocity source to provide redundancy?