They don't need to be unmapped just for the other one to use it. source: I wrote GPU drivers for over 10 years.