I haven't hosed a windows session yet, and maybe just one game was crashy ever, while task switching for the last two decades. I ran Windows Server though typically.
I remember this not-so-fondly. This wouldn't be a problem for a game that is composited alongside the desktop in a "Windowed" mode, but for a full-screen application it is an "exclusive" application. The Source engine handles this context switch absolutely terribly. Task-switching out of Team Fortress 2 is a stuttery mess, where for around ten seconds it'll just loop the tiny audio buffer it last filled on top of a black screen before anything is usable again.