historically they have been in archived in "C:\Windows.old" and later are removed when a new update is installed or if you use disk cleanup->system files. there used to be logic to clean this up proactively if disk space was too low, but my hands on knowledge here is >4yrs old.

this is different than the side-by-side (WinSxS) support that allows parallel versions of the same dll to live on the system. that used to grow at huge rates when you had lots of system updates + lots of apps.