Shouldn't a real OS and filesystem be capable of removing something in use?
IRIX always could. So can Linux a whole lot of the time, if not all the time, as can BSD and Mac OS.
Here's something:
Once I upgraded a pile of IRIX boxes to a new version of some CAD software. A few of them had the older version running...
Didn't matter!
The files got cached, and those versions continued running the next day, until shut down. Next launch, the new version popped up.
One could just run a program, then delete the program right out from under the running process, and that process didn't know. IRIX just cached it all, and when the process was done, cleaned up the cache.
It's a choice. Windows specifically uses DLLs as non-dirty swap, i.e. if the memory manager want's to free up pages it may free memory pages used for mapping a least recently used DLL because it will not require it to write any "dirty" memory. A DLL page can never be dirty. Windows knows that it really easy to read in the image again (from the DLL) - so it can just kill the page and reuse it immediately. But then it has to know that it can get the exact same memory page back from the DLL. If the DLL has been deleted or changed that would lead to some strange errors.
When you install(1) a new version of a file, the old one is deleted and the new one written in its place. The file handle in existing processes continues to point to the original, and the filesystem will wait until everything's closed it before actually discarding the file data.