Shouldn't a real OS and filesystem be capable of removing something in use?
IRIX always could. So can Linux a whole lot of the time, if not all the time, as can BSD and Mac OS.
Here's something:
Once I upgraded a pile of IRIX boxes to a new version of some CAD software. A few of them had the older version running...
Didn't matter!
The files got cached, and those versions continued running the next day, until shut down. Next launch, the new version popped up.
One could just run a program, then delete the program right out from under the running process, and that process didn't know. IRIX just cached it all, and when the process was done, cleaned up the cache.