Windows (at least NTFS) does allow you to delete files in use as long as it has been opened with a sharing mode of FILE_SHARE_DELETE. Unfortunately most regular programs don't do this.

An exception to this is memory mapped files as they are not allowed to be deleted that way. Executables are executed in Windows by memory-mapping them; so runnning programs are hard to delete.