That's not entirely correct. The MMU generally handles virtual to physical memory address translation and the OS is only ever involved if there is a page fault. Outside of OS architecture and very specific and intended application, virtual/physical memory is completely transparent. When I hear "virtual memory" I assume reference to swap space unless otherwise noted because the technical meaning has such a specific domain.