That's just not true. If inactive memory is something that's already backed by disk (like a memory mapped file), it'll be discarded. Only if it's read-write memory that's /currently held allocated by a running process/ will it be swapped out to disk. Unless you're doing something pathological (like suddenly allocating lots of RAM and forcing paging - what purge utilities do...), the architecture /speeds things up/.

If it's really true that, without doing anything special, things were always being swapped out to disk for you, it necessarily means that there was a process that had allocated (lots, it sounds like) of RAM and written to it, so that stuff had to be paged out to disk to free up RAM without losing data.

It sounds like you were running purge commands or utilities to 'free up RAM'. That is counterproductive. It causes the system to release cached 'clean' (i.e. as already on disk - a mapped file, essentially) mapped RAM and swap read-write memory out to disk, only then have to re-read it all when you actually need it. In other words, using purge 'utilities' actually puts the system into the worst possible state.