Your system would grind to a halt anyway; swap is simply giving the kernel more options to deal with the issue. Without swap the kernel has no options when it cmes to unreclaimable pages.

Obligatory link: https://chrisdown.name/2018/01/02/in-defence-of-swap.html