Swap is indeed supposed to prevent this AFAIK. You can though try some tools like EarlyOOM and see if it helps : https://github.com/rfjakob/earlyoom