The issue is actually being worked on. For example, the draft RFC for fallible collection allocations is now in final comment period [0].
Edit: fix link
Actually most large allocations in Firefox are fallible and the browser will only crash when it can't deal with an allocation in security-sensitive code. So that's a terrible example!
The proper solution in Rust is to use the Result type.
As for what Linux does: it will invoke the "OOM Killer" which will kill user processes to clean up space.
Which is also pretty catastrophic. Honestly, I'm not sure which I would prefer more: a debuggable kernel panic that would be easy enough to recover from, or a still-running but possibly lobotomized machine.
You can configure the OOM killer to simply panic on OOM. You can also control which processes are prioritized for OOM killing, as the sibling comment mentions.
Not mentioned in the article is that you can set per-process memory usage limits, where subsequent calls to malloc() will return NULL if trying to allocate more. Some applications will try to properly handle that condition, many will just segfault or do some kind of controlled abort. For a lot of applications failing is the right behavior on memory allocation failure. I mean, presumably you weren't trying to allocate that memory on a whim, you need that memory to function properly! So there really isn't any sane way to continue functioning without either disabling functionality or hanging until memory becomes available. Either way, I generally prefer something to unambiguously fail so I can restart it than have a "gray failure" where the system tries to keep limping along.