Comment by warrenm | Hacker News Reader

Comment by warrenm | Hacker News Reader

Parent Full thread

You should just about always add at least some swap.

Anyone who thinks otherwise hasn't read much or seen very many systems.

start here:

- https://www.kernel.org/doc/gorman/html/understand/understand...

- https://www.kernel.org/doc/gorman/html/understand/understand...

- https://www.linuxquestions.org/questions/linux-kernel-70/why...

- https://kerneltalks.com/disk-management/swap-addition-in-lin...

- https://linuxaria.com/howto/linux-memory-management

- https://kernelnewbies.org/Linux_4.8#head-a0b674ff33af7e3e7b8...

- https://kernelnewbies.org/Linux_4.11#head-680ed3eee137456d6a...

- https://kernelnewbies.org/Linux_4.14#head-5327850512ba28763d...

- https://help.ubuntu.com/community/SwapFaq#How_much_swap_do_I...

- https://access.redhat.com/solutions/15244

- https://serverfault.com/a/684800/2321

- https://serverfault.com/q/329928/2321

- https://serverfault.com/q/25653/2321

- https://serverfault.com/a/825915/2321

- https://askubuntu.com/q/184217/3544

- https://antipaucity.com/2011/08/08/why-technical-intricacies...

I've been in environments ranging from a few dozen individual servers to 100s of 1000s. In companies ranging from mom-and-pop shops to international investment banks. Every last one is configured with swap. Because unless you have done all the profiling to truly know you don't need it, you should be running it.

The majority of your links seem to be chosen simply because they have the word 'swap' on it somewhere, not because they make the case why having 'some swap' is a must.

I wrote a kernel from scratch, besides that I did quite a bit of transaction oriented stuff and built the server side of a very successful messaging platform. That does not make me an expert either but I do have some minimal understanding of what goes on under the hood. If you are simply configuring swap space on a better-safe-than-sorry basis then I would rather not have you near a system whose response has to be deterministic because it almost certainly will fail in some unpredictable way sooner rather than later.

Yes it is difficult stuff, no you do not get a free pass just because you messed with important stuff on a duct-tape-and-glue basis. See, swap is just a stay of execution, once a system starts swapping it is as good as out of control anyway so in the vast majority of cases that I am familiar with you want that situation to announce itself loud and clear and in a way that gives you back control. You then do postmortem and you fix it so it will not happen again. That is far preferably over having systems that remain partially broken but pretend to be fine.

I know plenty of people run after each other claiming great insight by quoting the various help-yourself sites but it amounts to absolutely nothing when you suddenly find yourself trying to figure out why your transaction queues are overflowing because some system in the pipe suddenly decided 100ms is as good as 10 as long as things keep flowing.

If on the other hand your systems are not important enough to properly set them up with static resources committed to long running processes and with strict limits on what logged in users can do from the command line then this advise is going to fall on deaf ears because you did not need reliability in the first place. Note that in many business context latency is far more important than throughput.

if swap is 'cargo cult administration', pretending it won't save you a few times even with all the safeguards you claim that you perfectly do is 'narcissist administration'.

if your systems are so advanced that you've done all of this up front profiling, you'd know why the system in the pipe decided 100ms is as good as 10 because you'd be monitoring and alerting on memory/swap and would get the message when it hits 80%-90%, and be logging in while leaky process is blowing up into your swap space.

no need for a 'postmortem' if you can have a surgery..

If you're architecting your systems in such a way that there is a dynamic component that can cause you to hit the swap then your could also simply buy that much more memory so it won't save you.

All you need to do is to monitor your memory usage, any growth that you do not understand is a reason to stop what you are doing immediately and to figure out what is going on.

A few GB extra swap space will not save your bacon, will make it much harder to get the system back under control because it is no longer responsive compared to a process killed with a supervisory process restarting that process immediately and logging a fault.

And if your systems can't handle that you have bigger problems, likely you then also won't be able to deal with hardware faults, crashes, power failures and other errors.

A postmortem is far preferable when it is about a process that is re-launched in a small fraction of a second after which the system is back to normal if the alternative is a system that causes a whole cascade of stuff to go out of whack down the line.

Pretty much only Erlang/OTP get this sort of thing right to begin with.

Errors - including out of memory errors - should be expected and should be dealt with in a deterministic manner.

There have been many times, in 30 years, where I've been glad I've got swap enabled so I can recover a nearly-out-of-control production system in time, with the right procedures, instead of everything just being killed.

Thats enough for me to turn it on. But, also, gigs of RAM is another must-do cargo-cult thing...

Did you do root cause analysis and did you make sure that same condition could never happen again or did you figure that since the swap file saved you that no further action was needed?

The 'many times' has me worried.

Many times .. over 30 years. And yes, it was mostly due to bugs in the code, which I wouldn't have been able to assess if my machine had just OOM'ed everything. That's the point: swap gives you a little leeway for these analyses.

>you do not get a free pass just because you messed with important stuff on a duct-tape-and-glue basis

Hahah. That made me laugh.

Before you go insulting people, maybe listen to what highly-experienced folks, like myself, have done.

Reply on news.ycombinator.com