Comment by rdtsc | Hacker News Reader

Comment by rdtsc | Hacker News Reader

As a fun heuristic I have come to believe that mmap advocacy and use for storage is somewhat correlated with medium experience level.

At first junior developers don't know about it, then, they read a blog, hear something at a meeting about it and all of the sudden they fall in love with mmap. Then they want to re-write every storage related thing using it claiming it will be lightning fast and efficient. "You just write to memory and kernel writes to file, that so awesome, here I refactored your storage backend to use it over the weekend!"

Some might even venture to write whole database products based on it.

But then one day they "grow up" and realize what a great pain mmap can be, their storage backend is being hammered constantly by input data and dirty pages are not flushed fast enough, and then they are they start blocking other processes for seconds at a time. THey start tweaking madvise setting trying to control behavior, adjusting number of pdflush threads, etc. Or, and also sigbus errors are fun...

And yet it exists, and works gloriously: http://symas.com/mdb/doc/

The kernel is smarter than your app. Trying to force it to do something it doesn't want to do is a very bad idea.

Kernels generally aren't all that smart. They're usually dumb and simple programs that do things based on some average defaults that look okay and might work for most people. The kernel needs to be told what to do especially if data integrity is important. Like if some dude thought it was a great idea to store up writes and do them in large transactions and you're writing a filesystem that needs verifiable writes, you turn that off or force it off if you can't disable it otherwise. Relying on the utter sausage factory that is your average kernel is a recipe for disaster. That is especially true for Linux in my experience.

I said the kernel is smarter than your app, not psychic. Of course you need to tell Linux what to do with specialized applications.

However, any option worth disabling can be disabled. Nothing needs to be forced off like the writer of the article did. mmap'd writes should be flushed as soon as possible, not dragged out, especially in the SSD age we have today. What the writer did was actually counterintuitive and makes no sense. None.

The kernel does have access to more information than your app, but your app has a specific job to do, while the kernel must provide reasonable service to all possible applications and may not be optimized for your specific case.

mmap() is still fantastic for mostly-read datasets. If you serialize and queue your writes, and 'replay' them with a little bit of heuristics about page alignments you can also help the kernel a lot with the flushing job.

I've discovered mmap 25 years ago on SunOS, I haven't outgrown it yet -- it has served me well over the years, and I expect it will continue to do so ;-)

The ignorance is strong with this one. Not worth the effort to enumerate all that's wrong here.

The Apollo DOMAIN OS was entirely built around single-level store, i.e. memory-mapped files. It scaled well beyond anything Sun was building in those days, even across large networks.

What would be the "high experience level" alternative?

Read and write, most of the time.

It's funny how code written by masters tends to look a lot like code written by novices, while intermediate-level code is full of baroque architecture and clever tricks that don't work. The difference between master code and novice code is that a master knows when he can get away with simplicity.

When you've written a world's fastest anything then you'll be in position to judge. But not before then.

How do you know I haven't? Besides, I said "most of the time". There are obviously important applications for mmap.

A fastest for loop that does nothing qualifies?

What about fastest product that run 1M TPS but crashes and is down 10 days per month? Is that still fastest?

Not really sure the point you're making, maybe you need some form of back pressure? How would that differ with varying I/O models? Fear of the operating system interfaces is a heuristic for me, especially an unwillingness to work on their implementation when the company has more than on order of 100 engineers.

striking already mentioned mdb, and I will mention kdb+

I'm not sure what those pains are that you are talking about. mmap allows use to share buffers with the OS in a way that is much more efficient than you can do with plain reads; it costs you some asynchrony and (practically these days) the ability to use a 32-bit processor.

Other than that, it is the same - you still have to reason about locality of reference and stuff.

Reply on news.ycombinator.com