Haskell Optimization Handbook | Hacker News Reader

Haskell Optimization Handbook

github.com

61 pointsghuntley13 comments

Author here! I figured it was only a matter of time before this showed up on HN after the haskell foundation announced we had moved it the HF org. If you have any recommendations then by all means please open an issue, but bear in mind that the book is still very much a work in progress. And most chapters are just todos at the moment.

My goal is to have a handbook that consolidates and demystifies optimizing GHC Haskell because I think this resource is sorely missing in the Haskell community. So that includes reading and understanding Core, Stg, and Cmm as well as understanding the tools that already exist for GHC Haskell but are under documented in addition to the real advanced features, like altering the RunTimeRep your data types to control their behavior at runtime. Needless to say there is a lot to do :)

Link to site: https://haskell.foundation/hs-opt-handbook.github.io/

(rather than the source code)

In case this got you interested in Haskell, and you want a good way to start your Haskell journey (and have something to apply the optimization handbook to), check out IHP. It's the Rails/Laravel of the Haskell world. Here's a demo video https://www.youtube.com/watch?v=UbDtS_mUMpI You can start here https://ihp.digitallyinduced.com/Guide/index.html or check it out on GitHub here https://github.com/digitallyinduced/ihp

That long list of topics, and yet nothing on the critical one: understanding laziness. It turns out cargo-culting around a lack of understanding doesn't lead to good performance.

Though I have only skimmed the book, this strikes me as an unfair characterization. For example, this case study discusses laziness extensively: <https://haskell.foundation/hs-opt-handbook.github.io/src/Cas...>

I've skimmed that section and.. it's a case study of tracking down and fixing some space leaks.

That's not what I'm talking about. I'm talking about understanding laziness, such that you don't ever write code that's as bad as the starting point there. Treat space use as a correctness property, and make use of documented space invariants to show what you intended. When you do all of this properly, space use becomes a very tractable local property. And sure, some bugs will slip in - but you probably will find them with much less work than the writeup in the section you linked to.

fair enough—the book certainly seems to favor "make things strict enough so that your usual algorithms work" over "rewire your brain for laziness"

I think that is a fair assessment of that chapter. The goal of the chapter was to take a project that has never done any kind of optimization and to show an optimization engineering pass. Basically one has to be sure the implementation doesn't have any obvious easy to fix leaks before considering a different algorithm or something like that.

So I would argue that the real message of that chapter is demonstrating, step-by-step the methods used to find the memory leaks: info-table profiling and biographical/retainer profiling and ticky-ticky profiling.

One of the very first chapters is "Heap Profiling and Inspection," which is where too much laziness would manifest. So I really don't think this is fair.

We'll have to see when the book's done, but a practical, techniques-and-skills focused approach to Haskell optimization seems like it could be a really good resource. There are plenty of other tutorials out there ready to explain how `foldl` evaluates one more time.

A chapter dedicated to understanding laziness is indeed doable, but my target audience is Haskellers that have already read through LYAH, Real-World Haskell and perhaps UPenn's CIS 194 class; each of which cover laziness and so I want to focus on things that should be more widely used or known, such as info table profiling, eventlog or the one-shot monad trick.

But that doesn't mean that laziness doesn't come up! For example, its impossible to demonstrate using (or defining) unboxed or unlifted types without discussing laziness. The same goes for using GHC.Exts and explaining the difference between Data.Map and Data.Map.Strict.

Those books define laziness, and provide a couple examples - but they do not teach a programmer how to use it correctly.

Correct use of laziness involves choosing sufficient space invariants, implementing them, and documenting them. This is critical for writing efficient code in Haskell, and rejecting the "everything strict" cargo cult at the same time allows you to recover compositionality.

There was a period of time around 15 years ago, back before core libraries really started to understand this concept, and they would often have updates that silently made things too strict, breaking my code that was using their previous laziness in ways they hadn't predicted.

And it bugs me when I see new resources being set up to train people to write code that prevents my creative uses of their libraries. We should teach people how to write efficient Haskell code that's still Haskell code. It's great that we have so many advanced strictness tools when they're needed. But they shouldn't be reached for before we know if they help.

Great! If you could open an issue and perhaps lay out what you would like to see I would be more than happy to add a chapter like this. This book should serve the community and I think you've described a good gap that the book has which we could close with such a chapter.

You're right. I should at least give you an outline for the topic.

Reply on news.ycombinator.com