Hacker News Reader

Comment by cousin_it | Hacker News Reader

The top link on /r/haskell right now [1] is someone complaining that their program runs out of space due to a subtle interaction between laziness and IO. I think it's safe to say that we have tried laziness as the default and have learned that it's the wrong default, because it plays havoc with space and with IO.

[1] https://www.reddit.com/r/haskell/comments/5h6emf/haskell_run...

View on HN

Look, I fixed their code in so many ways:

    main =
        runConduitRes                        -- dealing with finite resources
             ( sourceFileBS "input.txt"      -- read input.txt as binary data
            .| decodeC utf8                  -- decode assuming UTF-8
            .| linesC                        -- split into lines
            .| mapC parseList                -- parse each line into list of text
            .| mapC (get 5)                  -- get sixth element of list
            .| catMaybeC                     -- discard lines with no sixth element
            .| encodeC utf8                  -- encode as UTF-8
            .| sinkFileBS "output.txt"       -- dump into output.txt
             )

And it's still very functional, if not more so!

This will run in constant space and linear time, it will buffer reasonably, it will not leak file handles, it will gracefully clean up on exceptions, it will not crash when it fails to parse something correctly, and it makes the encoding assumption explicit (you cannot split into lines unless you know the encoding).

Dealing with I/O is not hard when you use the correct primitives.

It seems to me that part of the problem he was having is that he didn't really understand how these methods were being implemented internally. I know that all languages inevitably have these problems but how do the amount of leaky abstractions compare in Haskell to other languages?

I don't think Haskell is worse than any other language in that regard. What perhaps sets it apart is how much of its functionality is implemented in third-party libraries, which may be difficult for a beginner to come to grips with. "Why should I download a library to use efficient arrays? Shouldn't they be built in, like in Python?"

I can come up with two explanations for this reliance on third-party libraries.

1) Haskell has always been a quickly evolving language attracting research-minded people which in turn go on to develop really cool libraries that are much better than conventional ways of doing things. The interpretation of this explanation is that it's simply not possible to keep the standard library up to date with the latest library developments.

It may also be the case that

2) Haskell has always been a really powerful language capable of offloading important tasks to libraries. What would need to be built-in functionality in other languages can be implemented as libraries with no sort of special treatment in Haskell, so people do it that way because they can, and because it keeps the base simple.

I'm leaning toward #2, largely because I know that's the Scheme approach, and I get the impression that Schemers and Haskellers have similar feelings about protecting the design purity of the language. By contrast, the Python community has always been pragmatic, arguably to a fault.

I can say that newcomers also seem to get a lot of advice that's directly at odds with how the Haskell community seems to think people should use the language. For example, Learn You A Haskell starts right off the bat by encouraging people to use the "list of characters" version of strings, even though that approach courts serious performance concerns.

It might be simpler than that. Array syntax is not built in in Haskell therefore the array implementation needn't be built in.

I'm sure I misunderstand you, because it sounds to me like other languages were designed on the basis of "Wait! How did this array syntax get built into my language? Oh well, now that it's there I guess I might as well build in arrays..."

I would guess those languages get built-in array syntax specifically because they have built-in arrays. The arrays come first, the syntax later.

I suspect that typically both arrive at the same time!

> > how do the amount of leaky abstractions compare in Haskell to other languages?

> I don't think Haskell is worse than any other language in that regard.

Many Haskellers are happy about this kind of stuff:

    min = head . sort

which is the very definition of a leaky abstraction. The blame lies with laziness, because it allows code to depend on implementation details of other code in crazy ways: "sort is O(n log n) unless you ask for only the first element, in which case it's O(n). What if you ask for the last element? Uhhh..."

Absolutely agreed! 'head . sort' is efficient by accident!

I wouldn't depend on invocations of sort to be anything but the worst case.

>"Why should I download a library to use efficient arrays? Shouldn't they be built in, like in Python?"

Python does in fact use a third-party library for efficient arrays (Numpy) and its use is so widespread as to be a defacto language unto itself.

I'm not sure why we keep pretending Haskell is supposed to work well. It's half prorgramming environment, half research environment. This shows in MANY ways. And don't get me wrong, the Haskell team tries, but ...

There really isn't a serious attempt to build an industry-friendly programming language with functional features and laziness as a default. You'd make many different decisions than Haskell:

- No more language extensions, you'd fix the feature set.

- Better transparency on memory growth and GCs in tooling

- It'd be easier not to generate a ton of garbage, so the GC could be designed such that very large working sets could be handled with low latency.

- You wouldn't structure IO the way the have. Modern programs have demands for many, many side effects and threading an IO monad through every place they can touch is common practice, but not useful. A reduced power version of IO that would let developers push events to an IO-enabled functional reactor would be in the stdlib.

- Monad Transformers would almost certainly not be included, preferring: http://lambda-the-ultimate.org/node/4786

We shouldn't hold Haskell as the perfect expression of functional and lazy programming. It was never meant to be and it shows. It's just the tool we have right now to go to bat with.

I believe PureScript solves most of these problems.

Thanks! After your comment I finally got around to checking out PureScript, and it looks nice (aside from the crazy tooling requirements which are typical for 2016).

Sure. I'll be moving to Idris as soon as it matures, since by enforcing totality you gain the advantages of laziness without the problems.

In the meantime Haskell may well be the best option for various circumstances even with that flaw though. (And note that strict evaluation in non-total languages brings its own problems)

No, that's not laziness per se, that's "lazy IO", something somewhat different.

EDIT: It's not even a lazy IO problem since no lazy IO functions are used there.

The problem is laziness. Lazy IO just makes it happen with files, which is more noticeable. In regular lazy code it happens with memory instead (space leaks).

these are not problems, only small details that you have to be careful of..not a large tradeoff to pay

That's like saying that bounds overflow in C isn't a problem, just something you have to be careful of.

To tell you the true, I can't understand why the code the OP talks about has a problem, and I can't even reproduce the problem (and never saw any of the problems of lazy IO in practice either). But I also can not say the code has no problem, and that is a big issue with the language.

> That's like saying that bounds overflow in C isn't a problem, just something you have to be careful of.

I think there's a significant difference between undefined behavior (hello security flaws!) and the program crashing in a (reasonably) well-defined way.

EDIT: I was wrong. It seems to actually just be a hideously inefficient in-memory representation of the data (akin to List[Char]) perhaps plus a space leak.

That's what I get for not reading the code carefully...

Reply on news.ycombinator.com