Hacker News Reader

Comment by dons | Hacker News Reader

There seems to be some confusion here.

Iteratees weren't introduced to solve the problem of "eating up resources" per se. They're a new abstraction for doing high performance IO -- in any language. You've always had several choices for IO in Haskell: lazy IO (like pipes in shell programming), or strict IO (like say, Python or C IO).

Iteratees blend these somewhat, giving a third way.

So you could say that IO isn't well explained by computer science yet -- as new approaches are still being discovered. That doesn't really say anything about Haskell, other than it was the first developer community to really take iteratee-style IO seriously.

View on HN

Well, perhaps I am one of the many who don't really grasp iteratees.

In any case, my point about efficiency in the context of lazy evaluation not being well understood, still stands.

Yes, the space semantics of lazy evaluation is difficult to learn, particularly when coming from fully-strict languages, or languages with little support for lazy programming.

Many other programming techniques face similar barriers: e.g. memoization, concurrency or logic programming result in similar brain twisting -- yet can be suprisingly useful.

So I argue that programmers shouldn't be scared of technology -- embracing and learning new techniques can only improve your practice.

>So you could say that IO isn't well explained by computer science yet

I don't think I could ever be sold on using a language where IO is an unsolved problem. Computer Science has defined it quite well for me in Python

Python doesn't really have the equivalent power of Iteratees.

Iteratees let you compose chains of highly-efficient producers, transformers and consumers.

As a relatively trivial example, how would you write something in Python that performs this:

    for word in open("really_big_file", "rb").read().split(","):
        socket.send(word[::-1])

but does so with acceptable performance and memory use?

Using a generator I think this does what you want. What does the equivalent haskell code look like?

    def read_chunked(f, delim=',', size=1024):
        buf = []
        while True:
            data = f.read(size)
            if not data:
                if buf:
                    yield ''.join(buf)
                break
            if delim in data:
                spl = data.split(',')
                buf.append(spl[0])
                yield ''.join(buf)
                for s in spl[1:-1]:
                    yield s
                rest = spl[-1]
                buf = [rest] if rest else []
            else:
                buf.append(data)

    for word in read_chunked(open("really_big_file", "rb")):
        #socket.send(word[::-1])
        print word[::-1]

I'm pretty sure I got the gist of what you were after. What do you think?

Well, this is basically manually implementing Iteratee inline your solution. The idea is that Iteratees allow writing this as:

  main = run_ $
       enumFile someFile
    $$ splitWhen isComma
    $$ map reverse
    $$ iterHandle stdout
  where
    isComma = (== ',') . chr . fromIntegral

Basically Iteratees are a generalization of the technique you've implemented, but:

* Generalized for any error type and support various modes of error handling/propagation (they can go up to the producer, or down towards the deeper consumer as error inputs).

* Are generalized to many types of "buffers"

* Have a high-performance counterparts of standard "list functions"

* Are a standard base for many libraries to extend (http, xml, etc), and are faster, more powerful and more useful than standard strict IO libraries (Haskell's base libs, and Python's stdlib).

So both Python and Haskell have standard "strict IO" libraries. Iteratees are really a new idea -- that Python does not do well yet. So I don't think Haskell's progression of the state of the art in IO handling is really evidence that "IO is unsolved in Haskell".

That is a nice feature. thanks.

Reply on news.ycombinator.com