Things in Haskell which don't exist elsewhere at all | Hacker News Reader

Things in Haskell which don't exist elsewhere at all

reddit.com

170 pointsmietek87 comments

I've read most of LYAH, and I have no problem understanding mathematical concepts, but man, Haskell people truely don't know how to convey this "awesomeness" they keep talking about.

1. Where do you use automatic differentiation? I've done machine learning, signal processing, etc., but never even heard of it until now. Why should I care? Your pitch should include that (especially when the wikipedia article doesn't really provide real use cases).

2. What's special about lenses? I tried reading http://www.haskellforall.com/2013/05/program-imperatively-us... but there's no summary of what this is about, and from the first paragraphs it seems like a Haskell workaround for setters and immutability. Again, I feel like the community is not pitching these things correctly. People like me start reading, don't understand the point of it all, and give up.

And I can go on... What is the target audience of these features (or Haskell itself)? Is it people like me, or is it more hardware validation engineers, automatic proof system developers, database people?

EDIT: If anyone is interested, edwardkmett has replied to me in http://www.reddit.com/r/haskell/comments/1k3fq7/what_are_som...

Re: applications for automatic differentiation, there's a list of applications here: http://www.autodiff.org/?module=Applications. That's a list for the approach generally though, not the Haskell version.

Haskell's great contributions to its users are quite often advantages conferred by the type system. A number of the libraries mentioned (e.g. ad, lens, mtl, QuickCheck, MTL, among others) are made either possible or efficient by Haskell's ability to work with static types in a sophisticated and nuanced way.

Users of dynamically typed languages talk about "fighting" the type system—and yes, there can be a bit of that, espencially early on—but once you become a proficient Haskell user, you learn to use the type system and the guarantees provided you by the language to enforce properties you want to be true, and thereby effectively use the compiler as a pre-emptive, exhaustive testing mechanism for the correctness of your code. A good Haskell programmer will set up types in such a way that the logic errors of their program will be translated into type errors, so an incorrect program cannot even be compiled.

So, I suppose, the target audience of Haskell is people for whom program correctness is a stronger motivation than, say, the speed gains of C/C++ or the development time increases of dynamic languages[1] or the like. (EDIT: To avoid the charge of excessive language zealotry, I should clarify that I don't think program correctness is the overriding factor in all cases, and consequently I don't think Haskell is the correct tool in all situations, as well.)

[1]: I personally don't find Haskell particularly slow to develop in—in fact, I find it much faster and easier to develop in than the dynamic languages I know—but I do use it literally daily, and, to be frank, a new programmer will likely find it tricky and confusing until familiarity sets in. Similarly, Haskell can—used correctly—have spectacular performance, sometimes matching C and beating out C++, but again, it takes experience and understanding to milk the best performance out of Haskell.

Lenses begin with getting and setting. Since Haskell is immutable, setting is a little different—it's a method of quickly generating a mutation function for a type. Lenses keep getting and setting bound together as a first-class value which creates the idea of it being a value representing a "focus" on a particular field in a complex, nested value.

Then you can compose these lenses together and retain these properties. You can also pick lenses which operate with different multiplicities from 0 to infinity. Finally, you can use their seemless connection to object isomorphisms to create a very general interface for working with various kinds of "similar" objects in Haskell.

At the end of the day you can write a first-class lens which represents getting and setting over a set of parameterized Map keys deeply inside of a stateful computation, mapping over anything that looks like a string, and viewing/modifying it as decoded JSON.

   obj . ix "aKey" . each . _Object . ix "3" . _Number .~ 4

might modify a record in a type like this

   IsString s => SomeState { obj :: Map s s }

where the string values of the Map have a JSON schema like

   {"1" ..., "2" ..., "3": <aNumber> }

But you haven't presented anything that is special about lenses. For example, your lens expression would be trivially translated to this Python:

    for o in obj["aKey"]: o["3"] = 4

The OP:s point was that lenses seem like a workaround for Haskell's lack of mutable state.

EDIT: What I'm meaning is that to show lenses unique benefits, you would have to come up with an example in which the code is more succinct than the equivalent code implemented without lenses in another language.

But, that isn't anything resembling a faithful translation. It's non-first class, non compositional, only a "setter", lacks a decent error handling code (I didn't mention that, but it's built into "mistargeted" lenses) and depends upon parsing the JSON in some other step.

Lenses let you think of a JSON-encoded string as an actual JSON string without ever explicitly doing the decoding due to their close connection to Isomorphisms and "partial Isomorphisms" (called, non-standardly, Prisms here).

Furthermore, lenses don't really have anything to do with mutable state—they just happened to form a convenient wrapper for using the State monad, but that's really a coincidence.

Succinctness is difficult to grasp. It'd be a good exercise that I'm not going to try in an HN comment to even translate the entirety of the concept embedded in that one line into, say, Python. It would start to feel like an XPath implementation.

(Edit: Also, as usual, the whole typesafe thing. That Python fragment can lead to runtime errors. The Haskell one never does—using it inappropriately is simply impossible.)

Still, you haven't shown why anyone should be impressed by Haskell's lenses. It's like you do not understand that isomorphisms and first-classness is totally irrelevant to me and most programmers unless it leads to better code.

Here is a challenge for you or anyone else who loves lenses:

Take a small snippet of real source code you or anyone else has written and uses lenses and post it here. I'll then translate it into Python that has the equivalent effect. If the translation is impossible or is less pretty than the Haskell original, I'll award you $1000 internet points.

    import Control.Lens

and let us use two components from lens:

    rewriteOf :: Setter' a a -> (a -> Maybe a) -> a -> a

which can take a rewrite rule and apply it to any 'self-similar' setter recursively in a bottom up fashion until it ceases to apply and

    uniplate :: Data s => Traversal' a a

that says that if we have an instance for Haskell's built in generic programming framework 'Data', we can get a traversal of the immediate descendants.

Now

    rewriteOf uniplate $ \case 
      Neg (Lit a) -> Just $ Lit (-1)
      _ -> Nothing

will walk a syntax tree looking for negated literals starting recursively from the bottom of the tree, applying that rewrite rule on the right hand side until it no longer applies and fold it back up the tree. This works in a lazy setting where you can have potentially infinitely many targets and you didn't have to write any code to define the traversal.

The data type itself was just something like:

    data Term 
      = Var String 
      | Neg Term 
      | Lit Int 
      | App Term Term 
      | Abs String Term 
      deriving Data

Let's say you've serialized a tree structure of versioned data as JSON. Branches are arrays and leaves are objects.

    data OTree = Obj Object | Node (Data.Vector.Vector OTree)

    instance FromJSON OTree where
      parseJSON (Array as) = Node <$> traverse parseJSON as
      parseJSON (Object o) = return (Obj o)
      parseJSON _          = fail "OTrees are objects and arrays"

    instance ToJSON OTree where
      toJSON (Node as) = Array (fmap toJSON as)
      toJSON (Obj o)   = Object o

Now some of these objects have keyed "version"s which are arrays of semantic versioning numbers. Write a function which decodes and re-encodes a new tree with each of these semantic versioning numbers incremented at the patch level. If the version isn't in that format, just ignore it.

In Haskell you'd want to write a generic traversal over the objects of the tree useful whenever you want even access to the contained elements.

    eachObj :: Traversal' OTree Object
    eachObj inj (Obj  o ) = Obj <$> inj o
    eachObj inj (Node as) = Node <$> traverse (eachObj inj) as

And then here's the finale, the actual lens code specific to the task.

    upgrade = _JSON . eachObj . ix "version" . _Array . ix 2 . _Number +~ 1

That's a contrived example and not "real source code." Furthermore you are leaving so many symbols undefined that it is hard to see what is going on. Where does 'traverse' come from? Nevertheless, here is how you would do it in python: https://gist.github.com/bjourne/6219037

It's extracted, not contrived. Updating nested attributes on a tree of objects as a nice one-liner. The most contrived bits were that I didn't use the built in tree type so I had to define more stuff explicitly.

Traverse comes from Data.Traverable but is exported with lens as lens can be seen as a generalization of traverse.

The best thing that exists in Haskell and nowhere else I've found isn't a library at all:

It's the experience of grubbing around in unfamiliar libraries (in my case a database lib and a visualization lib), not knowing much Haskell and struggling to get the damn thing to compile. The very first time I got the types to line up, the program ran and I had my timeseries graph from the database.

This is a magical, life changing experience.

Any other language I've used would have been many iterations of, "this is null", "you passed a string where it was supposed to be a list of strings" (looking at you python)! Now that I know it's possible, I much prefer to do my thinking up front, and know that there's a compiler out there that's got my back :-)

That's a property of libraries as well, for instance avoiding the perturbation problem at compile time with with ekmett's AD library.

Many Haskell libraries are focused on allowing you to define entities that correspond to your desired semantics, while many libs in OO-languages are just about constructing the correct object-graph that happens to give you the result you want.

Could you please elaborate on this perturbation problem? It's the first time I hear that term.

http://conway.rutgers.edu/~ccshan/wiki/blog/posts/Differenti... was linked in the reddit comment.

I used to think this too, but realised that it is not quantitatively different to me whether I spend the time getting something to compile, or spend the time getting it to execute correctly.

I tend to prefer dynamic typing these days as a lot of what I do feels easier to achieve when I can throw something together to test ideas without thinking about types, and then refine the idea based on actually using a cobbled together prototype, and I feel static typing often got in the way of that.

Especially coupled with Smalltalk style runtime inspection / modification on error (e.g. for Ruby I use "pry" which lets me drop into a shell anywhere in the program and modify or inspect state and then continue execution).

> I used to think this too, but realised that it is not quantitatively different to me whether I spend the time getting something to compile, or spend the time getting it to execute correctly.

When do you determine you've gotten it to execute correctly? Achievement Unlocked: You Can Stop Testing Now?

That you get the results you expect.

How is that any different from Haskell? You think that merely lining up types and having it compile determines that it also "executes correctly"? There are also algorithmic and other mistakes.

>How is that any different from Haskell?

With haskell, the compiler verified tons of possible errors to guarantee against them. With smalltalk, you verified those tons of possible errors yourself. People make mistakes.

People make logic mistakes that'll fly straight past a type checking compiler too.

In my experience, most of the type related errors I make in Ruby "fall out" when testing functionality anyway, so that the amounts of tests that actually need to care explicitly about typing tends to be tiny.

You'd be amazed how few do. I'm frequently amazed. Furthermore, when you start getting into advanced types it's amazing how many errors become simply impossible since your types encode the logic your program is intended to represent.

For an example, here's a blog article linked elsewhere on this page which talks about encoding various "dimensions" of infinitesimals in an automatic differentiation program that thus eliminate a whole class of value errors.

http://conway.rutgers.edu/~ccshan/wiki/blog/posts/Differenti...

In my experience the most problematic bugs are the bugs in peoples thinking about the problem. Those bugs are easy to translate into "working code", because they carry through all the way from the top.

Then I much rather want a program that is easily malleable, than one where the flawed logic is encoded in the types.

If the problem space is known in detail upfront, then sure. But then writing test cases to cover them is just as simple.

Not least because if you really want to, writing test cases and type checks to enforce typing is trivial to do in dynamic languages too. In fact there are quite a few examples on how to do advanced type checks on Ruby code. With the level of introspection Ruby does, you can e.g. easily "wrap" every method with detailed enforcement of type invariants.

You need to run some functional tests to exercise the code paths to trigger said type checks, but you need functional tests anyway.

People don't tend to use those methods because in practice they turn out to rarely capture many interesting errors once you write functional tests anyway.

>Those bugs are easy to translate into "working code", because they carry through all the way from the top.

Actually, they aren't. Try haskell or ocaml or clean or SML sometime. You have to have a clear and consistent model of what you are doing in order for your code to type check. I can't count the number of times I've had my incorrect thinking about a problem be forced out of me during the initial development by the compiler.

>Not least because if you really want to, writing test cases and type checks to enforce typing is trivial to do in dynamic languages too

No, it is actually a huge pain, you need to write a type checker. Writing a "is this an int?" test does not give you the assurance of type checking. That only finds out if the value happened to be an int under that particular execution of the application. A type checker completely guarantees that there is no way for it to be the wrong type ever. Even if the only thing static typing got you was the complete elimination of NULL/nil exceptions, it would be worth it. But it gets you way more than that.

>People make logic mistakes that'll fly straight past a type checking compiler too

Which is why I want to be able to focus my mental power on those mistakes, rather than all the other mistakes the compiler will handle for me. There's only so much brain power available, I don't like wasting it.

>In my experience, most of the type related errors I make in Ruby "fall out" when testing functionality anyway

My experience is quite the opposite. So much so that I believe the only way people can actually come to that conclusion is by not realizing how many errors are actually type errors, because they have never used a type system that is expressive enough to be helpful.

I think it's a genuine weakness of static types that they inhibit prototyping when you first begin. HM type systems tend to be liberal and powerful enough that once you learn them well they're easier to prototype in—but that comes with time.

In particular, type-driven-development is wonderful. You can write light, compiler-checked specifications of code you haven't bothered to write yet and see if your logic all typechecks even without implementation. I can run through lots of ideas that way very quickly since the only thing I have to produce is the high-level type skeleton of my program.

Then I just go through and implement it all.

It's a lot like test driven development except that your executable tests are much more lightweight and carry a strong sense of the logic of your program. If your tests don't hang together, you'll find out eventually. If your types don't hang together, your compiler tells you immediately.

This is a really interesting comment. It would be worth turning into a longer post with examples.

Put it in my queue. I'll post it here when I get around to writing it.

I think better guidance and inspection in figuring out type errors would be of huge value in Haskell.

Have a google for "hole driven development haskell" for a few possible ideas.

Oh! Yes! Coding in Agda is incredible sometimes.

Right, I didn't mean to imply there was nothing interesting happening in that direction!

> Any other language I've used

Have you tried Lisp? I've always found that to be a really friendly environment when it comes to working out why something doesn't work. :)

Haskell doesn't need to provide a nice environment for working out why something didn't work—the compiler tends to just tell you precisely why your program isn't working. It's a bit cryptic to learn its language to start, but I seriously executed a 10 module refactoring yesterday by just going in, changing the core code, and then letting the compiler spit out a categorical list of each place that broke and the exact fix needed.

Edit: that said, it still has one. GHCi is a nice Repl.

Isin't that true for statically typed languageS?

Partially, but not the the same level as Haskell. It's pretty easy to get broken code past the compiler in C# for example.

Could you give some concrete examples?

NullReferences in C# (and lots of other languages).

First example is nullability. Anders says ~50% of production issues with c# involve null deference errors. Haskell tags in compile time which types may or may not be null.

In c# if your method is supposed to return one thing and set an object attribute of another, and you forget setting the attribute, the compiler won't help.

In Haskell, you don't use mutability much, instead you return both things. If you forget to return it you get help from the compiler.

Another example: in Haskell, you write a sort function but you forget to handle the base case (always returning an empty list instead). You get an inferred type like:

  sort :: Ord a => [a] -> [b]

Which tells you you forgot to actually return the elements.

Another example, in Haskell you can represent a red black/avl/b tree while also type encoding all the invariants.

For rbtree this type level encoding is about 7 short lines long.

Then all the dozens or hundreds of lines of code implementing the tree are guaranteed to maintain the invariants, no unit testing needed at all.

Another example, ST is a monad that let's you write deterministic imperative computations that can be wrapped with a pure interface (hiding the imperative innards). A small RankNType and a phantom type are used to tag both the ST computations and the mutable data, giving a compile time guarantee that mutable data doesn't leak between different computations. This guarantee means you can trust ST computations to actually be independent and thus the pure interface is safe.

Thanks! Some of those I knew from OCaml.

Its not in generally; statically-typed languages (e.g., C, C++, Java) make impossible certain classes of runtime errors that occur in dynamically-typed languages, but don't actually reduce the need to test each path through the code you've written to assure correct results, nor do they reduce the number of defined paths through the code assuming that the defined interface is adhered to (they make it impossible to not adhere to certain parts of the advertised interface, but even in a dynamic language you don't need to test undefined cases, as there is nothing to verify, so that doesn't actually narrow the test surface.)

Haskell's type system and syntax are richer in ways which I think makes it possible to unambiguously describe more results in code in ways which makes unit testing certain aspects superfluous. I'm skeptical of most "you don't have to test" claims, but I can see more of a case for reducing the need for testing with Haskell than with many other statically-typed languages.

Though it won't surprise you, OCaml development is very similar. You're right: it's like shining a UV light on my code and seeing all the type mismatch bugs.

The entirety of that thread has some very interesting discussion (http://www.reddit.com/r/haskell/comments/1k3fq7/what_are_som...)

Wow, that's awesome! Makes me really want to learn Haskell, something I have been putting off for far too long.

I can write libraries in COBOL and then proceed to claim "these libraries only exist in COBOL, therefore COBOL is better than other languages", which is to me exactly what this post reads like.

Actually, the full title is "Things I love in Haskell that don't exist elsewhere at all"

... followed by a list of (mostly his own) libraries, which is what my comment was getting at.

edwardkmett is a bit of a Haskell guru, so it's natural he would have a lot of his own work to point to.

Regardless, his argument makes little sense.

I eagerly await your COBOL lenses library.

Perhaps his "argument" doesn't make sense because it isn't an argument, it's a list of examples. Of course, insinuating that you don't trust the comment because the author points to his own libraries is a golden example of an ad-hominem fallacy, but in order for such fallacies to be relevant, this would have to be a debate, and it isn't, it's just you aimlessly complaining.

If you wanted to make it into a meaningful debate, perhaps you could furnish us with a non-Haskell example of:

- A differentiation library that avoids the perturbation problem at compilation time,

- A library for defining compositional uniform access across structured data types,

- A dependency injection framework with an algebraic basis,

- A speculative evaluation library that can safely discard running processes,

- A name binding library free of ordering effects

While I'm quite sure with some searching you could find one or two of these, most of them depend on the strong safety guarantee provided by Haskell isolating side-effectful processing in the I/O monad. Other languages can do this (or have other ways to do this) but none are popular enough to have such a wealth of libraries as to have these problems already solved. Unless, you know, I'm factually off-base. I look forward to seeing what you have to show us.

!!!

That's the first time I noticed that mtl was a DI library. Transformer classes are just such a nice concept in and of themselves that I never found value in mapping the concept back to something like DI. I'm definitely using that going forward.

But Edward Kmett is not the author of the Reddit thread! You make it sound like it's been staged to showcase his stuff. He can certainly point to the (awesome) libraries he has built in answer. The original question was

Besides features of the haskell language itself, I was wondering if there were any libraries in haskell that stood out as being unique--not a sinatra-like web interface for example. Something like erlang's mnesia or rails for ruby, maybe, or hazelcast for java or any frameworks (for web programming or whatever), that were truly a step above comparable libraries or don't even have a non-Haskell equivalent. Something that once you use it, you hate not using it in other projects.

That's pretty ungenerous of you. There are factual differences between languages, and Ed's list is full of illuminating examples. You've wandered into a conversation on /r/haskell between Haskell users about why they like Haskell. It's insane to expect them to be less excited or biased.

I'm not complaining about /r/haskell posters making pro-Haskell posts. I'm guilty of similar things.

I'm just not sure what it's doing on the HN front page, and I find the contents of the post lacking in terms of convincing outsiders into a favourable opinion on the language.

There's nothing wrong with such a comment on /r/haskell, but I think it's out of place here. I realize the fact that it is voted up to the front page proves that many people think otherwise, but when I last checked, people were allowed to voice contrary opinions on the Internet.

I haven't seen anyone trying to prevent you from voicing a contrary opinion, only people politely trying to explain to you that they feel you are missing an essential point of understanding. (OK, I was a bit snarky, but it was on point)

You're right, I was a bit overreacting with the opinion bit, and slightly thick-headed in accepting the arguments of some of the other posters.

We've all been there. No one who looks at Haskell comes away unchanged. :-)

No, you've exactly missed the point of the thread and of Kmett's reply. Every single thing he lists (except perhaps the containers library) uses some unique features of Haskell, or is uniquely well done in their Haskell implementations. Read it again!

Maybe, but the response doesn't really explain for most of the cases why that is the case or why we should care for anyone that doesn't already use Haskell.

The target audience is r/haskell. I highly doubt the point of it is to explain anything to people that don't use Haskell.

That's fine, but as another commenter points out, while that's perfectly fine on /r/haskell, it makes it a poorer post here without additional commentary as to why these things matters.

It's a first-person source. You can gain more value from it than a third-person one, but it requires more work to get those advantages.

I agree with this, but I think the appropriate reaction is to ask what he's talking about rather than complain about self-promotion.

Yes, if we only look at things with a naive high-level generalization then we can force anything to be constrained by our existing world-view. However, as it turns out, there are some details worth paying attention to here.

I played with reflection in Ruby for a short time and found it really fascinating, until I realized everything got pathologically slow quite quickly - especially if you add one reflection effect on another.

All this stuff sounds cool. Is it fast and scalable?

Generally, yes. An experienced Haskell guru can optimize the heck out of everything (see [1], for example), sometimes even beating C.

But still there is a lot of traps, a lot of situations where high-level nature of the language makes writing an efficient implementation harder and obscures that (unless you're an expert in GHC internals). For example, darcs is known for accidental problems on large projects (though I can't corroborate this from my experience), maybe of algorithmic nature.

From my experience: choosing a right library matters much. Choosing one numeric library from cabal instead of another had drastically boosted performance of a number-crushing task for a crypto course. A simple program using HXT (xml processing library with beautiful mind-boggling API based on arrows) has exploded on a large input.

Also, I am interested pretty much what's the largest site written in Yesod/Happstack/Snap and how well they behave there (these frameworks' benchmarks are mostly good and promising).

[1] - http://benchmarksgame.alioth.debian.org/u32/haskell.php

The two things aren't especially commensurate. A lot of the fun of Haskell is type-level stuff, which is resolved by the compiler at compile-time. There are no direct run-time artifacts. Moreover, several of the things Ed talks about have to do with concurrency, which isn't even something you can do with Ruby. Speculative execution and STM both enable real multicore concurrent processing. Even if it did perform as badly as Ruby, without the GIL and with real concurrency it's in a better position. Adding complexity in the type level does have one major performance effect though, which is on compile time. Does it count as fast and scalable if the compile time reaches into the minutes? In terms of compilation time, Ruby will always beat Haskell, because it doesn't have any. This is a meaningful difference when you're trying to iterate rapidly on your webapp, but most of the web frameworks for Haskell separate out templating to help with this problem.

Most people eventually complain that high-performance Haskell doesn't resemble the Haskell we're taught to program. This is less and less true as things like iteratee-based I/O and really advanced libraries like bytestring and text displace older facilities, but it is still something to think about. Haskell tends to perform well by sacrificing lots of memory. This is why you see discussion about "space leaks" rather than "memory leaks" and whatnot. There are a lot of ways to address the problems that come up, but you need to have a strong handle on the way Haskell works before you can accurately diagnose and treat the problem. This is a significant barrier to using Haskell in production. It doesn't take long to acquire the expertise to throw Ruby into production. The road with Haskell is a bit longer and a bit more taxing, because it's so very foreign.

Haskell tends to perform well by sacrificing lots of memory. This is why you see discussion about "space leaks" rather than "memory leaks" and whatnot.

I agree with your comment, but these sentences are a bit weird. Usually space leaks occur where you should have evaluated things strictly rather than lazily or the other way around. It's a bug (with the amount of memory that we have now).

But I don't agree that Haskell tends to perform well by sacrificing lots of memory. For example, you mention the iteratee proposals/libraries. Iteratees are well-known to be fast and can be used to do constant-space I/O. Bytestrings are much more performant than their precursor (lists of characters, aka String), but are also not demanding in terms of memory use.

Thanks for clarifying that. What I am trying to say with those two sentences is pretty unclear as worded. I'm trying to convey that the kind of Haskell you write as a beginner tends to perform well despite being wasteful. As you get better, you worry more about the waste and use more sophisticated stuff that's less wasteful. It obviously performs better, which isn't really what comes across. But it is usually a little more complex. The penalty is really slight for things like strict folds, strictness annotations and using better libraries like ByteString, but I'm sure you'll agree that it has a much more substantial effect on the look and feel of the code when you get to things like unboxing, using "raw" iteratees, and so forth.

can't the same be said for pretty much any language? If you did C++, and found performance problems, you'd drop down to assembly, which imho, is even more unwieldy than using more advanced haskell libraries.

> Haskell tends to perform well by sacrificing lots of memory

Not more so than any other garbage-collected language

Well, laziness has especially nasty pitfalls sometimes.

Indeed, but it's not "perform well by sacrificing memory" rather, "sometimes use a lot of memory unnecessarily".

> several of the things Ed talks about have to do with concurrency, which isn't even something you can do with Ruby.

Concurrency has always been possible to do with Ruby just fine. There's just been a limit on system level threads until 1.9 (and there still is a limitation in terms of the interpreter lock). There's on the other hand nothing that has ever stopped us from multi-process concurrency, including using shared memory (though with the caveat that Ruby won't let us put Ruby objects there).

There's a wide gulf between Ruby concurrency and Haskell concurrency.

There's a rather wide gulf between a moderate claim that there's a wide gulf between Ruby and Haskell concurrency, and the claim I responded to that it "isn't even something you can do with Ruby".

I am curious, though, what Haskell provides that can't be done with any of the current Ruby implementations? (not least because I'm back to working on my ahead of time/static Ruby compiler)

Generally parallelism is really much easier to express in a pure language and thus "sparked" parallelism is almost trivial to use in Haskell. I'm not sure that MRI even allows for parallel processes in its green threads due to GIL? Other Ruby implementations might have different restrictions there.

Also, as Edward mentions in his post on reddit, speculative parallelism is a Haskell one-liner.

    spec guess proc actual = 
      let speculation = proc guess 
      in s `par` if guess == actual 
                 then speculation 
                 else proc actual

(written in 5 lines for extra clarity on the syntax)

> I'm not sure that MRI even allows for parallel processes in its green threads due to GIL?

The GIL has no relevance when talking about processes. The GIL affects simultaneous execution of threads on multiple processor cores. It also does not prevent concurrency for threads or fibers. What it does prevent is for two system/kernel threads to be executing the Ruby interpreter code itself on different processors or cores at the same time.

This is generally not all that big of an issue, as e.g. any C extensions that are thread safe can tell the VM to yield from the thread that calls it, and any Ruby thread that does IO etc. will also be put back in the waiting queue and so won't hold the GIL.

> Other Ruby implementations might have different restrictions there.

jRuby, Rubinius and MacRuby are all GIL-less as far as I remember.

> Also, as Edward mentions in his post on reddit, speculative parallelism is a Haskell one-liner.

It's a little bit wordier in Ruby, but not much. There may better ways of doing it, but this spawns the "guess" thread (g), then spawns an "actual" thread (a), that if it finishes first will kill the guess thread (otherwise the guess thread will already have terminated, and the "kill" will do nothing. The main thread then waits for the g thread to terminate, either with a result or because it was killed. We then just try to kill "a" because we know either g terminated first, or a terminated it after it had finished - either way a is not needed any more. We then return whichever thread local variable has content.

    def spec guess, actual
      g = Thread.new { Thread.current[:ret] = guess.call }
      a = Thread.new { Thread.current[:ret] = actual.call; g.kill }
      g.join; a.kill
      g[:ret] || a[:ret]
   end

This requires "guess" and "actual" to be any object with a call method, which includes Proc/blocks/"lambda" statements (or "->" in Ruby 1.9). E.g:

   puts spec(-> { sleep(3); return "GUESS" }, -> { sleep(1); return "ACTUAL" })

In MRI 1.8 this will only execute on a single core. In 1.9+ it can use multiple cores, but will be subject to the GIL. On Rubinius and jRuby etc. it won't have to deal with a GIL. A multi process version would be immune to both. There are additional caveats to think about, like signals.

Of course this requires "guess" and "actual" to be thread safe.

Yeah, you're getting there at the end when you have no GIL and multiple cores being utilized. Without that, your speculative parallelism isn't faster, it's just as long unless it just happens to be threadsafe C extension IO.

Which is exactly the point. Haskell performs better with fewer caveats all the time.

No, it is substantially faster with the GIL too, for almost all "real" Ruby code. There are exceptions: code that spends most of their time in the interpreter. That's pretty much no Ruby code. If you're doing numeric computation etc. with present Ruby implementations, that would be impacted by it, you're doing something wrong - the method call overhead in all current Ruby implementations is too high for that to be a good idea.

And as I pointed out, the GIL is an issue in only one of at least 4 available Ruby implementations as of last count. So you're discounting the implementation based on the one Ruby implementation that is potentially scaling worse. Great. Let's find an old slow Haskell implementation to compare to too.

> Without that, your speculative parallelism isn't faster, it's just as long unless it just happens to be threadsafe C extension IO

Performance wasn't what was being discussed. The claim I responded to originally was that concurrency isn't "even something you can do with Ruby". I've demonstrated why that claim is flat out wrong, for every version of Ruby.

You then claimed "Generally parallelism is really much easier to express in a pure language and thus "sparked" parallelism is almost trivial to use in Haskell. I'm not sure that MRI even allows for parallel processes in its green threads due to GIL? Other Ruby implementations might have different restrictions there."

And I demonstrated how easy implementing speculative execution is in Ruby too, and pointed out that Ruby allows parallelism just fine even in old MRI version with green threads, new MRI versions with system threads, processes, or any of the number of non-MRI Ruby implementations that don't use a GIL, but with different tradeoffs.

You can continue to shift the goalposts if you like, but I've addressed the original claims sufficiently. And bringing up performance is irrelevant - I've pointed out the performance limitations myself repeatedly.

(and to whomever downvoted my reply above: that's the kind of childish reaction I'd have expected at Reddit, not here)

https://news.ycombinator.com/item?id=6198068

Haskell allows you to run "green" or userspace threads in the millions, similar to what erlang and scala (akka) provide (and F#), with dispatcher designed to run i nthose numbers. So it's something like Celluloid, but I don't know how many people have used celluloid in production, in haskell the runtime's been well documented (Simon Marlow's oreilly book) and tested in heavily loaded systems.

Do note that the changes described in the paper to allow green threads in the millions won't hit GHC until version 7.8.1 is released.

You can use millions of threads in versions <7.8.1 as well, they'll just run a lot slower if they're all actively doing IO.

I just ran a small test that started and ran one million green threads on Ruby 1.8.7, and it worked just fine. I won't make any claims about the performance, but it certainly works. 1.9+ supports both green threads (fibers) and system/kernel threads, and would be much better to test.

MRI is slow regardless, so unless you're bound by extremely slow IO from other components, you're going to run into other limits long before you reach those kind of numbers, though.

What in the world did you do? If you are defining method_missing all over the place, then yes, it will be slow, because you're effectively adding an extra indirection through an interpreted method lookup.

There are plenty of automatic differentiation libraries out there for C++, python, Matlab etc. Not sure how you spin that as a unique feature of Haskell.

The Haskell type system allows for avoidance of the perturbation problem (combining mismatched infinitesimals is a type error) for one.

Reply on news.ycombinator.com