Facebook's New Spam-Killer Hints at the Future of Coding | Hacker News Reader

Facebook's New Spam-Killer Hints at the Future of Coding

wired.com

139 pointsnols48 comments

Even as someone who knows Haskell, it isn't actually possible to read this article and figure out what specifically about Haskell is helping.

My suspicion is that it is Haskell's great support for domain-specific languages. This is both in the sense of languages that are really "just" Haskell APIs but work in such an different manner that they are quite different, and in the sense of writing a full compiler [1] (or any subset of a compiler) for a new language.

For an example of the former, one of my favorite demonstrations is the "probability monad", in which one ends up with a sort of sublanguage inside of Haskell that is probability aware: http://www.randomhacks.net/2007/02/22/bayes-rule-and-drug-te... [2] If you want to start to really grasp the power of Haskell, you could do worse than meditate on how you'd implement that in your $FAVORITE_LANGUAGE... and then, most likely, burst into tears when someone shows you just how much of Haskell still works, unchanged, even in that environment (e.g., everything in Control.Monad), rather than having to be implemented in a parallel type hierarchy (or, put another way, producing a new "color" of functions [3]).

While I doubt they use something exactly like that, it's not hard to see how that sort of thing could make it fairly easy to write spam classification code with a lot of the basic plumbing abstracted away entirely.

There's a lot of examples of the latter, but I'm not sure of a really solid open source example to point at. I know it's a common move in industry, though.

For bonus points, these things compose together just fine; you could easily take the probability monad and build a new language on top of it.

[1]: And I do mean real compiler, by anybody's definition, all the way down to assembler if you like. Haskell's a pretty good base language for that sort thing.

[2]: Or see http://blog.plover.com/prog/haskell/probmonad-refs.html

[3]: http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...

This is such a strange article. It starts out describing how Haskell was well-suited to their problem (or at least tries to) and then suddenly says:

The thing is: Biilmann no longer uses Haskell. It’s not entirely practical. Not enough people know how to use it, and this is unlikely to change. “Haskell is like a programming language from an alternate future that is never going to happen,” he says. “It solves all these problems it promises to solve. But it’s so different that there is no chance it will become common.”

Biilman isn't at Facebook - he is simply another person who likes Haskell. He's the one that doesn't use it any more.

Yeah it seems like they are just writing an article for the sake of writing an article.

And based on the number of grammatical errors, quickly, at that.

Those are two separate facts:

1. Haskell may be a very good language to use for this particular problem set, probably better than the languages currently used for it.

2. People in general don't use Haskell and will likely resist attempts to change that.

So in theory Haskell is a good choice (because reasons) but in practise it isn't (because developers in general won't be a able to understand/maintain the resulting codebase) so he is using something else instead that avoids this problem (but he may think is a less than ideal choice in other ways).

Yeah, I'm not at Facebook, but runs Netlify.

I still have several services written in Haskell running in production and they've been rock solid for years now.

For the kind of problems I used to solve with Haskell (typically tasks involving a bunch of concurrency and IO) I use Go now. Not because it's a better language, but because Haskell is so drastically different that having anyone else maintain these services would require tons of training. Meanwhile Go is extremely practical and very easy to learn.

tldr: use Haskell because it solves all problems, but actually don't use Haskell use Go/Rust instead because they are more suited to the mainstream programmer, but Haskell is more powerful.

What.

title is already a big hint that the article is bad

It's WIRED of course it's bad.

And the thing is, that is largely why I love Haskell and Erlang... and then never use them for a project. Its really frustrating to me.

If you like Erlang but can't use it, consider either Go or Elixir. The former is probably the "more enterprise" and conservative bet, but Elixir is growing enough that it's probably not going away any time soon.

(Honestly, whenever I have to go back to programming in a language without cheap threads I am often sad with the BS that comes out of my fingers. Single-threaded languages, even ones wrapped around an event loop with various hacks to try to obscure the single-threadedness, are so 20th century compared to the ones that were built for it from day one.)

Go is a language with serious flaws. It clearly was not made with threads in mind, and just gives low level primitives instead of just exposing a model like Erlang actors.

I have high hopes for Rust, though. Even quite a few people at Google seem to have converted to the Rust camp, away from Go.

> It clearly was not made with threads in mind

What exactly are you trying to do? Golang was designed from the start for high concurrency programs. Instead of threads, they use go routines.

https://golang.org/doc/faq#goroutines

I've used both extensively. The practical differences between a goroutine and an Erlang process are small. The most significant difference is enforcement of isolation vs. isolation-by-convention (and bear in mind the latter is not the same as "no isolation"), but this is a bigger difference in theory than in fact.

OTP being missing matters less than you might initially think, too. I implement what bits of supervisor trees can be implemented here: https://github.com/thejerf/suture , and in general, with the way Go is structured, gen_servers and such just sort of melt away into "things you interact with over channels" without much fuss. (IMHO, OTP is generally a good thing, but the reification of gen_X so deeply into the language was an error. I actually prefer Go+Suture's way of bringing up a server; much simpler, virtually no loss of capability in practice. Again, the theoretical difference turns out to be larger than the practical difference.)

(And before someone angrily smashes the reply button, pleases meditate on the distinction between a "small difference" and "no difference", especially the subjective nature of such judgment calls. You can certainly quibble with the size I assign to the differences but please don't reply as if I claimed no difference, or as if I have not demonstrated awareness of the difference between a strictly-enforced no-sharing environment vs. unenforced no-sharing convention, and, likewise, let's not pretend "isolation by convention" is the same thing in practice as "traditional shared-memory threading". Memory sharing can bite you in Go, and while I've gotten pretty used to the necessary patterns to avoid that (spending years in Erlang first is really good training for that), I don't deny it is non-zero cognitive effort required.)

Go is good, and yes, every language has serious flaws. I think what you really meant is Go is nothing special. It's a nice language with nice syntax, but the ideas behind it aren't different.

The reason people like Rust is it is different. It's what people wished C++ could have been so many years ago.

With that said though, each of the languages have its place and uses. Sure many things can be done with both of them, but some things only Rust can do/will be able to do(many unstable things still). Both languages will be successful imo.

I can see Go and Rust being taught as the primary languages in the academic world and in the enterprise world. If I were starting out in school, I'd probably pick Go and Javascript.

I'm not sure in what setting would Erlang be inappropriate, but somehow Elixir would be fine. Elixir actually adds a further layer of complication compared to Erlang, which is a straightforward pattern matching, expression-based language with recursion and polymorphic functions for looping and conditionals.

Elixir looks Ruby-ish, but that illusion breaks down very quickly since the underlying semantics are anything but. Not to mention Elixir doesn't shield you from having to be aware of the Erlang runtime, the VM and OTP.

As someone who uses python, what are some reasons for me to switch to Go?

The language itself is far simpler (fewer built-in basic complex data types, composition over inheritance, no overloading, no slots) but still batteries-included w.r.t. the standard library. You'll likely get a performance improvement unless you are writing some really zany code. Static types and the compiler will actually make your life much easier when dealing with larger code bases (no more wondering what type a parameter is, if it compiles there is a much better chance that it will just work, fewer worries about unhandled runtime conditions). Less magic (no decorators or deep class hierarchies) which means its easier to trace control flow in someone else's code. Verbose, but still very clean error handling. IMHO, the import system is pretty broken and awful, but importing someone else's packages straight off of github is still much better than pypi / wheelhouses. Compiled static binaries by default means that building somewhere and then shipping a binary is quite easy.

Thanks - that's what I was looking for. I'll read more about the deep class hierarchies to see if it would be a problem. Shame to hear about the importing, I kind of like Python's importing (sans the circular dings).

Sure - the import thing really just depends on what you're doing and how you use imports. Personally I found it to be really cool at first, but later realized it's more of an 80% solution that makes great demos but runs into practical problems (not flexible, tools around versioning or and relative paths are hackish). With hierarchies, you can mostly implement them, but they language pushes you away from them, which is really a feature.

Depends on what you use Python for.

tldr: functional languages may be the future. Or maybe not.

Always in motion, the future is.

Hm, I took away this:

tldr: functional languages were the future for Facebook's new spam fighting tool, they weren't for a startup owner who has previous professional Haskell experience. Consider for your next project and possibly live the future now.

For technical reasons Haskell is a good choice.

For people reasons it is not (currently at least).

The second point trumps the first at this time. Unless I'm missing something the logic here is not as intractable as some seem to think.

It would seem that for people reasons Haskell isn't a good choice. However I would say that right now, for people reasons, Haskell is one of the best choices you could make.

Haskell is a very good flag for the kinds of people that make good programmers:

* It's hard to learn.

* Chances of getting a job are minimal, therefore people learn it mostly for their own self edification, and have the kind of grit that keeps them studying for a year or two with not external reward.

* Learning Haskell teaches you some fundamentals that aren't readily discoverable elsewhere.

No one is hiring Haskellers right now, so what you have is a pool of very able programmers, who would love to do Haskell (in fact many would love the chance to use Haskell so much that they'd take a pay cut to do so) and very few people offering them what they want.

If you're a start up, you want to hire a stellar team (possibly at a bit of a discount) and your domain is a good fit then you should seriously consider hiring some Haskellers and letting them write Haskell.

At Wagon we use Haskell as our main language. And we are hiring. :)

http://www.wagonhq.com/jobs/haskell-engineer

I would guess that you'd corroborate my characterization of hiring Haskellers as an embarrassment of riches then? :-)

My thinking is sort of along the same vein: you can train people, you can't "train" a language. And if you can't train the person, why did you hire them? The cost of training is much like the cost of evaluating frameworks and tools and choosing the best ones (with the huge caveat that you invested in a person, and they can leave... but if their strongest suite is the obscure language, it puts a damper on things...)

Saying Haskell is ideal to the problem but not what they use is not a non sequitur. Just because a language's intrinsic characteristics make it ideal does not make it the best tool for a large project with an evolving programmer base. The point stands that Haskell represents an almost platonic ideal that other languages and libraries are striving for.

I think the real key is to accept that with well-defined interfaces, polyglot environments really do have an advantage, because different languages really do have sweet spots.

And fortunately, POSIX gives us those well-defined interfaces. POSIX process boundaries, pipes and Unix & TCP sockets are excellent interfaces to separate tools written in different languages. A well-written RESTful service listening on a Unix socket can work wonders.

If one's using POSIX, then one's team can work on each tool in the language which best suits it and in which the primary lead is most product. If someone moves on, then the rest of the team can support it because a) good programmers can pick up good languages quickly (I don't want to hire or work with mediocre programmers) and b) each service is small enough that anyone can understand what it does by looking at a small amount of code.

At work we use a Haskell DLL loaded into Matlab. Haskell's type system allows us to have a fast, formal core to depend on, and Matlab allows us to do ad-hoc things at run-time with a fairly powerful REPL.

I mention this because there are ways to benefit from Haskell while still maintaining the sorts of "practical benefits" that are cited here.

I love the idea of old languages coming back in fashion. I secretly love MUMPS.

I am with you. I am so hooked on the built-in persistent storage support. I have never seen a language as cool in that regard.

From Haxl paper:

"We present an Applicative abstraction that allows implicit concurrency to be extracted from computations written with a combination of Monad and Applicative."

The more your code is Applicative the more concurrency you can extract. It's a matter of good abstractions rather than the language. I'm sure Rust can handle Applicative and Monad just fine.

Rust does not yet have Higher Kindded Types, and so cannot yet. We do want to add them eventually.

My jaw actually dropped when they said "they used a programming language called Haskell."

Were all my prejudices against the business practicality of functional languages wrong?

Nope.

"Biilmann no longer uses Haskell. It’s not entirely practical."

EDIT: ... and to say, if Facebook can't attract superstar Haskell programmers, then what chance any other startup?

Biilmann isn't at Facebook; he's just some random developer at a startup with experience in different languages who once used Haskell for a work project. He may as well be yet another HN commenter, and it isn't surprising that not everyone who has tried Haskell is using it! The reporter probably just found him easier to interview than the people actively using Haskell, for whatever reason.

Haskell at Facebook is still going strong with, I believe, two teams using it and actively recruiting. (I've been contacted by a recruiter myself, even.) And they've hired Simon Marlowe and Bryan O'Sullivan who are pretty much as superstar as they come. Recruiting woes are greatly exaggerated if not inverted for startups seeking a handful of engineers—Haskell actively attracts interesting candidates.

Biilmann here, I'm definitively not at Facebook no :)

I've run Haskell services in production for more than 4 years and still operate 4 of them to this date. I also contributed the wai-eventsource module to the Yesod project and build a Haskell powered eventsource addon for Heroku.

It's not my impression that there are tons of other people in the bay area with as much experience in actually using Haskell for real world production stuff. Most of them are probably at Facebook now and the journalist behind the article did interview them...

That said, it's absolutely true that for Netlify I now use Go for the tasks I would have used Haskell for earlier.

Haskell is an awesome, mindblowing language, and when I started using it for production tasks it was by far the best tool for the job at the time. By now, however, Go is a far more pragmatic choice for most of the tasks I used Haskell for before.

Unless your whole stack is Haskell based, the context switching involved in going back and forth between more traditional languages and Haskell are really brutal. Apart from that, it's very easy to bring someone new onto a team and teach them enough Go to be productive in a day or two. You just can't do that with Haskell.

That said, it looks like a really, really good fit for what they're doing at Facebook with regards to spam filtering. I can totally see how you can make a safe DSL for rules that will be statically type checked and handle concurrency without any need for developers to think about it, and the Haskell runtime is incredible for these kind of things.

If you want to attract Haskell programmers, simply allow working remotely.

That is exactly it; I know plenty (including myself) but relocating for a job is not of this day and age (travelling for team meetups is another story obviously). And obviously less obscure languages have more opportunities for working remotely.

I've read over and over that 'haskell is difficult', but why exactly is that?

Its mathy syntax? Its unique concepts? The practical side (installing, configuring and libraries can still be troublesome)?

Mutable variables are a hard habit to kick.

I think it's mostly the immutability. That can be quite frustrating sometimes.

Amazed that the article compares Haskell with Go and Rust but failed to draw comparisons with Scala (with some learning curve but not as great as the ones in question).

tl;dr they rewrote facebook software in haskell, so haskell might be good. It definitely changed the pace enough to make a boring rewrite more engaging. Never admit that publicly.

pretty dumb article. An editor could save it by shifting the take away towards the concepts polygot and the concurency requirement and it might make a bit of sense.

Reply on news.ycombinator.com