Leaving Microsoft | Hacker News Reader

Leaving Microsoft

haskell.org

304 pointsprotothomas97 comments

FYI, Simon Marlow is incredibly big in the Haskell community and is the leading expert in Multi core parallelism via STMs in Haskell. Some of the work he has done for the GHC compiler is frankly fantastic.

This is incredibly big news in the Haskell world. The FP community is starting to break boundaries with the industry in the last 5 years with Clojure (and Scala's functional support).

Currently, FB seems to use haskell only for basic lexing and parsing[1]. It would be tremendous to see them using it at FB scale. I think there's going to be gain for both parties - For FB, the publicity and hacker outreach, and Haskell - industrial reliability.

[1]: http://www.haskell.org/haskellwiki/Haskell_in_industry

Please note that although the latest commits on [1] were 2 months ago, for a long while it hasn't been updated and/or usable to a decent degree.

It seemed to have been deprecated in favor of [2] when the original author of lex-pass left Facebook.

Sorry if any of this information is a little off/incomplete, I think it's been 2 years when I oversaw a BSc paper revolving around extending lex-pass.

[1]: https://github.com/facebook/lex-pass [2]: https://github.com/facebook/pfff

The corresponding BSc paper may be found and downloaded here http://blog.seric.at/2010/07/19/automatic-anti-pattern-corre...

Pedant calling - Erlang is a functional programming language that has been doing very nicely thank you in a lot of big companies (Amazon, Facebook amongst others) for a good while now.

And F# in the financial world, along with Scala.

Other Functional languages are being used more as well, but since a huge amount of financial software sits on either a .Net or a Java stack, using FP languages which can easily interop with the rest of your libraries makes life a lot easier.

Very enlightening. I always thought that the "financial world" were locked into Fortran/COBOL.

That's more true in certain Enterprisey, conservative parts of the financial world, backoffice bank records and that kind of thing. In investment/quant/trading parts there's more experimentation with other languages, both functional languages like F#, and even particularly "weird" languages like K (an APL successor).

My rule of thumb these days is that the smaller and more obscure the hedge fund, the more likely it is that they're using an unusual set of languages. The more PhDs they have, the more likely they're using something weird like K!

I used K (well, actually the combination of Q and kdb+) in my previous job at an investment bank, and I've used it (to a lesser degree) in my current job at a hedge fund (disclaimer: I have a PhD...).

K/Q is a beautiful language - it's uncompromisingly terse, extremely powerful, and very fast. The interop with kbd+ (a lightning fast, column-oriented, in-memory database) is gorgeous.

It has an unfortunate reputation for obscurity, stemming mainly from the people who think it's fun to write your entire program as a K one-liner and remove all the whitespace. You can write perfectly clear code in Q - super modular, concise, and well documented. It's just that a lot of Q programmers don't bother.

While k is extremely terse, a lot of investment banks these days are using systems which utilise q and it's associated kdb+ database. q is basically a wrapper around k which makes it a lot easier to read and code in and which allows for sql-esque select statements from databases. This combination makes it far more flexible and useful for dealing with financial databases than straight k.

The financial world is a big place. There's everything from backend systems at banks to stock exchanges, trading houses, stockbrokers, all with different requirements. I think perhaps the most ancient systems are the backend money ones at banks since there's not much money in scraping speed out of them, but I'd love to hear from someone in the field?

Certainly the enterprise stuff at Spanish banks is run on Cobol, and there's still a fair amount of active Cobol development going on. (By enterprise I mean the bread and butter stuff, accounts etc.)

No idea about the trading / derivatives / whatnot, though.

Hardly. They used to be in love with Smalltalk and even Objective C, even before it was cool with Apple and iOS. Also, lots of LISP. The financial world is incredibly hungry for talent and whatever technology can give them an edge over the competition.

There is almost certainly a hell of a lot of Fortran and COBOL floating around as well. Most of it is likely at retail banks where things centre around end-of-day processing runs and batch processing.

In the financial trading industry C and some C++ are very big, and FPGAs are starting to get attention.

In the insurance industry, there is a lot of Java and C#/vb.net. However, in some there are 50-year old programs running in COBOL and even BAL.

Jane Street uses lots of OCaml apparently.

And Haskell now, I believe.

The core financials may be in COBOL and assembler, but these days most things talking to them will be Java or C#; the middleware will most likely be Java as will many of the front-end apps.

And plenty of banks are migrating away from COBOL on the core systems (slowly), and for them Java is usually the new COBOL, and the JVM is the new s390 microcode.

Did not expect that many companies from the Netherlands using Haskell.

Probably because many university students are taught Clean (the 'other' purely functional lazy programming language) [1] which is developed in the Netherlands.

[1] http://en.wikipedia.org/wiki/Clean_(programming_language)

I would not bet on that. Utrecht (http://www.cs.uu.nl/wiki/UHC) is in the Netherlands, too. I also expect/guess that it has more CS students than Nijmegen

So every dutch university teaches a language they themselves wrote the compiler for. The university of twente used to teach FP in a miranda dialect called amanda, which ofcourse was developed at utwente :p

The University of Groningen uses Haskell and Hugs for their functional programming course ;). So, there seem to be exceptions to that rule.

I thought Clean was mostly taught only at Radboud, since it originated there.

Yeah, Utrecht University has some of the world's leading Haskell researchers, I think it is taught in Groningen as well. In Nijmegen they teach Clean, which is very similar to Haskell.

Did not know that about Utrecht Uni.

http://www.cs.uu.nl/wiki/UHC

Does it actually mean Haskell is going to permeate into Facebook's stack though?

Simon might just be looking for a new challenge, with Haskell as his side hobby.

> This is incredibly big news in the Haskell world.

I'm puzzled why you think this is good news for Haskell. The lead developer and architect of GHC and authority in parallel Haskell is announcing that he's taking a position that won't allow him to spend much time working on GHC, and he might even stop using Haskell completely.

It's great news for Simon but terrible news for Haskell.

Big /= Good

Unclear whether he meant big in the sense of good.

However, if it gets major traction at someplace like FB, could be quite good for the language. cf. 37signals and Rails.

According to Bryan O'Sullivan (bos) he will use Haskell at Facebook.

http://www.reddit.com/r/haskell/comments/13m8eu/haskell_leav...

> The FP community is starting to break boundaries with the industry in the last 5 years with Clojure (and Scala's functional support).

There is very little evidence of that. Do a search for Scala, Clojure or Haskell on the main indicator web sites (job boards, StackOverflow, TIOBE, indeed.com, etc...) and you'll find they are as insignificant today as they were five years ago compared to the mainstream languages.

While this comes up every single time somebody talks about functional programming making inroads, I still consider it a fairly unhelpful indicator. First, nobody's arguing that big companies are suddenly doing the bulk of their development in these languages. It's about using them at all. Second, job postings may be lagging and distorted in their own way (e.g., junior positions are much more likely to be publicly advertised, and so on).

The less said about TIOBE, the better.

I think when people talk about functional programming being used more in industry, they don't imagine it replacing Java or Python as the lowest common denominator language everybody uses. Rather, they imagine either small companies or small teams within larger companies using functional programming to be more productive.

So the sheer number of people using functional programming may not be very high even if functional programming is making inroads into industry. Now, I don't know whether functional programming actually is making inroads into industry or not, but my impression is that anybody relatively good at it could find a very nice functional programming job fairly easily.

Not to mention that they are also referring to functional programming being used within those existing languages. Even Visual Basic contains a pretty capable functional subsystem these days, with lambdas, higher order functions and so on (albeit lacking concepts like immutability and referential transparency).

Anecdotally (n=1, of course) I have seen and heard tremendous resistance to clisp in the industry I work in 5 years ago. Now, clojure and Scala are being adopted and recommended by even conservative PMs and architects.

The choices are way more today (Almost a tautology given old choices don't really vanish into thin air) and on more reliable platforms with tried and tested libraries.

anyway the spirit with which i meant that statement was that it is far easier to convince "senior management" and others with power to let stuff be done in clojure / Scala than it was to convince them about ml or clisp five years ago.

And I am not even talking about "ivory tower Haskell" (which i love using).

(Don't know enough about erlang / f# to comment).

My new recent project is in .NET and I happily using all functional like features C# 4.0 offers.

The datas do not match your assertions: http://www.indeed.com/jobanalytics/jobtrends?q=scala&l= http://www.indeed.com/jobtrends?q=clojure&l= http://www.indeed.com/jobtrends?q=haskell&l=

Yes, it does.

Your data is showing that Scala grew from 0.01% to 0.03% over the past two years. Sounds impressive, right?

It's a relative number. Whenever somebody quotes growth numbers instead of absolute numbers, they are trying to hide something.

Let's take a look at the absolute numbers:

http://www.indeed.com/jobtrends?q=scala%2C+java&l=

As you can see, Scala's line is a flatline around 0 compared to Java.

> Whenever somebody quotes growth numbers instead of absolute numbers, they are trying to hide something.

No, it is the data you asked us to look at and it refutes your claim that "they are as insignificant today as they were five years ago". They are _less_ insignificant. They may still be several orders of magnitude lower, but your claim is still incorrect.

If you are saying that 0.01% is insignificant but 0.03% is not, then you might have gone off a little bit too far on the fan boy scale.

A programmer who can't admit that his claim, expressed as an inequality, evaluates to false?

Just take absolute numbers: According to indeed, there are over 1000 open jobs mentioning Scala in the US, and over 750 in the UK. Taken together, the number of open Scala jobs in both countries exceeds the number of open Groovy jobs. It's still small compared to Java jobs (to be precise 1.5% in the US, close to 3% in the UK), but Java is by far the most demanded language, so this is not surprising. To say there is no pick-up in adoption of functional languages is disingenuous, IMO.

> To say there is no pick-up in adoption of functional languages is disingenuous, IMO.

The question is not about pick up (there is some, 0.01% to 0.03%, according to indeed.com) but whether that pick up is significant.

And most of those job ads mentioning "Groovy" are for using groovy as an add-on to Java, while those mentioning "Scala" are usually for using Scala as the primary language.

And, I should add, given this level of demand it's a good thing we just finished the Scala massive open online class with more than 10'000 developers completing the course with success.

Job boards have one weakness: They reflect supply and demand. There is not much demand for Scala programmers. This may be because all positions are filled. Or, a more probable cause, that Scala programmers are not hired on Job boards.

Note that I have had 5-6 Jobs. I do FP for a living. And I have never been hired through a Job board advert. I think this is common with many FP people. Usually, when you want to hire FP people, you go to the mailing lists and community and get the guys you need. It is much more effective, but it won't show up in the Job board search at all.

> with Clojure (and Scala's functional support)

There's this perception that Scala is less FP than a language like Clojure and I don't understand why, when in my (albeit limited) experience, I found the contrary to be true. This impression that's perpetuated is IMHO only folklore coming from people that only played around with these languages superficially.

Scala feels more like a functional programming language to me because:

(-) Code written in Scala tends to be more FP. One of the popular Scala frameworks, Play 2.0, is using Iteratees for dealing with responses and I/O in general, a concept used in Haskell, being pure awesomeness if you want to do Comet/Websockets. Play itself feels more reactive than Node.js, even though Node.js was built from the ground up for this. Also monadic types are used everywhere, from the standard collections to the Maybe monad (Option) to Futures/Promises. In Clojure you rarely see this, because Clojure is so dynamic that working with complicated container types is unfeasible.

(-) the Scala collections library provides more variants of immutable data-structures optimized for different things and those immutable data-structures in Scala are doing the right thing - first they play well with each other (e.g. mapping over a Map produces another Map if the function returns a tuple, but produces a Seq in case it returns a single element and there are many more examples where Scala simply knows what you mean).

Many times you pick concrete implementations like Vectors because of the provided invariants (e.g. indexing in O(1) or efficient adding/removing of elements on the right) and changing those invariants is IMHO bad design. Doing operations on a Vector like adding elements shouldn't produce a list, even if that list is lazy or whatever other reason. In Scala you know what operations are going to behave lazily because such invariants are encoded in the type. Adding/removing/transforming elements from a lazy Stream or view or other kind of lazy collection is still going to produce a lazy collection. And this is extremely useful in languages where laziness is not the default.

From the perspective of its rich collection of immutable collections, Scala is more functional than popular ML languages in general, except maybe Haskell. In Ocaml idiomatic code includes the usage of mutable collections, while in Scala the mutable versions are second-class citizens - and before you say that Clojure doesn't have mutable collections, think again - arrays are part of standard code and working with Java collections is quite comfortable and given as example in Clojure books.

(-) STM and atoms are great. I use STM in Scala a lot lately. However stuff like atoms and STM are still mutable state and come with many of the same gotchas as plain vars and pretending like this is otherwise is dangerous. Precisely for this reason it's useful to be able to implement your own atoms and your own STM mechanism that is more suitable to the problem at hand, that better encapsulates state and so on and because of the reality we live in, the internals of such components can get messy, while still keeping referential transparency for the outside world or other kinds of invariants - STM in Scala is a library (and quite nice in fact since it forces you to use Refs inside transactions with compile-time checks); Erlang-style Actors are a library too; Futures/Promises are a library too. All of them created with Scala.

(-) Scala is OOP while Clojure pretends like it isn't. I'm using "pretends", as multi-methods were introduced in CLOS, an OOP extension to Common Lisp, which made no effort to hide the fact that multi-methods are still all about OOP.

But just because OOP is very efficient (and has been used) to hide mutable state, that doesn't make OOP any less orthogonal to functional programming. Maybe tying up data with functions operating on that data leads to bad design, but besides the fact that type-classes are also part of idiomatic Scala and used everywhere, OOP is actually a quite nice and comfortable ad-hoc mechanism for polymorphism, being quite orthogonal to FP. Clojure is only able to have multi-methods and protocols because it is dynamic, but as far as FP is concerned this has drawbacks. Also in regards to multi-methods, I've seen code in the wild that dispatches on some global state instead of their arguments, since you can dispatch on anything and I'm really talking about mutable state.

IMHO I can't find evidence that Clojure is more FP than Scala. I'm more experienced with Scala than Clojure, so maybe this is why. But IMHO, Scala is more potent if you want to explore concepts of functional programming, being in my opinion the second choice after Haskell.

And I do think that this perception only has legs because Scala has a Java-like "while" loop and doesn't hide the fact that "vars" are really vars.

I think idiomatic Scala code should be FP. And although you can just quite Java-esque code in a slightly more concise way in Scala; I do think that people are starting to see that Scala is more than that - it introduces new ideas and idioms.

Scala and Clojure, the two leading alternative languages for the JVM, certainly have different strengths:

* Scala is statically typed, even using parametric types, while Clojure is dynamically typed

* Scala enables auto-currying when methods are defined in the Haskell style, while Clojure focuses on variadic functions

* Scala has easy-to-read syntax, with easy-to-remember operator precedences, while Clojure has syntactic macros instead

* Scala uses actors for concurrency, while Clojure uses its other approaches

Perhaps we shouldn't be considering them competitors but instead look for ways to more easily use them together, utilizing the strengths of each.

We could use Clojure macros to generate Scala source, e.g.

  (defn object-main [s]
    (str "object " pgm-nm "{\r\ndef main(args: Array[String]){\r\n" s "}\r\n}\r\n"))
  (defn println [s]
    (str "println(" s ")\r\n"))
  (defn call-method [obj meth p1]
    (str "(" (str obj) ")." meth "(" (str p1) ")"))
  (object-main
    (println
      (call-method 1 "+" 2)))

We could create a whole library of such Clojure functions to generate Scala code, which would give us syntactic macros for the Scala ecosystem.

And when Scala AST macros are mature, we could use them to generate Clojure code.

Using Scala and Clojure together would bring the best of both worlds to the JVM. Wouldn't that just be the grooviest?!

Your macro argument is a terrible one, you can do exactly the same thing with any sane language (and indeed I have done, getting python to create boilerplate java)

You dont need macros for this (which your post shows since I cant see any defmacro used there)

The first half I wrote (i.e. the 4 comparisons) was really the main point of my comment, rather than the "get started if you're eager" example code.

It should be made clear in the title that Haskell isn't leaving MS, rather it is one of the lead developers of GHC, Simon Marlow.

Agreed. It should be "GHC Developer Leaving Microsoft".

It's the subject of the mailing list message (the [Haskell] being the mailing list subject prefix). I was able to piece together what it meant, but I know that a couple of key people for ghc work for Microsoft Research. I agree in general that it's confusing, but it's not like this is the subject of blog post that was poorly chosen. The 'real' subject is just "Leaving Microsoft".

And how would we know that before we click on the link bait?

Complain to the policy or machinery against editorializing titles that makes every submission title converge towards the <title> on the linked page.

"GHC Developer Simon Marlow Leaving Microsoft"

Wow, talk about a shock. I hope that this is good news for functional programming and the programming language community, rather than just good news for Facebook.

Does this mean that Facebook is using Haskell more these days?

Nah, it means that Simon Marlow harbored a latent desire to go code in PHP.

Facebook is about equally PHP and C++ these days.

This really boggles my mind. Facebook's stack is intriguing.

Or that they will be, or that FB wants to create its own language ala Google.

I think they might be. Bryan O'Sullivan has been working for them for some time now.

From what I can tell, Bryan mostly works in Python and C at Facebook.

He recently gave a Haskell course there, I think. Bob Ippolito was talking about that.

They've been doing that for a while. One of the major projects was/is HipHop, a PHP to C++ compiler.

Hey, I'm an engineer on the HipHop team, and a minor-league Haskell enthusiast. A couple clarifications:

1. HipHop is itself written in C++. While this did not logically need to be the case, it is the case.

2. These days our JIT compiler, the HipHop VM (hhvm) is out-performing the PHP-to-C++ ahead-of-time compiler. HHVM is under the HipHop umbrella, and so is also written in C++. If you're vaguely aware that Facebook has some sort of high-performance PHP execution engine, but have avoided investigation because compiling PHP sounds like a drag, check hhvm out; it is fast, and you use it the same way you use a PHP interpreter.

3. HipHop is not our entire PHP stack, however, and functional languages have a significant beachhead in static analysis. pfff (https://github.com/facebook/pfff) is a sophisticated multi-language analysis and transformation framework written in OCaml. We use it for many, many purposes; e.g., at code review submission time, pfff symbolically executes the changed code to look for various dangerous idioms and highlight them as areas for the reviewer to focus attention.

4. There are other projects written in Haskell that, alas, we're not ready to talk intelligently about publicly. Watch this space, however.

Overall, FP has a sizable presence at Facebook. Haskell's presence is a bit smaller than OCaml, though that might be changing. Facebook is definitely "using Haskell more these days," especially since Bryan O'Sullivan (author of "Real World Haskell") came aboard. I expect Simon Marlow will continue to influence things in that direction. From 100k feet, Facebook's adoption of Thrift, and Thrift's tradition of supporting polyglot languages, means that a lot of "niche" languages can flourish within the larger production ecosystem.

Didn't know about HHVM. Clearly Facebook deserves more credit for all of these gems.

This may be either disastrous or magnificent. I think it may end up being the latter.

This email was sent yesterday, how come it's in Courier New?

GHC? Great Haskell Collider?

Glasgow Haskell Compiler.

http://lmgtfy.com/?q=ghc

Glorious Haskell Compiler, actually.

Or, if you look at ghc --version, it's "The Glorious Glasgow Haskell Compilation System".

Well the strong type system ensures that no collisions are possible. GHC is quite the software gem. I love haskell more and more. Everything that is so simple to write, has no right of being so fast.

I agree. The type inference is a beautiful thing. I had started a project in Scala and I'm finding it hard to stay interested when Haskell is so much more interesting to me. But Scala is a LOT more practical for my project. :(

i dunno i've seen type inference create horrible to debug situations - although mainly where it has been shoehorned into to C++11 and used with templates and auto to create code that fails silently doing nothing when a parameter has the wrong type

C++ "type inference" has nothing to do with actual type inference. Types in functional programming languages have a proper academic mathematical foundation, specifically in formal logic. This is in sharp contrast to C++, where it was added as a convenience to avoid specifying certain template parameters. And with so many C++ "features", this one adds yet more ambiguity to the language.

Care to elaborate a bit on that? Why is Haskell much more interesting?

I just meant it is more interesting to me right now. Scala is really interesting too, but I've been working with it longer. So its just relative to my current interests. I like being challenged to take a purely functional approach to problem solving.

One thing I like better about Haskell which is not time-bound to my own journey is that the language is so much cleaner and more concise. In functional programming it is idiomatic to compose lots of small functions that may be only a single (short) expression each. In Scala this may mean 20-40% of the characters in a line are simply type specifications for your function arguments. I find that oppressive.

One man's concision is another man's line noise:

http://www.reddit.com/r/haskell/comments/13l9vj/getting_star...

i'm assuming this guy has done real world work before? the idea of a compiler/language lead programmer effectively having lived in academia is terrifying on many levels.

best of luck to him at any rate. haskell and ghc are a cool language and compiler combo, one i have enjoyed for recreational programming time and time again.

"The idea of a compiler/language lead programmer effectively having lived in academia is terrifying on many levels."

The apparent cluelessness level of this statement is so high, I am unable to decide if this is just ignorance or very sophisticated trolling.

If the compiler dev working in a university or a research lab while developing the compiler is "terrifying", you should avoid C, C++, and Python, not to mention smalltalk, Pascal, Prolog, Common Lisp, Scheme, SML and OCaml ....

There is always PHP. Turing Completeness FTW.

It's tempting to down vote him, but at the same time this is a real sentiment that some people seem to have. Perhaps it should be grouped into the current anti-intellectual trend. Either way, it's a real problem.

>It's tempting to down vote him

Why? Ignoring the ignorance does no good. Trend you've mentioned is nothing new, sadly, but if you are on the weaker side (the one that intellectuals always tend to find themselves on), the way to win is not to try to down vote clueless guys into the light-grey invisibility, imo, but to try to explain to them what's wrong with their view of the world.

What is so ignorant about the statement? Enlighten me...

plinkplonk more or less said it all. PHP and Perl being the notable exceptions the vast majority of widely used langauges were developed by academics.

> i'm assuming this guy has done real world work before?

Yes, very much. He's one of the leading voices in the parallel Haskell world and a very accomplished programmer:

* http://hackage.haskell.org/package/stm

* http://hackage.haskell.org/package/async

* http://hackage.haskell.org/package/happy

The real world is crappy, and we're depending on these academic types to free us from the drollness of it, because we don't all have time to invent an STM. If I wanted to write CRUD apps all day and say that's real-world, I suppose I could, but I would rather enjoy enlightenment.

Function invocation overhead, you say? Why would I want that?

How about Garbage Collection? It'll never work.

It's their responsibility to create foundational research, and it's ours to filter through it and prove the good bits with fire.

'Practical' is a red-herring, I think. We have no way to predict the future, but we all care about progress.

err... do you know that (since 1.4 IIRC) the java compiler is the one written by Martin Odersky?

Wasn't 1.5?

Martin's work was related to generics, which got introduced in 1.5.

It was 1.3, but the interesting features were disabled in production builds until 1.5, http://docs.oracle.com/javase/specs/jls/se7/html/jls-0-prefa...

Thanks for the clarification.

…and Philip Wadler principle designer of the Haskell programming language.

http://en.wikipedia.org/wiki/Generic_Java_(programming_langu...

Reply on news.ycombinator.com