Hacker News Reader

Comment by jiggy2011 | Hacker News Reader

jiggy2011

All the languages he suggests apart from Scala are or are pretty close to being pure functional languages.

I'm not convinced that I would be more productive developing say a web app , database app or game using say Haskell rather than Java or Python especially since a large amount of programming is all about state and side effects.

I'm not against learning a pure functional language as a way to improve your programming especially for concurrency or to solve specific problems.

Are there any large pieces of software outside of academia that are written primarily in functional languages?

View on HN

"I'm not convinced that I would be more productive developing say a web app , database app or game using say Haskell rather than Java or Python especially since a large amount of programming is all about state and side effects."

Haskell isn't about avoiding side effects. It's about managing them.

Seriously, it's time to drop this criticism. Obviously Haskell can "do things". It may not have thousands and thousands of apps of every type, but when it has an X window manager, a DVCS, and a few web frameworks under its belt, it clearly isn't about "not having side effects". (Nor is that a complete list; my point is that is sufficient evidence to show that things can be done, not a complete app list.)

Every functional language you've ever heard of has reasonably serious apps, but I'm not sure there's one yet where you won't encounter at least one of 1. a tooling problem (Erlang, Haskell) or 2. an impedance mismatch between the tooling and the language (Scala, Clojure). Time is correcting this problem rapidly, but if you don't feel like being on the cutting edge (and that's a fine position to take), I wouldn't push them on you. (But I would say that all four of those languages I mentioned are indeed the cutting edge and have come off the bleeding edge. Real work can be done in them if you stick to their strengths, but you may hit some bad error messages, or road blocks you'd normally just download your way around.)

    "...especially since a large amount of programming is all
    about state and side effects."

Haskell provides lots of tools for reasoning about and managing side effects; it just doesn't (and, indeed, can't) allow them arbitrarily. In terms of interacting with the outside world—e.g. doing GUI work or interacting with a database—you usually have an effectful 'core' that manages side effects while a slew of pure helper functions operate on the data derived from those side effects. (This is good style in non-functional languages, as well, as it is a kind of separation of concerns.) Other functional languages are far less stringent about side effects, so both the ML family and Scheme allow state modification and side effects, although the functional style is preferred when possible.

In terms of writing algorithms, you'd be amazed at how little needs to be expressed in terms of manipulation of state. It feels natural to write stateful functions if you've spent large amounts of time programming in stateful languages, but that has more to do with your familiarity than a property inherent in the problem. If you were immersed in Scheme, you might say, "...a large amount of programming is about function definition and application", just like if you were immersed in Forth, you might say, "...a large amount of programming is about stack manipulation." You certainly could use e.g. Scheme as language for general-purpose programming, and I suspect the problems that would arise would not be problems with functional programming, but rather an unfortunate lack of modern libraries for things like Unicode handling.

I agree that you can vastly reduce state is a large number of cases. It just seems strange to write in a purely functional language when there are many languages out there that combine functional style with the ability to write in an imperative way.

I can quite happily write programs in python without any real side effects, indeed as you suggest I usually try and divide my functions between those which have side effects and those which don't. So what advantage would I get for picking haskell instead? Other than marginally shorter programs?

FP seems to be picking up steam recently, is this just because we are doing more computation in the cloud and CPUs are getting more cores rather than more raw clock speed or am I missing something?

The advantage is that you can reason about the correctness of code. Sure, if you write an entire code base depending on its size you may be able to reason about all the side effects in your head. But when moving to a larger project, or one developed by a team - all bets are off. It is really important to have the effects embedded in the type system. Even with a large test suite.

On top of this, having professionally written C/C++ for years in the embedded systems space I can say that Monads for IO is not awkward. In fact, it feels far more powerful because you are not limited to the awful semicolon operator for chaining effects.

Lastly FP != slower (even on single core). I agree Haskell can be hard to reason about performance wise at times due to choosing lazy by default. However, have you ever heard of Sisal? Many strict by default FP languages are quite fast, however they sacrifice purity. But to be honest, with experience Haskell can be damn fast also.

Interesting, my question is more though why now? Since LISP has been around for a very long time and FP has had very little mainstream interest until the last year or 2.

Good question. I think its a couple of diffrent things.

1. Java is not really evolving, people are frustrated

2. Multicore is difficulte and FP claim they are better at it.

3. Complexety of ever bigger codebases

4. Java Script not longer the worst language in the world, people start to understand whats powerful about it and want to know what more is possible.

Additionally, I think that Haskell really has to offer something over most other FP languages. Many features of dynamic impure FP languages have already been ported to languages such as Ruby or Python. What Haskell provides over, say Lisp, is exceptionally strong typing and isolation of side-effects. This is a quality that cannot be ported to mainstream languages easily.

So, why didn't Haskell become more popular earlier? I think that's simple: the GHC compilers has improved tremendously over the past years, Hackage has grown considerably, and there are now more books oriented at practical programming.

After spending some time exploring different functional languages, if FP does make it at all - my money would be on Haskell. There is something to be said for focused languages that does one thing but does it really well, and the feeling of security when a certain style is enforced over a codebase.

I probably wouldn't use it for anything today, but in another couple years maybe. The ecosystem seems like it's reaching critical mass.

I think Clojure is ready right now. For "standalone" languages haskell seams to be on a good way, some people are allready doing start ups with it.

Actually most Lisps offer optional means of static verification and static typing for when its necessary. Furthermore, Lisps have vital metaprogramming features that Haskell lacks which are essential for symbolic AI applications.

5. Code-hipsters on HN ;-)

A nice way of putting it would be "FP became better at marketing"

Galois is a company which uses functional languages, mainly Haskell if I'm not mistaken: http://corp.galois.com/

If I'm not mistaken, Jane Street is another functional programming shop: http://www.janestreet.com/

These are companies using OCaml: http://mirror.ocamlcore.org/wiki.cocan.org/companies.html

I also know of at least a couple of large investment banks that have significant Haskell projects in house for doing things like options modelling.

The other day i ran a blog about haskell at Standard Chartered (written in, um, Hungarian?) thru google translate. Very entertaining! But SC is probably becoming a substantial codebase

I think you may argue that some languages are in practice more or less pure by how much they encourage/enable avoiding mutability and side effects but of the 4/5 languages he discusses I'm pretty sure only haskell is purely functional. Scala Scheme and SML/ocaml are all impure functional that allow mutability/side effects. (Scala is considered an object functional hybrid by some and an object oriented language dressed as functional by others - try to avoid flamewar).

Some Examples: I think Twitter uses Scala(see above) for a number of their back-end services (see http://blog.redfin.com/devblog/2010/05/how_and_why_twitter_u...). Foursquare uses scala at least on the front end http://news.ycombinator.com/item?id=1139062 . This page has a few other big success http://www.scala-lang.org/node/1658.

Haskell has pandoc(http://johnmacfarlane.net/pandoc/) a cmdline tool to convert beetween markup formats(also a library with binding to several languages) I think it supports a wider range of formats/fidelity then alternatives.

Ocaml: MLDonkey is an open source eDonkey2000 client Coq – a interactive theorem prover.

Ocaml

I've written public web applications in Ur/Web (a functional language for web development, http://www.impredicative.com/ur/) and the backend for one in ATS (http://www.ats-lang.org). I've found them both pretty productive.

One company that's making heavy use of F# is Credit Suisse. They use it in their global modeling and analysis group.

The F# based WebSharper web framework: http://websharper.com/. WebSharper is based on the work of Philip Wadler, et al: http://groups.inf.ed.ac.uk/links/formlets/

Here's another article by Philip Wadler on Formlets: http://homepages.inf.ed.ac.uk/wadler/papers/formlets-aplas/f...

Of course Microsoft uses F# for Bing data processing.

> Are there any large pieces of software outside of academia that are written primarily in functional languages?

If you're asking about places where FP is often used, I know it's big in finance (though I have no idea what those guys are up to).

The haskell web development space has also gotten to a pretty mature point. I think you'd have no problem building a web app with Yesod or Snap.

http://www.yesodweb.com/ http://snapframework.com/

You could also check out http://cufp.org

Jane Street is pretty well known for their use of OCaml in finance.

http://janestreet.com/technology/articles.php

> If you're asking about places where FP is often used, I know it's big in finance

I've heard this a few times (mostly from the Scala crowd) but very little data to back it up, so I'm still skeptical.

I wouldn't say FP is big in finance. On the other hand, financial companies are probably over-represented among the industrial users of FP.

There are several prop trading firms (Jane Street, Tsuru Capital, etc) that use functional languages heavily. Also, many of the big banks (Credit Suisse, Deutsche Bank, Barclays, etc) use some FP in production, but as far as I know it's only used by small, strategic groups. FWIW I've interviewed or applied at many of those big banks in Tokyo and not one of them ever mentioned interest in FP skills. As far as I could gather the typical developer there doesn't know or is not interested in FP.

I don't know how far over to the functional paradigm arc leans but hacker news is programed in it. I know it's built off common lisp so there is that linage.

The naughty dog game studio used a form of lisp in their games starting with Crash Bandicoot until they got purchased by Sony(I think). Though, the language was much more imperative in nature instead of functional.

Autocad had autolisp built into it.

http://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp

I don't know of any others off the top of my head.

I wonder if the lack of state is why I keep getting the "Unknown or expired link" message all the time?

I see allot of functional DSLs (Excel for example + autocad as you pointed out).

Was crash bandicoot written in Lisp or was this just an internal DSL used for certain operations?

I don't have anything against functional languages and I'm sure all good developers should know at least 1 but the article seems to suggest that everyone should use them all the time.

There are more important things than having an expressive language, I use allot of Java which is seriously non expressive in many ways but I have more confidence in it to solve the category of problems I deal with than any other language that I have tried thus far.

I appreciate your interesting comments, but just FYI, it's "a lot" not "allot". I wouldn't normally point out a spelling error but I noticed it enough to be distracting and thought you might not know (in case of ESL).

Naughty Dog used an internally developed programming language for the whole game (or at least most of it.) A big part of the appeal was apparently the dynamic nature of Lisps, which allowed reloading of code at runtime to simplify debugging. This article is a postmortem about Jak and Daxter; the next page also mentions the drawbacks of using GOAL, as well:

http://www.gamasutra.com/view/feature/2985/postmortem_naught...

You get that message fairly often because the implementation uses continuations (http://en.wikipedia.org/wiki/Continuation). This is one functional style for making a stateless protocol (HTTP) into a stateful one. HN could use a different design (one with less state) and avoid these issues with a few different trade-offs. This is an implementation detail rather than something specific to FP vs. non-FP. Also, best I can tell, HN still stores all of its data in flat files rather than a traditional datastore.

> I wonder if the lack of state is why I keep getting the "Unknown or expired link" message all the time?

As front page keeps changing and reordering, the "Next page" link after some time becomes meaningless, so maybe it's because of that.

> I don't know how far over to the functional paradigm arc leans but hacker news is programed in it.

HN doesn't look as an outstanding piece of software.

Can you qualify that statement a bit? There's more to HN than meets the eye.

The dreaded "Unknown or expired link" error if you flip through a few pages. Thanks heaven there is hckrnews.com.

Merging discussions is not allowed.

No checking of duplicate links, if I'm not mistaken.

Non foldable threads.

No way to watch threads of comments.

No notifications when someone answers a comment of yours.

After you have submitted a comment, it just reloads the page instead of taking you right were your fresh comment is, to allow you to continue reading where you left.

Item lists in comments are not handled.

How do you format text in comments? Nobody tells you.

Basically, if you take HN at face value, it does not look like a smart piece of software. Maybe under the hood there is a mind-bending piece of AI, but as long as it doesn't care about users, users will never know about it.

The thing was written to give a linear history(if I'm not mistaken) so that when you click next page it's what the next page would have given you at the time you first loaded the page. He could just do a page=4 query string and not use continuations and make the pages the same for everyone with the risk of seeing duplicates.

It will vote for the original if the urls are the same when you submit it. This is a simple check though.

Foldable threads are client side.

I wish it had notifications.

The other items re formatting could be easily fixed on the submission page.

for the record arc is built on scheme.

You are right I remembered incorrectly.

since a large amount of programming is all about state and side effects

IANA web developer, but...

Doesn't web development typically put state into a database system instead of the scripts the server runs to generate the requested pages? It seems reasonable enough to have a function that takes a page request as input, calls some other functions to look up info from the database, then constructs and returns a page and a sequence of changes to make to the database.

You can do allot with a database sure but sometimes you want multiple transactions within a single request and data will change between them. You may also take input from other sources than a database such as flat files, web APIs emails etc and need to do a bunch of processing between them.

Not to mention if you are using Comet or something where you hold a request open for a long time and push and pull allot of data some of which may not be persisted immediately.

Sure you can probably write your controller logic in a pretty functional way but then you probably end up moving your messy logic into stored procs or something else.

sometimes you want multiple transactions within a single request and data will change between them

Is this an issue with race conditions, or with alternating between reading and writing the state during a single request, or both, or something else entirely?

You may also take input from other sources than a database

This doesn't seem like it should make a difference: you write a function to get the necessary data from whatever source, then call that function in the page request handler (OK, yes, I/O is technically a side effect).

I agree that the processing you do with that data might require some significant rethinking, depending on how it was written/planned out before (sure, state-passing style is always there, though I don't enjoy doing it... really though, I've found that a lot of what I was used to doing imperatively is not all that awkward to do functionally).

Not to mention if you are using Comet or something where you hold a request open for a long time and push and pull allot of data some of which may not be persisted immediately.

What's the conventional way to do this in an imperative language?

Is this an issue with race conditions, or with alternating between reading and writing the state during a single request, or both, or something else entirely?

Usually reading and then writing some state followed by waiting briefly to see if another thread will change the same value, if it does use then do something with the new value and return that to the request.

Depends on how much more difficult it is , does it just require and adjustment of thinking to a new paradigm or is doing everything harder? I'm thinking of stuff like Haskell Monads here. I admit I have not much functional knowledge beyond some basic scheme so I'm curious about this.

What's the conventional way to do this in an imperative language?

I guess allocate stuff on the heap which other threads can modify, use monitors to control concurrent access.

does it just require and adjustment of thinking to a new paradigm or is doing everything harder?

Depends on how restrictive your language is. Scheme makes it a lot of things easy, e.g. a sequence of statements is often easy to express with begin/let/let* , whereas Haskell is pickier about use of side effects. The biggest hurdle I had in learning scheme was getting used to using a recursive function where I would normally use a loop.

  (let* [(x (... DB query ...))
         (y (... API query based on x ...))]
     (begin (... write some state based on x and y ...)
            (... wait a bit ...)
            (... read the same piece of state ...)
            (if (... it is unchanged ...)
                (... then ...)
                (... else ...))))

The line between imperative and functional starts to look a bit fuzzy, but let* and begin can be constructed from lambda expressions.

We use Erlang to run our (several million messages a day) WebSockets implementation. It's been fine, way more stable and less resource intensive than the equivalent Tornado server.

Haskell is actually awesome for something like Comet or just about anything where concurrency is involved.

I'm launching a small Haskell-based product called EventSource HQ (http://www.eventsourcehq.com) taking advantage of just this.

Handling state across threads in Haskell is not a problem at all. Haskell is not bad or complex when it comes to mutable state, it's just very explicit about it. When it comes to doing stuff the involves multiple threads this is a huge advantage.

Haskell is complex in the sense that it brings a whole bunch of new abstractions to the table that can take a lot of work to get a hold of. But while this can make for a steep learning curve, it also comes with a huge payoff, since these abstractions are very powerful.

Stuff like Iteratee's and Enumerators or the newer Conduits can be tough to wrap your head around, but once you do the power of composable abstractions over IO streams can be extremely useful.

>I'm not convinced that I would be more productive developing say a web app , database app or game using say Haskell rather than Java or Python

Have you tried? Why do you think your baseless assumptions are interesting?

Reply on news.ycombinator.com