Hacker News Reader

Comment by michaelochurch | Hacker News Reader

Functional programming is the most important CS topic if you want to become a great (1.5+) engineer. No, you aren't going to be doing pure functional programming very often. In the real world, eliminating mutable state is neither possible nor desirable. It's extremely worthwhile to learn how to manage state, though. Hell, even the Unix philosophy (do one thing, do it well) was directly inspired by functional programming, though it couldn't be purely functional (infeasible at the time, plus semantic mutability).

TL;DR: If you don't know what "flatMap that shit" means, then it's worthwhile to learn it.

View on HN

As a strong Haskell programmer I'm curious about the statement that "eliminating mutable state is neither possible nor desirable". My first response is to say that I find it both possible and desirable. My second is to note that "elimination of mutable state" isn't really the goal.

The real goal is to give mutable state a very composable, repeatable, clear, and restricted semantics. This is what occurs in the State monad in Haskell, for instance. You retain mutability (observable w.r.t. the stored state over applications) but can slice up the mutation and the intermediate, immutable values and recompose them in various forms with impunity.

That kind of state model is incredible to learn. Monads might be scary because they're unfamiliar, but in a functional context they can do some beautiful things.

The State Monad isn't really mutating state though, it's really a monad for composition of incremental operations upon immutable data structures. The ST monad is for 'hiding' mutable variables and ensuring their state doesn't violate referential transparency.

From inside the State monad though, it "looks just like" (has semantics of) mutability. It's simply interpreted in an immutable lambda calculus.

The ST monad has the same game but a different interpreter.

It depends on your tools and your performance requirements. I guess I should say that "eliminating mutable state from your repertoire is neither possible or desirable". There are certainly many problems where you can.

I agree strongly with everything else you said.

Oh! Agreed completely now—there are times and algorithms that are definitely best done mutably. (Another plug for the ST monad, I suppose, since it can make it clear when you're doing such a thing.)

You should know it, but don't get all ideological and neglect your advanced object or imperative programming training. You might even want to look at gasp other paradigms like rule based coding to broaden your toolbox a bit.

tl;dr scala is the best object-oriented imperative programming language I've ever used.

This is something I think Martin Odersky did really well in this class... he spent a decent amount of time discussing tradeoffs between FP and OOP and how one might arrive at choosing the different features of Scala, even going so far as to cover the "expression problem" in Scala. I think it was a superbly tasteful presentation of Scala and FP.

I prefer the pure OO trait-based solution to the expression problem that isn't supposed to exist; pattern matching is not the only way to do it. Scala is just that good.

I think that Clojure might change your mind about that.

About which? Caring about imperative/OO paradigms, or scala being a good imperative language?

Clojure is, at least culturally based on what I've seen, much closer to purely functional than Scala.

Both are great languages, IMO.

Right. Scala is functional like C# is; yes, the functional abstractions are more powerful than C#, but like C#, its not forced down your throat. I'm still waiting for people to discover how Scala is the best OO language we have so far, traits are incredibly powerful and case matching on top of that is just icing.

Heh. When I first checked out Scala, what I liked about it was that it dealt with a lot of the stuff that annoyed me when going from Smalltalk to Java :)

Yes, Clojure is a Lisp, and looks like one.

Scala can look like Java or it can look like ML, depending on the author's style. It's very multi-paradigm.

Sure, being ideological and closed-minded is never a good thing.

But one of the takeaways is that lambda expressions/anonymous functions, first-class functions, and function closures, are very powerful tools that should be provided in any modern, high-level programming language.

The fact that they're working on adding these features to Java 1.8 is testament.

Would be nice if they could also add polymorphic function/method declarations and Hindley–Milner type inference, but oh well, you have to start somewhere...

This is very well said. I kind of think of it like learning calculus - if you aren't an engineer of very particular types you likely aren't using it directly, but the thinking patterns influence your analytical thinking tremendously.

What does "1.5+" mean in the context of being a great engineer?

michaelochurch, like a few others, has his own meme on HN - he has a well received blog post describing it in detail, (http://michaelochurch.wordpress.com/2012/01/26/the-trajector...)

Essentially he rates programmers on (IIRR) a 0-3 scale of impact - so a 1.0 developer neither adds nor subtracts to the overall quality of the code base, and a 3.0 invents C and Unix.

This quote is something we should all measure ourselves against - honestly and painfully at times.

  1.4 to 1.6: Solid adder: well above-average (top 10%) by 
  software-industry standards, but average (at 5-10 years’ 
  experience) in the context of the best companies (e.g. 
  elite startups, Google, and research divisions of IBM). 
  Can be trusted to independently solve most problems in an 
  elegant and maintainable way. Engineer can make reasonable 
  time estimates and communicate in both business and 
  technical terms with management as well as other 
  engineers. Can handle small levels of deadline/delivery 
  responsibility and should be encouraged to participate in 
  architectural decisions (“multiplier” concerns). Technical 
  leadership non-urgent projects is a serious possibility.

That's average base level in my book. And sometimes I mess up on those too. (Communication being a big bug bear for me)

Thanks very much for the information; I had completely missed that particular meme until now.

I'm a PHP developer who often uses w3schools as reference. What score do I get?

I don't know.

You are on HN so it's a good guess its productive. You know enough to know PhP and w3schools indicates there is more to learn and so I would guess you are on the right path

As for everything else - I don't know till I read your code

No idea, but there are certain semi-discrete barriers that I've noticed:

0.8: difficulty with fundamentals. There are people out there who just can't get code to compile, or struggle with core concepts like pointers and recursion.

1.2: corporate work frontier. Large "commodity programmer" teams tend to focus more on creating a home (slight, reliable, adder status) for a large number of mediocre players than on developing talent. Most jobs won't take you far beyond 1.1-1.2; even the team leads in most corporate shops tend to be only 1.3-1.4.

1.8: escalating difficulty of getting appropriate projects. You can get to 1.5, 1.6 through side projects, book reading, and exploration in your spare hours. It's quite rare, however, that people get to 2.0 without having the (very uncommon) opportunity to do 2.0+ level work full-time. To push beyond 1.8, you need to get more autonomy in your job than most programmers ever have.

What about things like data structures, general understanding of programming language semantics, basic problem decomposition for data hiding and abstraction, ... ?

> No, you aren't going to be doing pure functional programming very often.

Some people claim that one is often able to make programs with a mostly functional core; in which case most of the code is purely functional. Do you not think that is the case?

Some people claim that one is often able to make programs with a mostly functional core; in which case most of the code is purely functional.

Actually, I think that is the best way to do most things. Mutable state should be documented and sequestered.

Perhaps what I consider "pure functional programming" (i.e. a project with all-functional code, except perhaps an I/O layer) is unreasonably purist.

>No, you aren't going to be doing pure functional programming very often

I do every single day.

>In the real world, eliminating mutable state is neither possible nor desirable.

Eliminating state is not what functional programming is about.

>If you don't know what "flatMap that shit" means, then it's worthwhile to learn it.

Scala is a pretty terrible language to be learning it though. Subtyping makes things far more complex and difficult to learn, just for the benefit of java compatibility (which doesn't matter if you are just learning for the sake of learning).

I have a question if you care to answer. I was watching "Overview and Introduction to Lisp" (link below.) I was struck by the idea of a function taking a function as input and transforming it to another function as output. My only experience with something similar is JavaScript, but the idea of a function doing function transforms is new to me. Is this the fundamental thing to understand about Functional Programming?

http://ocw.mit.edu/courses/electrical-engineering-and-comput...

I don't know if I would say it is the fundamental thing, but it is certainly an important fundamental thing. I think if I had to pick a single most important fundamental thing I would say it is composability. That is the underlying reason behind things like monads and lenses for example.

Thanks. I'm liking what I'm hearing. I hear functional programmers say state is evil, but it seems like a necessary evil. What I agree with more -- and they say this a lot it seems -- is that complexity is evil. I like the focus on building something large from smaller, easily understood parts.

State isn't evil, having everything implicitly being able to mutate state is evil. If you keep mutable state, side-effects, etc locked up in their little boxes, then you limit how much they can cause problems.

"I was struck by the idea of a function taking a function as input and transforming it to another function as output."

That's a higher-order function (HoF). I'd say it's not the fundamental thing to understand about FP: you can even program in a functional way without using them.

And also you can have languages which aren't putting the emphasis on FP at all and yet who have higher-order functions: elisp, for example (anyone calling elisp functional is automatically setq'ed to crazy ; )

IMHO Lisp dialects aren't inherently functionals. Some are, some aren't. Even Clojure, which definitely puts the emphasis on FP, isn't purely functional, far from it. But you can use it in a functional way (and Java too, in a way -- I used to do "OO over immutable objects" and use "functional Java").

However you'll probably find that when using Lisps in a functional way, you can use higher-order function often. For example if you have, say, a referentially transparent function and want to add memoization for free then it's very convenient to simply (memoize myfunc).

Or if you want to "carry the state" of your program through monads (or to use other kind of monads): then you'll need higher-order functions.

To me if there is one thing about FP it is simply idempotent / referentially transparent functions which make it easier to reason about the program.

I'm simply beginning to "grok" FP, HoFs and Clojure so take this with a grain of salt : )

>you can even program in a functional way without using them.

For a very tiny limited subset of programs. Anything beyond the most trivial of toy apps will use them.

>And also you can have languages which aren't putting the emphasis on FP at all and yet who have higher-order functions

Where their usefulness is seriously hampered by the language not being functional. This is the primary reason people think HoFs are no big deal. They've never gotten to actually use them in an environment designed to make good use of them.

Subtyping isn't just for the benefit of Java compatibility. What makes you think that?

Such arguments are in general hand-wavy and born out of folklore or personal biases and I hate to appeal to the popularity argument, but in the presence of overwhelming popularity of languages that do have type-systems with subtyping, you do need at least some sort of proof that subtyping is bad in one way or another. Even GHC has extensions for subtyping, without being burdened with Java compatibility.

And what are you comparing Scala with exactly?

If you're comparing it with Clojure or other LISPs, well that's like comparing apples to oranges, because there's a world of difference between techniques used in static languages like Haskell/ML and Scala on one hand and LISPs on the other. "flatMap" in particular is not something that you see often in Clojure. You don't quite see many instances of monadic types in languages like Clojure either. Scala's Option[T] or Haskell's Maybe are impractical in Clojure so you'll never see it in Clojure unless you're looking at code that was written by a Haskell developer. Etc...

Compared to Haskell, Scala does have some disadvantages, however even if you're learning it for learning's sake, it's easier to get started in Scala because of the ecosystem ... you've got everything, including capable IDEs like IntelliJ IDEA and people that complain about SBT never tried Cabal, plus with the JVM you can always use Maven or just plain Jars manually copied in a directory.

>Subtyping isn't just for the benefit of Java compatibility. What makes you think that?

No objects = no subtyping.

>you do need at least some sort of proof that subtyping is bad in one way or another.

Why do I need proof of something I didn't say?

>Even GHC has extensions for subtyping,

What?

>however even if you're learning it for learning's sake, it's easier to get started in Scala because of the ecosystem

The ecosystem matters the least when learning for learning sake. The complexity and difficulty of the language matter, and scala is a poor choice for that reason. You can't just learn FP with scala, you also have to learn all the hairy OO/FP integration and the ensuing problems around it.

> No objects = no subtyping.

You are confusing OOP with subtyping.

On my remark about Haskell/GHC with extensions, I'm talking about things like the Dynamic type, existential types, HList and about experiments like OOHaskell.

> The ecosystem matters the least when learning for learning sake.

Actually it matters the most, as good learning can only happen if you go forth and experiment on your own. That's why some languages like Python are popular, because as soon as you read a couple of chapters from a book, it's usually easy to let your imagination loose and build something. In comparison to learning math where proofs are like puzzles, in software engineering the real joy comes from building stuff.

I get plenty of joy out of solving Project Euler problems.

> On my remark about Haskell/GHC with extensions, I'm talking about things like the Dynamic type, existential types, HList and about experiments like OOHaskell.

The point of subtyping is implicit coercions: using a type T1 when T2 is expected, without extra conversion. Dynamic and existentials are not subtypes (you need fromDyn or the data constructor introducing the existential). OOHaskell is as far as I know a failed and abandoned experiment. Who uses it? I don't know HList enough to comment, but it's based on GHC's constraint system and very much unlike subtyping in Scala. I agree with you that OOP is not the same as subtyping.

Here's how variance complicates stuff in Scala: http://www.reddit.com/r/haskell/comments/zxcks/haskell_vs_f_...

>> The ecosystem matters the least when learning for learning sake.

> Actually it matters the most, as good learning can only happen if you go forth and experiment on your own. That's why some languages like Python are popular [...]

No, the comparison with Python is not fair. Python is a polar opposite of Haskell.

People learn Python as a scripting language. You can learn the basics of the language extremely fast. There's little didactic value in Python the language once you know imperative programming. Then real work comes by learning libraries, frameworks, APIs etc.

People learn Haskell as a way to broaden horizons, see a different paradigm. GHC, with standard library only (covering a lot less than Python's batteries) gives enough material to experiment for months if not years. It takes a while to understand GADTs, type classes, kinds, higher-order types, type families etc.

Learning Haskell is much closer to learning proofs/doing puzzles than software engineering. The fact that you made your program compile after two hours is rewarding like solving a puzzle.

I have seen people attempting to build a concurrent web server in Haskell in first week, not knowing how laziness or class instances work and blaming Haskell for making things weird and unintuitive. The practical mindset of Python does not pay off in Haskell. So I agree with OP - the ecosystem is least concern when learning FP for learning sake.

>You are confusing OOP with subtyping.

No, I am not. OOP can certainly exist without subtyping, but scala has subtyping for objects. Haskell does not have subtyping.

>I'm talking about things like the Dynamic type, existential types, HList

None of those add subtyping. What would even be subtyped?

>experiments like OOHaskell

An abandoned experiment that is not a GHC extension.

>Actually it matters the most

You are free to think that, but nothing you said after that support that notion at all. We're talking about learning functional programming for the sake of learning functional programming. Haskell is better for that than scala is. Ecosystem is entirely irrelevant.

> No objects = no subtyping.

That statement is patently false.

Only if you try to interpret it out of context. The only subtyping in scala is on objects. Without objects, scala would not have subtyping.

"Scala is a pretty terrible language to be learning it though. Subtyping makes things far more complex and difficult to learn, just for the benefit of java compatibility (which doesn't matter if you are just learning for the sake of learning)."

Functional and OO programming are complementary and is the point of Scala. The class explains their relationship.

The class is called "functional programming principles". As I said, if you want to learn functional programming principles, scala is a poor language to do so, as you can not focus on that subject and have to learn a ton of extra stuff as well.

"Scala is a pretty terrible language to be learning it though."

Pretty good course though. If you want something like that, maybe because you find it helpful to have exercises with deadlines and things, then I'd reccomend the course. But if you have some alternatives you wanna recommend, that'd be awesome.

For Haskell, the only similar thing I know of is the Channel 9 stuff with Erik Meijer. And that's cool stuff. But you're much more on your own there than in a Coursera course. So if you want that kind of thing it might not be a good match.

Even Benjamin "SML, SML uber alles" Pierce spends a portion of his Types and Programming Languages text contrasting objects with existentially-typed abstract data types, and finding that, IIRC, objects are often the more practical and convenient approach.

Reply on news.ycombinator.com