Some Functional Programmers Are Arrogant
devjoy.com
devjoy.com
It's like with vegetarians. "I don't eat meat; there are plenty of plant based sources of protein and I don't want to have to have animals killed on my behalf". Nothing about this is factually incorrect, none of it is arrogant, yet, because it's purposely avoiding something the listener consumes, the listener may take offense, and assume the vegetarian is taking a holier than thou attitude toward them. They assume that because the vegetarian views eating meat as something to be avoided, that the vegetarian must be judging them for eating meat.
Similarly, because the FP programmer views mutable state as a dangerous practice, the listener who uses mutable state views the FP's attitude as judgemental, "You're not using best practices!"
But, like the vegetarian who encounters someone who wants to try giving up meat, and is asking for recipes and ideas, FP programmers will, indeed, bend over backwards for someone who is trying to enter the functional world.
Where do you think I would be able to read the essay when it's complete? Will there be a great deal of mathematics in there? (I have a poor/loose grasp of anything past high school mathematics, barring algorithms).
ps: just read this https://www.info.ucl.ac.be/~pvr/fdpefinalweb.pdf
it seem they extends the SICP approach, substitution -> state | concurrency ...
I planned to do the opposite. state -> subst
Yeah, I guess it is in the opposite direction. I was mostly thinking in terms of describing a language in terms of another via 'progressive enhancement'.
Before the degree of my ORA was known, I did play at being vegetarian for 6 months for ecological, not moral reasons. I don't see a moral difference between eating a fruit and and an egg, or salmon and a herb, milk/blood and maple syrup. The arguments that fauna are somehow more alive than fauna has never felt very strong to me. We've all been evolving and finding our niche for about the same amount of time. We've all got our own versions of response to external stimuli. Its just much harder to empathize with an apple tree than a chicken, especially when it comes to the ultimate sacrifice.
However, the ecological arguments for vegetarianism around sustainability and CO2 emission seem sound for the standard american lifestyle.
Of course that starts to flip if you hunt (I don't). Large scale agriculture does more damage to the environment than culling native mega fauna, which is arguably beneficial. It flips again, if you have your own garden, supplemented by local, organic specialists. And then you reach a much better level of reuse/yield if you can feed scraps to chickens and pigs and fertilize your ground with blood and bone meal. (it always makes me laugh when vegans think their organic vegetables are 'cruelty free').
So, no. I don't think the implications of saying you are vegetarian are immediately obvious. The more I research it, the healthier and more sustainable a diet of mostly fruit and vegetables supplemented by the meat of a couple of pigs, 6 or so chickens and the occasional hunt seems to be. But if you genuinely believe that fauna are some how better than flora I guess that simplifies the equation and somewhere down that path you get labelled a zealot.
I suspect the same is true with programming. What you want is a system of mostly functional business logic that is supplemented with objects when it makes sense (modeling the state of an electrical circuit?!?), and the odd 'declarative hunting trip' when you need to cull the date from the database :)
It is possible exist on just objects, or just functions but variety is the spice of life and the sum is often greater than the whole.
This sort of does away with problems of mutability, since you can change things, but only exactly once (which is what you intend for e.g. current).
Do you enjoy talking your fern out for a walk and the companionship you feel with it?
The latter instance of "fauna" ought to be "flora", I suspect.
Everything we do is an abstraction.
The Von Neumann architecture is all about state and mutability, and yet FP tries to stay as far as it can from that.
OOP is really a supplement layer over imperative programming and doesn't deny any of its underpinnings.
PS: Computers are not inherently digital, either.
As I understand it, real world FP isn't about "staying away from" state and mutability, its about providing tools for managing state and mutability as a means to manage the complexity of reasoning about programs using it. E.g., Haskell's monadic approach essentially makes every Haskell program one that produces (as the result of the main function) the imperative program that is actually executed by the runtime.
(I'm not sure whether this is a joke, a brilliant idea, or a piece of FP advocacy. You could probably get a CS graduate paper out of trying it, though)
Second, even if we consider PFP alone (which is, indeed, different from imperative programming), I often find that some of its proponents are lacking in understanding of the theoretical underpinnings of imperative programming. For example, day before yesterday I gave a talk at CurreyOn/ECOOP about why monads are harmful in imperative languages (i.e. any language that isn't PFP), and PFP experts present weren't aware (although, TBH, neither was I, but I'm not a PL researcher) of this proof[1] that imperative continuations are at least as powerful and easy to express as monads (I later claimed and provided evidence that they're more powerful).
There is actually much to debate about the relative merits of PFP and imperative, but I certainly wouldn't say that PFP is "avoiding badness". It's avoiding some badness while introducing other kinds of badness (monads have some serious disadvantages), while imperative programmers can avoid the same badness without resorting to PFP (e.g. as Clojure and Erlang do).
[1]: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.43.8...
Challenge accepted: FP languages emphasise restriction of mutation.
(I seriously believe that any attempt to define FP will end in tears. It's a marketing term and has no real likelihood of producing legitimate technical value.)
Functional languages are languages used predominantly by functional programmers.
Being functional probably has more to do with how a language is used than the set of features it boasts.
Can you do FP in any language? Maybe not COBOL, but you could do it in C with function pointers. But it's still easier to do it in Haskell than in C, and it's still easier to do imperative programming in C than in Haskell.
So you have a point. But languages have different characteristics, too, and those differences let us classify languages, if not as "FP" and "non-FP", at least as "more FP" and "less FP".
FP as a programming style to which a programmer may adhere to a greater or less degree is easy to define. FP as a binary is-or-is-not characteristic of programming languages is, well, a misguided concept. To the extent it makes sense to talk about FP -- or OOP, or any other style -- as a feature of a programming language, it is more in terms of the degree to, or better the particular manner in, which it facilitates that style of programming, rather than a binary descriptor.
Favoring immutability is a great property, but it is not a hard dividing line. The Java community started encouraging immutability no later than 2001, and Scheme was called functional long before anyone categorized its main features as "restricting mutation".
I'm not confident that this shows that monads are bad or harmful though. I'd like to see the larger argument.
I also wouldn't say that PFP is "avoiding badness" as much as its avoiding "everything" in name of explicitness. Over explicitness can be interpreted as a problem if you want—I'm very willing to take this argument—but I'm not sure how to paint it in broad good versus evil terms in the same way that I am with something like, I dunno, the COME FROM statement.
But there is much to be said against PFP itself (and much to be said in favor) like the lack of clear execution models, the lack of clear thread context (which is, of course, related), the difficulty in analyzing computational complexity, in debugging and profiling, and implementing many simple algorithms. Many of those are related to laziness more than referential transparency itself, but RT necessitates laziness in some cases, whether the language itself is strict or not.
And, of course, there is the big question of "wherefore referential transparency". If it is to avoid problems caused by shared mutable state, then there might be better ways, like making shared state transactional to some degree (Clojure, Erlang). If it is to make automatic verification better, then there might be easier ways (like some of the many model-checking methods), and if it is to make human reasoning better, one should ask whether equational thinking is really the best way for people to think about programs (I think the human mind is much more adept at simulating state than at math, and that an essential, "basic" abstraction such as monad transformers is already too high-order for people to wrap their heads around). Then you have all the details, like how far you can take Curry-Howard-style, type-based proofs while maintaining inference and how far you can go while keeping type errors understandable.
I think that because much of PL research in the last decade has been focused on PFP, people are under the impression that it is somehow more "mathematical" or theoretically sound (not in the logical sense) than imperative, completely forgetting that fully-verifiable, and theory-supported imperative languages (like Esterel and its descendants) have been used much more successfully in the industry than Haskell and friends.
The argument of PFP vs. imperative is important and interesting, and there are both mathematical as well as psychological arguments to be made in favor of each. However, in recent years, many have come to see imperative as pedestrian and PFP as mathematically-advanced, which is completely false.
EDIT: I fully recognize that there are also cultural differences here. PFP emphasizes abstraction while imperative emphasizes algorithms. Personally, I think algorithms are more important than abstractions, but I realize that depends on the domain.
I would be interested to see your proof of that.
I'd say that's subjective, and that it suffers from enough ailments: no stack context for post-mortem debugging and profiling. I know Haskell has this "cost center" thing, but I don't know enough about them to tell whether they are truly sufficient. Anything that harms posthoc reasoning about a program with a debugger/profiler/log can be plenty painful.
For an example see this paper:
"More haste, less speed: lazy versus eager evaluation" by Richard Bird, Geraint Jones, Oege De Moor
http://journals.cambridge.org/action/displayAbstract?aid=441...
ftp://ftp.comlab.ox.ac.uk/pub/Documents/techpapers/Geraint.Jones/FP-1-96.ps.Z
I think the primary things I'd ask about or try to answer revolve around the use and interpretation of monads and the meaning of referential transparency.
Firstly, "using monads" is perhaps better read as "being explicit about the design of the monadic context you are in". Their original use is much more about creating nice semantical interpretations of imperative programs than their practical use today in Haskell, and, in this broader context they are tremendously useful in imperative PLs.
Indeed, it's perhaps proper to say that an imperative language is one where its statements can be thought of as living in what I like to call "some, unstated ambient monad" which has a number of control flow and stateful properties. In this sense, a lot of your argument, I believe, boils down to saying that (1) people appear to have intuition for a certain monadic context and (2) designing a language to implicitly live in that context is friendly. I agree strongly with both of those.
Some remaining points include a lot around how monads are implemented/interpreted in Haskell today. That's a fine critique, but not one against the broader idea or against PFP at large.
Finally, wrt to referential transparency: this ultimately comes down to statements about what it means to have names and binding in your language. Strangely, such a primitive question spirals out into having enormous consequences. The nice thing about referential transparency is that it, more or less, is simpler than any other kind of name/reference that people often pick and therefore it's a good default state IF your goal is to make everyone opt into any more sophisticated ideas.
Again, I'm not a personal proponent of saying anything like "100% explicit is always the right answer". As noted, it's annoying, complex, and has external impact outside of mere value-semantics matching. So, in this sense I agree a lot here.
But in a broader sense, I strongly believe that it's no accident that the research behind PFP is as strong as it is. The core idea is not a CS one but instead in the free connection to logic provided by the Curry-Howard or BHK interpretations. As we design logics which are increasingly expressive and translate them to PLs we find increasingly that these PLs have properties of extreme regularity. This make them great foundational modes of which other languages should be based. I don't think those forces will be going away.
But languages are not always best suited for extreme explicitness and expressiveness. Metaphor and intuition are powerful tools that languages should take advantage of. These "metaphor langs" can be profoundly nice, have rigorous translation to foundational language, and thus share fantastic semantics. Ultimately today that's not so different from the embedded DSL story.
Right, I don't know how a non-Haskell (or Haskell-like) PFP language could look like, but I'd love to hear some ideas (PL is not my field).
> ... being explicit about the design of the monadic context you are in
Obviously you're not advocating 100% abstraction explicitness, either, but the desire for that kind of explicitness -- and this is totally subjective -- seems to me like valuing abstractions over algorithms, which I see as inverted priorities (like I said, PL is not my field :)). I think that the motivation isn't serving a human need but a mathematical need, or, rather a researcher's need (see the end of my comment).
> Indeed, it's perhaps proper to say that an imperative language is one where its statements can be thought of as living in what I like to call "some, unstated ambient monad"
I think we can be more specific. Imperative languages "live" in continuations, and those are at least as expressive as monads. I can't see how monads are more or less explicit than continuations, and why naming the monad somehow conveys intent more than naming the operation (as an example, returning a list from a monadic function in the list monad is no more explicit than calling "produce" or "yield" in a list generator) -- the only difference is whether that name is at the "top" as with the monad, or at the bottom, as with continuations. Related to this, effect systems are completely orthogonal to the question of continuations vs. monads, and restricting a subroutine's choice of what kinds of continuations it can pause on is as easy as restricting a pure-function's return value to restrict what monad it can participate in.
> The core idea is not a CS one but instead in the free connection to logic provided by the Curry-Howard or BHK interpretations... I don't think those forces will be going away.
I agree but see this as a case of searching under the lamppost. Curry-Howard makes things easy to verify (in fact, makes them trivial, as the burden is on the developer -- the language "just" provides soundness and inference). Because it's easy to work with mathematically, that's where a lot of PL research is. However, from a pragmatic perspective, this is mostly moot if the result is psychologically (or economically) incompatible with human developers' preference. No one is debating the properties of PFP. But their desirability -- or lack thereof -- is not a CS question.
I'd immediately suggest that Haskell is not a particularly pure language itself and that you should take a look at { Coq, Agda, Idris } for far more ideas about where PFP ought to go. You can also without too much difficulty imagine an ML derivative which is pure although they are historically not.
In all of these cases, laziness is not a component and many of your "the lack of clear execution models, the lack of clear thread context (which is, of course, related), the difficulty in analyzing computational complexity, in debugging and profiling, and implementing many simple algorithms" arise directly out of laziness alone.
> you're not advocating 100% abstraction explicitness
To put my cards on the table, I am advocating for this at least some of the time. :)
I'm interested in the idea of there being a tension between favoring abstractions versus favoring algorithms. I understand what you mean anecdotally, but am not sure how to put it into a larger context. In many ways I feel abstractions create the hard outlines of algorithms and are indispensable. This I feel is true even if you're, e.g., programming in Forth. The difference is not a lack of (even higher order) abstraction but merely a difference in how much of that structure you communicate to your compiler.
Re: continuations.
Having access to arbitrary continuation semantics is equally powerful to "all monads combined together" in the sense of "what you can encode" but, then, simultaneously, ultimately weak in the sense of "how many ways can you interpret this". On the other hand, restricting the set of continuations you have access to is completely identical to choosing a monad.
You say that the only difference is whether you pick things at the top or bottom. I say that this isn't even an actual difference. In both cases you're effecting the exact same thing and, if you like, it would be fair to analyze it as a monad.
Monads aren't equivalent to Haskell's encoding of them. Even Haskell admits many encodings. The first-order ones like we're used to seeing are popular because they're simple to understand and implement not because they're the only way. As an example, here's the monad stack at the core of Attoparsec, a fast parser combinator library
Parser a
~
forall r.
(Buffer, Int, Bool) ->
((Buffer, Int, Bool) -> ([String], String) -> Result r) ->
((Buffer, Int, Bool) -> a -> Result r) ->
Result r
Result r
~
Fail Text [String] String
| Partial (Text -> Result r)
| Done Text r
Re: lamppost
I agree a lot with what you're saying here. In particular, I think there are two ways to attempt to answer the (silly, but then you take it seriously) question "what is the greatest programming language possible?"
First, you can ask for power measured in "How completely does this language serve to articulate and circumscribe any thought which I might, as a human, want to have?". Following this line necessitates following logic carefully not because it's magically better but because mathematicians have spent many centuries delimiting interesting, complex ideas and at least 1 century working dedicatedly on a language for describing them. It's silly not to crib off this effort... or to think any other field is going to surpass it.
Second, you can ask for whatever language best fits the human mind with all of its quirks. This is a question of psychology, of course, and design and I would never claim that it's not incredibly important... although I do often give it short shrift.
The reason being that I have some personal chips on the idea that these two goals actually will wind up in the same place more-or-less. This isn't to claim that mathematics doesn't have massive ergonomics issues today... but instead to claim that it is in many ways an ultimate distillation of the logic all of our minds are marinating in... just as a consequence of living in this universe and partaking in this shared experience.
I feel like Mathematics took on the challenge of mapping out the entire human headspace from the inside out and psychology took on the same challenge but working from the outside in. I have to believe they'll converge someday.
And then, maybe finally, someone will write a language with good error messages.
> In both cases you're affecting the exact same thing and, if you like, it would be fair to analyze it as a monad.
Perhaps, but here's the thing -- I don't care. In fact, I am truly and utterly sorry for having taken the time to understand what monads are -- sort of (well, not really, but you get what I mean). I consider it a waste of time that could have been better spent reading another DB or concurrency paper (my current fields of work). As a veteran programmer in the industry (almost twenty years) I really don't care much about programming language concepts or verification methods (I am interested in compilers, especially Graal, which I think is a great breakthrough, but that's a completely different matter). I want to write an algorithm in a way that is straightforward, easily communicable with my colleagues and successors, be able to read and understand other's code, and be able to reason about it both before as well as after it runs. With the recent resurgence of PFP and it's (IMO, sometimes unfortunate) influence on imperative languages, I find this emphasis on expression and abstraction very distracting. When programming in a "richly typed" myself, I often find myself drawn to using clever abstractions that are necessary neither for expressing the algorithm nor for making the code more maintainable or readable, which is why I enjoy using languages with relatively few abstractions (C, Java, Go). I find expressing the algorithm much more straightforward, faster (because I don't waste time coming out with a nifty abstraction and then being pleased with myself about it), and not a bit less readable. Ever since I started using automated tests (along with the rest of the industry) I've never even found correctness to be a problem.
Which leads me to ask, what problem is PFP trying to solve? Now, I am entirely willing to believe that it's not trying to solve an acute crisis, but just to make programming "better" (which hopefully means cheaper). But if that is the case, I have been waiting for convincing evidence for the past 16 or so years -- ever since first hearing of Haskell and its imminent revolution at university -- and so far: nothing. I am not saying that PFP is necessarily a false prophet (although I still think imperative code with even crude effect systems is just as effective, and much clearer than PFP -- and no, I don't think they're the same at all even if they could be both reduced to each other: I am not a PL researcher, nor do I personally care to know much about that CS sub-discipline; from the perspective of the programmer the two are completely different), but I do see it as something of a vaporware. A lot of hype and no result. That the largest Haskell program -- after twenty years -- is the Haskell compiler is a very bad signal (and yes, it is the biggest program).
As for mathematics and the human mind, well, I think the search space is just too big on both ends. Math can express just about anything (including, probably, types of intelligence very different from the human mind), and the human mind is very flexible, a universal computer, which, in spite of its many constraints, is probably universal enough to not be meaningfully reducible to its physical components (just as software is not meaningfully reducible to the physical processes powering the hardware it runs on). So I don't really know if the human mind or math can converge to anything, other than that our mathematical exploration is guided by our human minds, so we might be subconsciously heading somewhere interesting.
> other than that our mathematical exploration is guided by our human minds
I think that's exactly it. I'm not honestly a good enough philosopher to be clear on how I feel about the origins of mathematics. So, blurrily, I'll just state immediately that I think they are an embodied thing (I am no Platonic dualist). In that sense they are not constrained by our language but instead our language is continually growing to be so strong as to admit the vagaries of our thoughts.
And this is why I like PFP: because it endeavors to capture a wider world than just algorithms—although it is one which places algorithm at its core by accepting the need to build and witness over placing trust in the completeness of the universe.
This justification is wildly impractical, of course. I can't even from this perspective recognize the smell of practical—so the closest I can come is to say that I think Church's Thesis, if it is even true, taught us something really interesting: that it's quite easy to represent all functions (nat -> nat). Since this encompasses nearly everything we can "practically" witness we're playing in a giant, easy sandbox (this is not me saying that programming is easy, of course!).
So there ends up being a lot of degrees of freedom on how we accomplish our giant, easy tasks. If you want to eliminate them then you must take aim at harder tasks---and that is exactly what PFP does do. Or at least what the research apparatus attached to PFP does. So out here is a wild ride to the core of human expression. That's one that I want to ride.
True, but 1/ with so far very little to show for it (it is easy to make easy programs easy; no one has seen PFP make hard programs easier) and 2/ I am far from convinced (although I haven't explored other avenues myself, as PL is not my field) that there aren't other approaches which do the same and are more compatible with how we think.
I think that other than the happy "coincidence" that is Curry-Howard, I see nothing in LC to suggest that it is the "right" way to model computation, and CH itself is just a form of verification (that requires programmers to prove their programs). Maybe it's the right way to do verification and maybe it isn't. Suppose Hoare logic approaches proved to be easy. Would people still see LC as promising? I think it's seen as promising just because verification is easier -- not because there is evidence to suggest it's the right way to express unverified algorithms -- but nobody has quantified the cost/benefit of various verification techniques (both formal and informal).
Other than the mathematical results which are to be expected to be found under this powerful lamppost, I see little evidence to suggest PFP is even on the right track. PFP has had one obvious achievement, though: thanks to PFP, there's a lot of research into type systems, some may prove to be very useful even in imperative languages (like Rust, although that language is also far from being proven, either, but I have a feeling we'll learn of its effectiveness -- or lack thereof -- much sooner than we will of Haskell's :)) But I find that achievement rather disappointing compared to those of the algorithmic CS disciplines (distributed databases, machine learning, JITs).
I do recognize that PL has a much harder time proving the worth of its results. A machine learning algorithm could be implemented in any language. Producing and then getting people to adopt a new language, in contrast, is extremely expensive, as it requires not only implementing the new work, but porting man-decades of work. But sympathy aside, as someone in the industry, I view PL research as that CS discipline that's supposed to concentrate most on the human side, yet contributed least to to our industry's achievements (while making more noise than most other CS disciplines).
And this is where I'd like to circle back to the original post and the discussion of whether PFP people are arrogant or not. I don't know too many PFP people with major achievements in the industry, but I don't suppose they are any more or less arrogant than others. I do, however, find it annoying (and ridiculous) to be accused by some PLers of having an anti-intellectual stance or refusal to try new things simply because I'm very skeptical of new languages. I and others like me are very quick to adopt new algorithms, but new languages are not only extremely costly to adopt -- they have so far delivered precious little more than big promises (but oh, do they love making those promises!). So if I were to describe PL researchers and PFPers in particular, I'd say they overpromise and under-deliver more than the rest of academic CS, and have certainly had the least impact on the industry. I can suggest (with little knowledge of actual facts) that the reason for that is that PL, as a UX discipline, should be most accommodating to human psychology, but in practice it is often as uncompromising as any other pure-theoretical field.
I don't think of CH as being a way of doing verification, as you mention it's not even a very good way of doing that necessarily, but instead a note that human reasoning can be accurately transcribed in types. The formal verification angle on this is just window dressing.
Also LC is not even seen as the "right" way. It's obviously too simple. But it is at the core of other more "right" ways because it as a notion is very composable.
PFP is an important part of this research because it's got the most complete type theories. Impure arrows can be modeled too, but they're harder. Nice impure models share more similarity with PFP than with standard imperative programming, too.
I suppose in my mind PFP has had tons of victories... but they're just not in the domain that you're looking in. That's fine, I believe some will exist there someday. I'm not actually much of one to proselytize for PFP in industry ;)
And on arrogance I am completely with you. When people get a little power it goes to their head and if you make a transition from "practical" imperative programming to "mathy" FP you feel like you just leveled up a lot. Then you ignore that "mathy" imperative programming (and "mathy" OO for that matter) exist and are healthy and probably have a lot to share, too. That's sad.
Finally, I agree with the idea that at least somewhere in PL there ought to be a UX discipline (see my writing elsewhere around types as UX for more on this) but I am also pretty convinced that this is not the entirety of PL research.
Types should be studied even if it is conclusively proven that humans cannot stand their UX. Why? Because they are as powerful a language for expressing formal thought as humans have yet invented. We'll figure out the UX someday, but if we step away from the seat of utility then we'll have lost something vital.
Is that human reasoning or logic? :)
> But it is at the core of other more "right" ways because it as a notion is very composable.
It is and it isn't. Where it counts, it's got some composability problems, like with composing monads. Moand transformers are way too high-order for me to follow (or care to learn to follow) -- remember, NASA said first-level and second-level pointer indirection are fine, but not third-level; I think this applies to high-order functions, too. OTOH, "scoped" continuations[1] compose perfectly, while still being statically typed and explicit about which "monad" they're in[2].
> Nice impure models share more similarity with PFP than with standard imperative programming, too.
I totally believe that, but from the programmer's perspective, that is completely irrelevant. That's something we want to stay hidden, left to those doing PL research. Give us the results -- we really hope we don't need to learn how you got there :)
> Then you ignore that "mathy" imperative programming (and "mathy" OO for that matter) exist and are healthy and probably have a lot to share, too.
Considering that 99% of algorithms are expressed imperatively and invented by imperative programmers, I'd say that's likely true. People doing DB, numerical/scientific computation, machine learning, distributed computing and concurrency research work mostly in C, Java and Matlab -- not Haskell or Idris.
> Types should be studied even if it is conclusively proven that humans cannot stand their UX. Why? Because they are as powerful a language for expressing formal thought as humans have yet invented. We'll figure out the UX someday, but if we step away from the seat of utility then we'll have lost something vital.
I totally agree!
[1]: Scoped continuations are nested continuations that allow functions to pause at each level of the scope (think how nested try-catch blocks work with different exception types) -- I'll post my talk when it's available online.
[2]: Think Java's checked exceptions.
As I stated earlier, your stacks of scoped continuations are exactly equivalent to monads but, depending on how you restrict them, a little less flexible. I don't mean this to say something trivial—I mean it to say that every composition problem solved in each place is identically solved in the other.
I'm confident that your 99% claim holds historically to various degrees, but not completely. Plenty of mathematical algorithms were (a) constructive and (b) first formulated declaratively. We have spent the last 30-50 years describing many more algorithms at a much more furious pace at a time where publicity depended deeply upon using an imperative language... so historical explanations have that going for them.
But they compose better, not requiring monad transformers while still being type-safe. For the continuation to decide on which scope to pause on is like a monadic function returning a monadic value of a monad not directly enclosing it, but one or more levels up.
I'd like to add that while I agree 100% with this, from the perspective of the industry, abstractions exist to increase code reuse, assist in code maintenance, and help verification: in short they exist to reduce the cost of development, and -- from a non-research perspective -- that is the only metric by which they are judged.
Of course, that is not to say that types shouldn't be studied for pure research, or even that that research might not lead down the line for significant cost reductions.
Don't blame purely functional programming and referential transparency, which are unquestionably good ideas, for the downsides of laziness, which is indeed a bad default. Purely functional programming doesn't need laziness, and, in fact, is much better off with a formal distinction between values and computations, as in the call-by-push-value paradigm [0]. Pervasive laziness actually has negative consequences for equational reasoning, because it forces all free variables to stand for potentially nonterminating computations. To restore the practicality of equational reasoning, Haskellers often have to pretend free variables only stand for productive computations (that big-step-reduce to a term in WHNF), but this assumption is completely unwarranted by language itself.
> "wherefore referential transparency". If it is to avoid problems caused by shared mutable state, then there might be better ways, like making shared state transactional to some degree (Clojure, Erlang).
Although it doesn't exactly have transactions, Rust's strict aliasing and mutability controls is a much better argument for "there are better ways to avoid the problems of shared mutable state". Clojure and Erlang take the easy route of making the consequences of your bugs less catastrophic [1], but do very little to actually avoid concurrency bugs.
> If it is to make automatic verification better, then there might be easier ways (like some of the many model-checking methods)
Model-checking doesn't scale to truly large state spaces. Typeful purely functional programming is, to a large extent, about trimming the state space to exclude nonsensical stuff.
> and if it is to make human reasoning better, one should ask whether equational thinking is really the best way for people to think about programs
Yes, equational reasoning is the best way for people to think about programs. Equational reasoning, being barely more complex than high school algebra [2], is so easy that a normal programmer can do it casually. The most complicated operation you ever have to perform is substitution: replacing all free occurences of a variable by an expression.
> (I think the human mind is much more adept at simulating state than at math, and that an essential, "basic" abstraction such as monad transformers is already too high-order for people to wrap their heads around)
The untrained human mind might be more adept at simulating a single sequence of states than at math. However, even the best trained human mind is comically bad at analyzing classes of sequences of states, formally or informally. In particular, in the formal case, this kind of analysis often requires using logical relations, such as weakest preconditions [3], whose bodies can contain arbitrary quantifiers. Now, as anyone who's used formal methods can attest, rigorously manipulating quantifiers is a bitch - you often have to refer to their formal rules of inference, because it's so easy to use quantifiers wrong if you rely on intuition alone.
> I think that because much of PL research in the last decade has been focused on PFP, people are under the impression that it is somehow more "mathematical" or theoretically sound (not in the logical sense) than imperative, completely forgetting that fully-verifiable, and theory-supported imperative languages (like Esterel and its descendants) have been used much more successfully in the industry than Haskell and friends.
You're right that it's perfectly feasible to ground imperative programming in a formal foundation. However, as I explained in my previous points, in practice, the formal techniques for reasoning about functional programs are straightforward enough to learn and use by anyone straight out of high school, whereas the techniques for formally reasoning about imperative programs are both cumbersome and subtle - that's precisely why they aren't used except when incorrect programs are completely unacceptable.
> The argument of PFP vs. imperative is important and interesting, and there are both mathematical as well as psychological arguments to be made in favor of each. However, in recent years, many have come to see imperative as pedestrian and PFP as mathematically-advanced, which is completely false.
Again, I partially agree. Imperative programming is far from pedestrian. It's too complicated, in fact, so complicated that the only way we can cope with it is by handwaving away our proof obligation.
---
[0] http://www.cs.bham.ac.uk/~pbl/cbpv.html
[1] Erlang's marketing is very honest about this: Let it crash.
[2] The extra complexity comes primarily from the fact that high school algebra is single-sorted (all free variables are real, or at most complex), whereas equational reasoning in Haskell is multi-sorted (because Haskell has a rich type system).
[3] https://en.wikipedia.org/wiki/Predicate_transformer_semantic...
I think many people question that they are good ideas. Not pure functions -- but forcing all functions to be pure. If you do that, you are basically forced to use monads for all effects, and that is very questionable.
> Rust's strict aliasing and mutability controls is a much better argument for "there are better ways to avoid the problems of shared mutable state". Clojure and Erlang take the easy route of making the consequences of your bugs less catastrophic [1], but do very little to actually avoid concurrency bugs.
Yeah, Rust does something nice, too, but every operation allowed by Clojure is also allowed by Rust. I don't see how it's any better. Erlang's "let it crash" has nothing to do with data-race bugs. Once you have a GC everything is a lot easier when it comes to concurrency. Rust works very hard to ensure safety without a GC, which is necessary for domains that can't use GCs (like programs in constrained environments). And in any case, I think most people would consider Rust to be even less functional or more imperative (whatever that means) than Clojure.
> Yes, equational reasoning is the best way for people to think about programs.
It's good that you're so certain about this, but I can just as easily say "running your program in a debugger and a profiler is the best way for people to think about programs". How do you know? And nobody has even shown that apriori reasoning is any better than posthoc reasoning (debugger, profiler etc.) Deciding that we must reason completely about our program before we ever run them completely ignores a whole set of useful tools that we have at our disposal.
I also challenge you to program the world's most used sorting algorithm, TimSort, with equational reasoning. And consider this -- Haskell is grossly insufficient to really reason about programs in this way. Idris is the very minimum to even start, and even that doesn't even begin to scratch temporal logic.
> replacing all free occurrences of a variable by an expression.
That is a mathematical argument. Not a psychological one. You could use the same argument to claim that teaching a person to catch a ball is easier if you teach them Newton's equations than let them practice. I mean, we know simply "replacing all free occurrences of a variable" is Turing complete, right[1]? Who says people are good at that?
> However, even the best trained human mind is comically bad at analyzing classes of sequences of states, formally or informally.
And yet 100% of production software in the world, including avionics, medical devices and the like is written in imperative languages. I am not saying this makes your claim outright wrong, but it certainly challenges it, especially considering little evidence exists PFP actually makes things globally better (i.e. not by fixing one undesirable property and introducing another).
> that's precisely why they aren't used except when incorrect programs are completely unacceptable.
But if you want to be honest, you must admit that PFP isn't used anywhere. Imperative verification is much more pervasive. And nobody said people writing programs are willing to invest any extra effort to make them more correct than they believe those programs need be. Programs need to be as correct as they are now only developed at a lower cost (although, if you could offer more correctness completely free -- without any negative consequence -- I don't think people would object).
> It's too complicated, in fact, so complicated that the only way we can cope with it is by handwaving away our proof obligation.
Again, you're thinking about it from a mathematical viewpoint, but mathematics has absolutely zero relevance to whether a programming language is a "good" one or not. It can only promise certain properties, not their desirability. Yes, imperative programming is too complicated for machines to verify in every circumstance, yet claiming it is too complicated for humans pretty much ignores reality (or, at the very least, requires some serious explaining). And where proofs are required, imperative PLs can be just as rigorously proven -- e.g. Esterel, which has been used successfully by the industry much more than Haskell. In other case, I don't see that we have any obligation for proof, and when we do, Imperative languages can do just fine.
What is the source of that obligation? Are, say, provably correct programs more important than fast programs? Than programs shipped soon but are only correct "enough"? Who says correctness is the most important concern? I can tell you that software users certainly don't think so. We had a correctness crisis in the nineties, but the transition to memory-safe languages and wide adoption of engineering practices have pretty much resolved it. It's gotten good enough that continuous deployment, or "how do we ship our software in ever shorter cycles" is now a much bigger concern for most companies than "how do we ensure our software doesn't have bugs". This isn't handwaving but what people actually want. I don't think math can tell them to want something else. It's great that you can build a perfect table, but it doesn't help me if what I need is a chair.
There is one domain, though, where correctness is important even in 99% of software which doesn't require absolute correctness: security. But even with security, memory safety has done a lot to make things better, and where it hasn't, dependent types would be necessary to prove correctness, anyway. You can also help with richer type systems in imperative languages, like Java 8's pluggable type systems[2].
To summarize, I think we know precious little about the relative merits of PFP vs imperative (and by imperative I also mean functional-imperative). Unfortunately, no one has ever used PFP to write any large software other than compilers, so we don't even have the beginning of empirical evidence to even start an educated argument on the subject.
[1]: Well, sort of -- the equivalence between LC and UTM is something many misunderstand. There are plenty of computations that can be expressed by a UTM and not in LC. LC is equivalent because it is powerful enough to simulate a TM (but most certainly not to directly represent any TM program), but from that point on your computational model is no longer simply substituting free variables.
[2]: E.g.: http://types.cs.washington.edu/checker-framework/current/che...
Huh?
> But if you want to be honest, you must admit that PFP isn't used anywhere
Eh?
> Unfortunately, no one has ever used PFP to write any large software other than compilers
Whatnow?
Are you exaggerating for rhetorical effect, or do you really believe that?
As to the compiler thing -- that is the sad reality. In its 20 years of existence, the largest Haskell program ever written is the Haskell compiler itself. The largest company codebase is only a few million lines of code, and it is comprised of many programs, including a modified Haskell compiler (which, I am willing to bet, is their biggest program). Just for comparison, a medium-sized company codebase is around 50 MLOC. More production code is being produced in 2015 in Delphi, COBOL and Fortran than in Haskell.
This is my biggest problem with PFP: all discussions about its effectiveness are completely theoretical. Calling pertinent evidence "anecdotal" is an insult to anecdotal evidence. There are between 0 and 2 (much closer to 0) data points of large, complex, long-maintained Haskell programs outside the compiler itself (out of thousands of such projects that are started each year). I honestly don't know how effective PFP is, but we can be certain of one thing -- no one else does either[1]. Maybe it's the next big thing as everyone has been saying for twenty years (I remember around that time the talk at my University about how Haskell was about to take over the world) and maybe it's a total dud. Nobody has a clue (and that you've managed to write your API endpoint in two hours instead of two months or whatever isn't evidence of anything; evidence would be writing an air-traffic control system in one year instead of five, and maintaining it for a few years with a changing team of 3 instead of ten).
[1]: I remember that for a good few years most of the industry thought C++ is the best thing that had ever happened to it -- until projects started requiring maintenance. I do not for one second think Haskell is like C++, but vast, undeniably pertinent evidence was proven completely misleading because it wasn't longitudinal enough, let alone, shallow, scattered evidence, which is borderline relevant.
You aren't forced to use monads specifically. What you're forced to do is distinguish pure from impure computations somehow. In the CBPV papers, it's clearly explained that functions are just one particular kind of computation. In CBPV, there's an adjunction between the categories of value types and computation types, and it's this adjunction that gives rise to a monad [corresponding to the effect(s) that computations might exhibit]. If we consider pure computations only [no effects, not even nontermination], this adjunction is actually an equivalence of categories.
---
> Yeah, Rust does something nice, too, but every operation allowed by Clojure is also allowed by Rust. I don't see how it's any better. Erlang's "let it crash" has nothing to do with data-race bugs.
I'm not talking about data races. Of course the absence of sharing in Erlang frees you from data races. But they don't free you from breaking transactional invariants, such as "this doesn't even begin happening unless that condition is met". In spite of the absence of data races, Erlang is still about coping with failure, not outruling it. In Rust, you can use a nonforgeable witness that the precondition holds, and consume it when the operation begins. It's rare that a non-dependently typed language is better than Haskell at something other than performance, but that's exactly what's happening here.
> Once you have a GC everything is a lot easier when it comes to concurrency. Rust works very hard to ensure safety without a GC, which is necessary for domains that can't use GCs (like programs in constrained environments).
Garbage collection makes sharing easy. However, concurrency control (i.e., controlling access to shared mutable resources) remains just as hard. Otherwise, nobody would have invented such things as "global interpreter locks".
> And in any case, I think most people would consider Rust to be even less functional or more imperative (whatever that means) than Clojure.
Indeed, Rust isn't terribly functional (though it's more functional that meets the eye), which is exactly why I said it's a better argument for "there are other ways besides functional programming".
---
> I also challenge you to program the world's most used sorting algorithm, TimSort, with equational reasoning.
Well, in order to use equational reasoning alone, I need a purely functional algorithm.
> And consider this -- Haskell is grossly insufficient to really reason about programs in this way. Idris is the very minimum to even start, and even that doesn't even begin to scratch temporal logic.
Haskell's type system is insufficient to verify equational reasoning using the type system, but equational reasoning is so simple that programmers can do it manually anyway. [More on this later.] On the other hand, programming languages based on dependent type theory, like Idris, while often more beautiful, and certainly more powerful, don't yet pass my bang for the buck test for everyday programming: (0) They force me to annotate things that Haskell can infer on its own. (1) It is a royal pain in the ass to manually translate data between representations that are convenient for proving [e.g., Peano naturals] and representations that are efficient for calculating things at runtime [e.g., machine integers, big integers]. (2) Most dependently typed languages often don't have terribly efficient implementations. To the best of my knowledge, ATS is the only honorable exception to the last point.
---
> That is a mathematical argument. Not a psychological one. You could use the same argument to claim that teaching a person to catch a ball is easier if you teach them Newton's equations than let them practice.
Your analogy breaks when you think about it non-superficially. Merely understanding the laws of physics won't make your muscles strong enough, or your reflexes fast enough to go out there and actually catch a ball. On the other hand, when you've written a program alongside its proof of correctness, well, you've written a program!
---
> Yes, imperative programming is too complicated for machines to verify in every circumstance, yet claiming it is too complicated for humans pretty much ignores reality (or, at the very least, requires some serious explaining).
It's too complicated for humans to verify too. If it weren't, more people would be doing it casually. This is unarguable, unless you've tried it yourself.
> even with security, memory safety has done a lot to make things better, and where it hasn't, dependent types would be necessary to prove correctness, anyway.
I never said that everything has to be machine verified. If you give me a proof that I can understand (i.e., I'm confident I can correctly verify), that's more than enough for me. But, if the proof is past a certain complexity threshold, I'll ask you to convince someone else that I can trust (a better mathematician or a proof assistant).
Imperative languages can do that very, very easily. The definition of PFP, though, is that it is referentially transparent, i.e. no effects whatsoever (other than monads that are returned to the runtime at the top level).
> I need a purely functional algorithm.
Good luck finding one with the same performance... But the imperative one is now proven to be correct with no need for PFP. Writing a similar provably correct PFP one would have been harder (so it seems to me at least).
> but equational reasoning is so simple that programmers can do it manually anyway.
That's a nice claim. What we know for a fact is that programmers are able to write programs that are as correct as they're required to be with imperative reasoning.
> IT's too complicated for humans to verify too. If it weren't, more people would be doing it casually.
I don't understand exactly what you're referring to, but most programs as correct as people want them to be. The correctness crisis is over, mostly thanks to automated testing.
> If you give me a proof
Who says we want correction proofs to begin with? Most projects require some evidence of some measure of correctness. Certainly not proof of total correctness or even proof of partial correctness.
In an imperative language, can I write a map or filter whose argument is guaranteed to be pure? [Note that my notion of "pure" is stronger than Haskell's: I consider nontermination an effect.] Can you easily convince the type system that an imperative loop will indeed terminate? [Without supplying anything even vaguely resembling a proof.] With pattern matching and recursion, the compiler can check whether recursive calls are always passed smaller arguments.
> Good luck finding one with the same performance...
I'm pretty sure that, with substructural types, it's possible to update a single array element in O(1) in a referentially transparent manner. Conceptually, you're destroying the old array, and creating a new one with the same contents, except for the updated element. But, because the old array won't be used anymore, this can be done in-place. As far as I can tell, no other imperative facilities are needed to sort an array.
> as correct as they're required to be
What does this even mean?
Of course. See, e.g. Java's checked exceptions. If you marked every effectful function as throwing a checked exception, the compiler can enforce which effects are allowed.
> Can you easily convince the type system that an imperative loop will indeed terminate? [Without supplying anything even vaguely resembling a proof.] With pattern matching and recursion, the compiler can check whether recursive calls are always passed smaller arguments.
Model checkers can do this just as well.
> What does this even mean?
That programs aren't required to be 100% correct, just as they aren't required to be as fast as possible. It is perfectly acceptable for most programs to fail on rare edge cases. If there is an inverse relationship between the manifestation of the bug in the frequency of the circumstances that causes it, then you're fine (put simply: it's OK to fail with ever higher probability on ever rarer cases).
Mutation is not disallowed in PFP, by the way! You can write a perfectly great Timsort that's pure. You just can't pretend that your mutation effects are not observable when they are. (And you can use monads as type cues when that occurs).
Also, you can have a pure Timsort with linearly or affinely typed arrays. Only action-at-a-distance truly requires some framework (monads, algebraic effects, whatever) for embedding an effectul language.
I would be interested to learn how (with the same space complexity?), but anyway my point is that writing a provably correct Timsort using CH is harder than simply proving the imperative algorithm with a model checker.
So it's not abstraction vs. algorithms. The lambda calculus gives you both.
Parallel or is important because it uncovers something interesting: LC works with high-order functions but it can't do high-order programs, namely representing the computation process itself as a first-class entity. LC is not introspective in that sense, and most of its limitations flow from that (including the difficulty describing complexity, which is a feature of the computation process, not of the functions). Again, I'm not well versed in PL theory (or hardly at all), but I think that Lisp's macros therefore actually make it strictly more powerful than LC[1] (I think I may have seen this mentioned somewhere).
Indeed, macros have allowed Clojure to implement stackless continuations, and bytecode manipulation has allowed us to implement full continuations on the JVM without any JVM hacking. Similarly, this kind of direct computation representation allows all sorts of instrumentations and various interesting transformations that are not representable in LC proper (without macros). This isn't at all catastrophic -- you can live without those transformations -- but they can be very powerful and very useful.
[1]: And by "strictly" I mean with direct representation rather than by simulating a TM.
Additionally, you can always just Gödel encode a program, pass it in, get all of this functionality. That might be considered cheating (simulating a TM, although you could just as well simulate LC in both models). This is not actually any different from what the UTM is doing... it's just possible to ignore the difference between interpretation and normal operation because it's all just tape.
That's the big difference between (Erlang, F#, OCaml) and (Java, JS, Ruby, Python, C++). I can't speak for Clojure because I don't know it.
It's this: language choice is often a shared/communal one. Therefore, it's required not only to find personal reasons for the choice but also to convince a not insignificant number of your colleagues to make the same choice. This means that some portion of language advocacy is a war for survival in a way that vegetarianism isn't (at least until you have to choose what restaurant to go to with some vegetarians in the crowd—tension grows).
(To be clear, I'm just noting this, not trying to be a proponent of it!)
So that fuels asshole behaviors because one wants to find a justification not only that an alternative exists and works but that it is actually better. So it's not just a matter of coming off as arrogant but there actually being a seed of truth to that claim.
Now what I feel makes PL debates remarkably dangerous is that theres a prevalent idea that PL all comes down to taste---therefore anyone who is trying to make a PL sale is peddling snake oil at least partially. Every action they take is therefore hostile in that it is impossible for them to offer a legitimate cure.
In honesty, that idea is completely false. Language really matters a ton and is merely a very complex decision not a meaningless one. Each language offers its own sets of cures and downsides and actual decisions are tricky.
So when an FPer comes with arguments, legitimate but complex ones, that FP significantly improves programming experience/outputs then making the trade-analysis to your personal needs is a big undertaking. When, further, they do this with at least some motive to convert you, it's a forced, big undertaking. Finally, when you balk a bit at the potentially massive investment with uncertain return involved in analyzing a new language for a project and this is thrown back at you as a failure to recognize the legitimate value of FP. Well, that's easy to see as asshole-y.
What makes a language community fun? Lots of ways to get started, lots of positive, non-forced examples of benefits, and a community oriented around perfecting their own trade not proselytization. Once you start heading out of those bounds you pick up assholes.
Well, so is food. I'd say you just gave a good argument for why the analogy is even more on point that it might appear at first!
I'm not sure that is well-established at all. I mean, languages probably do matter, but just how much we don't know. Also, the choice of language may well matter more due to availability of tooling (debuggers, profilers, IDEs, ease of deployment etc.) and other extra-linguistic features (GC, separate compilation, dynamic linking) than actual language features. Even measuring outputs is very hard. The average lifetime of a codebase is roughly 10 years, so the development costs must be totalled over that entire period. The cost structure however, is very different in the first year and in the eighth.
I don't think we have good answers to any of the questions: how much languages matter, how they matter and why. It is precisely because we don't know that we can argue so much :)
Some of the PL discussions do, however, make me sad because while I believe the choice of a language matters (though I'm not sure that by a whole lot, and when it does it's mostly due to extra-linguistic concerns), the choice of algorithms matters so much more. I wouldn't want young developers to think that the abstractions they use to express their algorithms is as important as the algorithms themselves (as long as the code is reasonably maintainable). Also, I don't want young developers to equate software with the code the program is written in. A useful, efficient running program is much more than its code, and sculpting code is not the sole means of achieving high-quality software.
But we're not disagreeing here. The relative merits of all these factors are expensive as hell to compare.
I'd love to have a longer discussion with you sometime around abstractions and algorithms, though. I think I have a few points on the side of abstractions that pass far beyond mere maintainability, but I also really empathize with your argument. So, it'd be interesting to flesh it all out.
Scott Wlaschin does an excellent presentation on Function Programming Design Patterns (http://www.slideshare.net/ScottWlaschin/fp-patterns-ndc-lond...) where he talks about using functional patterns to properly model domains.
I suspect Scott did a search on "Design Patterns" and "F#" to see what was already out there. He was probably frustrated that the only thing he saw were references and translations of GoF design patterns.
From what I gather, Scott's slide was a funny way of getting out of the way that his presentation was not going to be another thing about translating GoF design patterns to F#. (Full disclosure, I actually did that presentation here: http://sgoguen.github.io/presentations/FS-Patterns-2/#16)
UPDATE: Just confirmed with Scott that was the inspiration of this slide and joke.
I disagree with your meta-analysis of where the root of perceiving "arrogance" comes from.
(I'll shall attempt to speak for non-practicing functional programmers and the following opinion is not my own.)
In my observations, it's that all the essays and evangelizations about functional programming do not manifest themselves in the industry as irrefutable evidence that it creates better programs, with less bugs, at higher rates of productivity.
For example, folks can point to Jane Street using OCaml, or a London finance shop using F#, but are they doing 10x better than Goldman Sachs using the clunky C++ language? Maybe. But we won't know because there are too many confounding factors.
Functional programmers can repeat all the "benefits" of the style with "immutable", and "referential transparency", "monads", etc, and describe languages in glowing terms such as "Haskell has this beautiful internal mathematical consistency that Java doesn't" or "Lisp with its homoiconicity lets you write a DSL as first-class concepts with macros whereas C++ makes you force-fit them into classes", etc, etc.
All those bullet points may be true but it doesn't manifest itself as obvious slam dunk winnings in the industry.
That's the root of the arrogance: the attitude of "superiority" about functional ideas does not match what people see on the street. In other words, people wonder why there are no AAA game studios using functional programming that blows everyone else out of the water crawling on their hands & knees with C++. Or some application software company using Haskell to deliver software with 10x less bugs, 10x faster, and 1/10th less cost than their competitors using Java. Or some YC company using 100% functional language and killing their competitors stupidly using Python/PHP.
So far, there are hundreds (thousands?) of tutorials about monads and recursive Fibonacci snippets but very few case studies (if any) of real companies killing the competition as a direct result of the functional programming language. Companies showing massive industry results is how you win hearts & minds. Fibonacci exercises are not enough.
What the functional community needs are better essays that go beyond "monads" and addresses why the mystery gap exists between functional theory and industry results. My guess is that the functional benefits are real but there are other aspects of software craft that overwhelm its advantages. In other words, the essays needs to look at functional style holistically.
As a historical analogy, the benefits of structured programming languages vs machine language were initially debated in 1960s. However, within less than a decade, everyone adopted it because the multiplication of productivity was obvious and compelling. The ease of programming wasn't just a 10x improvement, it was arguably a 100x improvement. Now, assembly is only done for isolated pockets of tight loops and resource constrained scenarios.
If the FP>OOP is similar to Structured>Assembly, then for some reason, the real world has not shown that same magnitude of increased benefits. Fix that perception and the root of "arrogance" goes away. Today, we don't say one is "arrogant" for explaining that structured programming will allow teams to write software faster with less bugs.
Imo, that's the type of self-reflection that functional essays are missing.
This is only true for problems above a certain (not entirely trivial) complexity -- otherwise, my structured code is WILDLY less efficient to code in. Further, structured code requires a keener meta-theory and more abstracting prowess to correctly use.
Similarly, FP has many of the same requirements to leverage the benefits over structured code in general that structured code has over assembly, and we just sort of get lost. Fully leveraging the FP techniques is sort of at the edge of monkey understanding of computation, and not everyone can really work well at that level -- which is why we don't let all the programmers design the architecture, regardless of language.
In the cases that we really applied it well, it seems to have consistently yielded massive improvements in reliability and throughput. Asynchronous, immutable data algorithms have been much of the latest innovation in the industry for certain applications (stretching from database queries against large data sets to firewall routing rules).
IMO, monkeys are kind of dumb and humans tend to think too much of ourselves, in general. Several of our technologies would work better (like they should in theory) were we slightly less dumb in our day-to-day execution.
Maybe not. (I presume you're referring to the Blub essay.) But that essay doesn't advocate FP; it advocates Lisp, with macros. I'm pretty sure PG would say that Lisp is looking down the power curve even at Haskell, because Haskell is not homoiconic, and you can write FP code in Lisp if you want to. But one still wouldn't call Lisp an FP language.
So I don't know whether that essay really goes where you want to go...
But in the real world, fp languages just don't seem to work out very well, not in companies I read about, not in classes I've taken. They only seen to work well in Fibonacci examples. (And functional features in other languages are great though)
I wanted to like fp because all the cool kids do. But I need to see results or some evidence that the drawbacks of fp can be overcome and functional programmers give me nothing but "if you don't get it, your stupid"
It also depends upon your use case.
I have a project that requires high concurrency that could be resolved with consistent-but-stale data. A large amount of our C++ code is around achieving that sort of thing and there's always bugs to be found. But its basically free in FP and a bug in this means a bug in the core implementation of the language (not likely).
But, we also have webapps. The mutability of this data is not dissimilar to how it would work in FPs, despite working in Java. We fetch data from a shared DB into memory that is not shared, manipulate them, and write them to the DB. Adopting FP would do little because our environment is mostly the same as that of the FP environment.
That said, the best tool for the job is the usually tool you know.
10x?! Crumbs. How about 10% better with happier developers? I'd take that. It's also a claim I'd be very happy to put my name to.
On the other hand I would not be surprised if there are some projects that simply cannot be delivered without programming in an FP style, since otherwise the complexity kills you.
Why? FP is just a theory of computing, equivalent to all the other existing theories. It allows to do some analysis which is not that simple with the other theories (and it is crappy with the other important analysis methods, most notoriously with complexity analysis). It does not "add" or "avoid" anything at all.
Pure functional programming subtracts something from your expressive power, but it also subtracts from the assumptions you have to make about code. Less expressive power is a bad thing, but fewer assumptions are a good thing. The inequality of assumption versus conclusion is right there in the functional arrow.
I'm not sure I can easily work the vegetarian analogy into that. Vegetarians notoriously frustrate catering, while I think people who agree never to mutate data are probably easier for programmers to cater for.
My sense, from spending a decent chunk of time in a couple different communities, is that they don't do enough policing of their jerks and that that makes itself most apparent to novices rather than established people. Many programming communities don't handle jerks well, for a host of reasons around social standing and conflict avoidance, but of the ones I regularly participate, I don't think a guy like Scala's I-think-I-can-use-the-adjective-"infamous" Tony Morris--though granted, he seems to wish he was Haskell's Tony Morris--would be tolerated past weeks, to say nothing of years.
I really like functional programming. But I find that notions like "shared-immutable programming" are more accepted by people who've been burned by the more socially toxic sections of the universe. The rebrand helps remove the association with the jerks, and that should speak to the impact of those jerks.
There's a fine line as to whether any statement is arrogant and a lot of it actually comes down to presentation. To what extent do you act like your views are obvious, or abuse the word "objective"?
Is the person who maintains that FP is better arrogant or not? I think that comes down to a mixture of framing (let's be clear about which things are "objective" and which are opinion), and intellectual humility. In any case, I think we should separate that from the fact that there are at least a couple of true assholes out there.
Well, sure! But you have to recognize the sorts of people you're dealing with, yeah? Software developers, as a whole, are not exactly awesome at the interpersonal thing, especially when conflict comes up. So circumlocutions and euphemisms are kind of foreseeable there. Which is of course a really unfortunate thing, because it allows useful idiots to go "well, I don't see arrogance" in a way that lets them implicitly excuse the real rot in the pile, but there you have it.
Is it possible to be arrogant without being an asshole? I always took arrogant to mean a specific type of asshole.
On the other hand such an attitude is not representative of the Scala community, which is filled with really nice and helpful people. I just went to Scala Days in Amsterdam and everybody I've met was super nice. Try it sometimes.
What is "shared-immutable programming"?
"Share nothing that isn't immutable." =) I like it more than "shared-nothing", because I share tons of stuff, it's just all inherently thread-safe and simple.
I also think that the parts of the .NET community I'm exposed to on a regular basis are professional, calm, and not prone to asshattery.
YM, of course, MV. =)
I would say #haskell is usually very friendly, but no community is perfect. I noticed the above once and it was actually quite easy and simple to fix it. I simply joined in on the ongoing question session and started implying that we should perhaps answer a bit more nice to the newcomer.
Tis human nature, I guess, always trying to find a way to distinguish yourself from the masses. A bit of a Star Belly Sneetches phenomenon.
https://en.wikipedia.org/wiki/The_Sneetches_and_Other_Storie...
Everyone else seems to be a bit more focused on the result of their work, not how they get it done. They certainly don't seem to take pride in their choice of tool. Imagine if a construction worker felt self-important because he uses the Aero Hammer 7, while they use the clearly inferior Aqua X Mallet. It's silly. I don't think this behavior is as widespread as many here think it is. (edit: widespread among the general population)
I could go on and on. The answer is a resounding YES to both of those questions by the way. They care just as much as programmers do about their software. It's not limited to software either. Writers who write by hand care about their pen. A lot. People who put a lot of energy into their craft care. Is that human nature? Absolutely. Is it human nature to then compare and try to set oneself apart from the rest? Of course.
> Of course they won't care because it makes no difference to them.
Exactly. My point is that it also shouldn't matter to programmers, in the same fashion. Why does it matter if you use Emacs or Vim or Notepad? Why does it matter if you use Ubuntu or Windows or OSX? If you can produce high-quality work that is maintainable in roughly the same amount of time as someone else, the tool you use doesn't matter.
People should have preferences and they should be able to use the tools that are comfortable for them. However, any self-important feelings based on these preferences are silly and don't really make sense. That was my intended point originally.
> Why does it matter if you use Emacs or Vim or Notepad?
> Why does it matter if you use Ubuntu or Windows or OSX?
> If you can produce high-quality work ... the tool you use doesn't matter.
Professionals care deeply about their tools. Writers care about pen qualities, musicians care about software and instruments, hunters care about their weapon. Programmers care deeply about our editors, languages, source control, and computer platform. Photographers care about their equipment.
Comparing Vim/Emacs with Notepad is like comparing whether to use a computer versus writing by hand -- nearly no professional writer will choose to use a pen and paper when they have the power of a word processor at hand, and similarly nearly no programmer will choose Notepad over a professional code editing tool.
You'll quickly find that many people who care about writing by hand care deeply about not just pens, but the ink they use within them -- the depth of nerdiness in the fountain pen forums is amazing. Consider looking at "fountain pen reviews", for example.
People usually don't care about post-it brands or pen brands (or band-aids or toothpaste) because, in most cases, they are __functionally equivalent__, so you can decide on price or convenience. If one is not a discerning user, in theory one might buy a set of kitchen knives, a coffee maker, a car, a suit, or a computer based solely on price. Most of us have been burned enough by low-quality tools to care enough to discern Good from Bad.
As programmers, we have the expertise to compare Python vs Ruby, or Emacs vs Vim, or Linux/OSX/Windows. We might not have the same depth of caring in other subjects (cars, cooking knives, pens, phones), and therefore do not engage in the same degree of holy-war that we tend towards when evangelizing our favorite tools.
I've found this an interesting topic. I've seen a number of C++ communities where they are clearly disrespectful to new members or people who are new to the language/library. But yet on assembler forums I've seen people who made comments like "hey when I get home from work I'll create a sample program for you".
Just remember one important fact before you call someone stupid for not finding that buried option in Visual Studio - there are people out there making twice as much as you are writing vulnerable shitty code. Try to show some humility to the guy asking questions - he is trying to at least make an effort to learn from you and others.
Are the functional programmers arrogant? Certainly. Are their critics arrogant? Yup. Are they both irritating? Sure.
I'm not a big fan of the social justice community, but they are right about arrogant programmers. They aren't racist or misogynist, they're equal opportunity assholes.
In my experience most people who are oblivious to the arrogance of the programming community are themselves the arrogant ones. There are a lot of friendly people, but if you're a real noob and try to contribute shitty code to a big open source project with real programmers, you're gonna get your ass handed to you.
Even the groups of programmers who claim to support "safe space" environments are chock-full of arrogance, they just don't use cuss words and belittle you in a more passive way. I assume it's pretty much the same in every profession when noobs make mistakes.
Of course, if you're genuinely a beginner you don't know you're asking the wrong kind of question.
Yes, it's frustrating to help someone who is strugging to run/consume Foo only to eventually find out what they actually wanted to do was Bar and that consuming Baz would have been a much simpler solution.
But if you don't have the patience to help, don't try to! Too much programmers think they can fix the world and go around trying to correct beginners[1] when they should just get out of the way.
[1] Many high rep stackoverflow users suffer from this, they've seen the same shit too many times and rather get out of the way and let others help the beginners they'll shut down those beginner questions. If a question was valid 5 years ago it should still be valid today, but they seem to forget what it's like to be a beginner.
Now then, perhaps I am a terrible programmer, and perhaps I just have no chance to understand what's being said because of this fact. But... why did ask me to join them then? Why didn't they let me go, if this is truly the case?
The underlying issue, after getting communication actually occurring in two directions, is I was being subjected to the results of several previous poor developers - I was getting the brunt of their frustrations which was actually aimed at other folks. Doesn't make the earlier parts of the conversation any less arrogant, or tolerable, unfortunately.
[1] See, for example, http://maniagnosis.crsr.net/2012/03/clean-coder.html
[1] http://blog.cleancoder.com/uncle-bob/2015/01/08/InterfaceCon... [2] My feelings are in line with "“Considered Harmful” Essays Considered Harmful" (The title being the one exception to the rule : ) http://meyerweb.com/eric/comment/chech.html
Where the reputation comes from is when someone talks about some problem they have in an imperative language and some Functional Programmer swoops in to tell them how it wouldn't be a problem if they simply rewrote their entire project in Erlang or something.
There is also a perception that functional programmers aren't willing to discuss potential downsides to functional programming. Haskell is the last language you'll ever need for anything you ever want to do and it is perfect for all purposes from embedded systems all the way up to business applications and realtime games.
http://chris-taylor.github.io/blog/2013/02/09/io-is-not-a-si...
It's hard not to be arrogant, especially when you feel strong conviction about some ideas. You actually feel like you need to wake people up from their intellectual slumber, so you'll find yourself saying things that draws attention. Getting your ideas attention is good, right?
I've had my ass handed to me on more than one occasion and my arrogance pointed out as well. It's been an important lesson because its seared lessons remind me to present ideas with empathy.
However, I want to live in a world with a diverse set of opinions that tolerates fundamentalists and their fundamentalist/reductionistic ideas on things like functional programming. Intellectually speaking, I think we need all kinds. We need people who are stubborn and will completely commit and defend ideas and we need people who try to unite ideas by solving what they see to be false dichotomies.
I think it's important to call out arrogance when you see it, but branding a community as arrogant is obtuse and disingenuous. It can be a useful act that forces a community to flush out bad actors, but it also has the effect of conflating ideas with bad actors.
It's not wrong for them to be advocates, either. "This set of problems goes away if you do it our way." Well, if they didn't repeatedly shout that in various forums, I probably wouldn't know that FP was a viable option with some benefits.
What's wrong is when some of them call everyone else unenlightened for not being on their road, when some of them insist that their road is the only possibly right road, when some of them state that any other choice is stupid. That's arrogant. It's not the whole community, but it's some.
> the whole article seems to boil down to "someone I like
> said something I don't like :("
In an interview once, I was asked what defines functional programming. After I provided an answer I thought was reasonable (function composition, statelessness, immutability), my interviewer politely responded to my obvious ignorance with a few corrections like, "you forgot first class functions!". The tight smirk on his face was as if to say, "Ha! Gotcha, n00b!"
I think a lot of functional advocates are out to prove that it's better than imperative and other paradigms, but it ends up coming off like a forced sports rivalry. What the hell does "better" mean, anyway? Pick a good enough tool for the job and ship something.
I would say that you interviewer sounded a bit like an asshole then, but I would not conclude that the whole of the community of FP are, from just that.
Agreed, and neither would I. My anecdote was merely to illustrate that there is a stereotype borne out by rare but memorable experiences attached to functional programming die hards that includes arrogance. FWIW, I think it comes in large part from the academic emphasis functional programming has held for many years. The folks who come out of that world have often bought in wholly to FP, and their attitude may as easily be regarded as pedantic rather than arrogant.
You could call them architecture astronauts: http://www.joelonsoftware.com/articles/fog0000000018.html
It's fun learning new tools and tricks, too, so I don't mind advocacy and outreach. But many folks in the cult of functional present it as a panacea -- "If only," they lament, "everyone would use functional languages, we wouldn't have all this terrible software because functional languages eliminate entire classes of bugs!" and other tropes. Even if all the hype is true, that doesn't mean that the functional model of computation is the best (easiest to grok, maintain, or whatever) way to solve every problem. There is room for a variety of ideas and programming models, and I'm glad functional is one of them.
In my opinion, the phenomenon arises from a simple fact: all of these programming models are terrible for writing some significant subset of software functionality. Just because you can write any piece of software in FP, OOP, etc does not mean it isn't an objectively poor engineering decision some of the time. Nonetheless, I frequently see assertions in various communities that these apparent deficiencies are really due to insufficient cleverness on the part of implementors rather fundamental weaknesses in the model for the use case, while ignoring real-world engineering constraints like cost, performance, and maintainability.
The perception of arrogance, or lack thereof, largely reflects the willingness of a community to acknowledge the fundamental weaknesses and deficiencies of their chosen programming model when interacting with people outside the community. Too often, people that question the objective limitations of a particular model are told that if they were smarter or better educated, they would not worry about those limitations. For obvious reasons, this comes across as arrogant.
One of great changes in programming languages over the last 20 years is the extent to which they have become relatively model agnostic and allow you to switch models fluidly. FP remains one of the last bastions of single model purists in my experience, which may explain why they are perceived as more arrogant by outsiders.
I can take this even higher. Some _people_ are just arrogant. It really has nothing to do with programming. They are just arrogant.
Don't play with those people. Play with nice people.
As a target, "all those jerks over there in the XYZ tent" is a really crappy one. There's no tent. There's no one group of people who represent the interests of a certain methodology. "We" can't even get people who are ostensibly in the group to agree on what the methodology consists, and you want to make a target out of "us"?
There are certain pockets you can talk about. "The developers of XYZ programming language who frequent this particular subreddit." Maybe. You're still probably missing the vast majority of practitioners of that language.
And what is the end result of this supposed to be? If we can even clearly define a target, what then? Throw away the methodology because of it?
[0] substitute Functional for OO or imperative or what have you.
It doesn't matter as much if FP'ers are arrogant if you can just learn it properly without them.
I've read blogs, academic papers, chapters in books and more about FP, but I've really yet to find a clear, concise and mutually agreed upon starting point. The best example so far has been a hard to find essay from Recurse Center. Perhaps one exists, but the fact that my searching didn't yield one (and I have searched) might be a more substantive reason for why more people don't FP.
> I know this because I was on the receiving end of this generosity from the moment I showed an interest. This is why it’s frankly baffling when this charge of arrogance is levelled. Why functional programming? Of all the areas of software development, why is the arrogance of functional programmers a thing?
It's not how welcoming FP people are to newcomers. It's how condescending and condemning some FPers are to those who are outside FP. "You're unenlightened/stupid/incompetent/wrong if you don't do it this way." Yes, I have heard that attitude, here on HN, more than once.
Immutable is good. It's not always the right answer, though. Some FP advocates don't seem to get that, and assume (and state) that everyone else is wrong to ever do it any other way. That's arrogant, because it assumes that those who choose some other way cannot possibly have valid reasons.
The guy basically goes on some pseudo-mathematical journey that has only a tangential relationship with programming. "It's easy to understand!" he claims multiple times. And it was! I understood everything he said. But I had NO idea how to apply anything he said to my application.
And so it goes with virtually every article I've ever read about functional programming. The truth is out there, but it's hidden in such a dense forest of obfuscation that I'm not sure I'll ever know what a monad is.
The search goes on, but I'm afraid all I'll ever discover is yet another blog post where all the functions are named f(), g() and h(), or something about prime numbers.
So I suppose arrogance may be assumed from seeing this concept explained so badly so many times. It's like people who understand functional programming are gatekeepers and do such a horrible job at letting people in that it seems like we've been locked out on purpose.
Here's a secret... It's explaining monads!
https://www.youtube.com/watch?v=YX3iRjKj7C0
and probably worth watching if you enjoy Uncle Bob's soulful stylings and are interested in his thoughts about programming language communities.
Based on how people reference complexity, I'd think FP languages would be considered to be at the top of the language community hierarchy. Even within FP, I don't feel it's a stretch to think many view Haskell and Lisps "above" say Scala. On HN, we may see more about Rust than we would out in the wild.
I do find that FP communities tend to seem inviting to at least some other language communities and are eager to introduce the FP world to certain other groups (say Python, Java) but perhaps less likely to other language communities (PHP).
The fact that some user of a functional language is arrogant has no bearing on whether you should use it for your project.
If you think that it does, then maybe you shouldn't be a programmer, for two logical reasons. One is that it is not logical to think that, and programmers have to think logically. The other reason is that for any stack of development tools, we can find someone who arrogantly advocates it, so if the existence of arrogant evangelists is a problem, you must necessarily be discouraged to use anything at all.
What car do you drive? If it's any good (and even if it isn't), there are owners of the same car who arrogantly believe that they are better than others. So, sell it: you don't want to be associated with them!
Some languages with a welcoming community nevertheless have arrogant evangelists.
You can make your own language and use it; where is the welcoming community then? And what if you're arrogant? :)
The community you're joining will greatly affect how competent you become in their respective language, so anyone who partially bases their language choice on community is being completely logical.
It is entirely logical to consider the group of people that you're going to have to interact with on a regular basis to get help while you're starting, and decide you don't have the energy to deal with those assholes.
And while every group has some assholes, some have more than others.
A few years back, I started using a framework based on the openness and friendliness of its community (the tech was good, of course). That lead to a job with the framework's publisher, which lead to travel, speaking engagements, a self-published book, friends and acquaintances around the globe, and my current job. I'm not saying you can't or shouldn't use a language/framework if its community is arrogant. But, the benefits of finding the right community are well worth passing over the others.
However I do agree that their can be a lot of arrogance in the FP world.
This is actually true of quite a few languages though. C (i.e. kernel) devs can be complete assholes towards people wanting to learn but are finding searching for things frustrating.
On the flip side the C and C++ communities, especially /r/cpp, /r/c_programming on reddit[1], are fantastic. Very newbie friendly with a lot of expertise there. Stephan T. Lavavej (/u/STL), the Microsoft STL maintainer, is very active on /r/cpp and responds to many questions asked of him.
[1] Also check out /r/cplusplus, /r/c_language, /r/cprogramming and /r/cprog
[0] https://www.reddit.com/r/haskell/comments/3cfax3/new_haskell...
I seem to be cut from the old hacker mentality that all information should be shared and the collective minds can improve it to perfection. I have increasingly found though a mentality of "ask me no questions and I'll tell you no lies".
I guess my being a narcissist isn't what it used to be.
http://2014.funswiftconf.com/speakers/andy.html
(Andy leads the mobile team at Khan Academy, and worked on UIKit at Apple before that.)
In the latter group, there is often language arrogance. But it applies to any language. How often haven't we heard "that's not a problem with Go, just use it longer and you will understand".
Now I wonder if there are still any arrogant COBOL believers around ;).
The idea of an object is fine, but it doesn't need methods attached to it. Also, by making an object immutable, anything can have a reference to it, and do what it likes with it because everything else pointing to it essentially has a 'snapshot' of the world at that point-in-time.
That makes it very easy to move packets of data around a system, without the burden of the attached methods. And if multiple processes have a reference to the object, they know that what they know is true now, is going to be true when they come back to use the object again.
Once you detach the functionality from the data, and once the thing you're pointing at can't change, then you realise that actually everything you do (applying functions) is a projection from an input to an output, and that state is an emergent property of the function. Therefore you don't need to manage this morass of ever changing objects.
A request can project to a response, a query can project to a list of results, etc.
Then all outputs are dependent on only one thing, the input. Not shared mutable state, not unknown side-effects, etc. Then everything is testable and provable, and it's very easy to test and prove, rather than requiring elaborate mocking frameworks, dependency injection, etc.
The more I did this, the easier I found it to keep a mental map of my programs. And ultimately that's the major thing that programmers have to deal with. It's our own weak and feeble minds that get in the way of developing more complex programs, and we've developed these elaborate techniques to help ourselves.
For me functional programming is another one of those crutches, and it's a damn powerful one. Because I know that once a function, that projects an immutable type to another immutable type is written, it stays written. It doesn't suffer from code-rot as the rest of the app grows around it. I can then safely park that part of the program in the 'done' pile and never worry about it again.
Finally, most of the things that are supposed to be OOP's strong points (like polymorphism, encapsulation and modularity), I feel, can be done better in functional languages. Pretty much every pattern in the GoF Design Patterns book can be better implemented using functional techniques.
Anyway, your mileage may vary obviously, but as a reformed OO programmer, I'd definitely advise you to give it a shot. It takes a bit of time to sink in, but it's worth it.
That being said...SICP 4 LIFE!