A different view on Functional Programming
matiasmorant.wordpress.com
matiasmorant.wordpress.com
What problem could possibly exist that only could be implemented in a convoluted way?
Sounds more like: me and my team made trade offs that were, in retrospect, bad and now we don't know how to transform our code into a new program will fulfills the same requirements but is easier to inspect and understand.
Really? There are entire industries which only work on "messy problems." ie game dev: implement a controller where jumping is both responsive and fun.
I pity people who have to work with SOAP or Salesforce.
Overall I really like the article because it spells what is an essential feature of fp for me much better than I could have. It is just that it is too far from reality of most programming jobs (unless you happen to be Peter Norvig, working on the Mars rover).
Now, you figure out that the decoder which will decode the video expects the video stream to look different (specifically: the encoder produces only one picture parameter set and sequence parameter set at the beginning of the stream, but the decoder only understands the video if every key frame contains its own copy of the PPS and SPS). After a mix of reading source code and talking to people online, you figure out that there's an (undocumented) way to configure the encoder to include a PPS and SPS with every key frame. You figure that this is really something which must be specified by the h264 specification, so either the decoder is breaking the specification, or the hardware's encoder is; but at the end of the day, it's your fault if the encoded video isn't displayed on the receiver.
Then, you figure out that while you're trying to encode 1920x1080 video, the encoder is only able to encode 16x16 chunks, so the output image is actually 1920x1088. This isn't documented anywhere, but after speaking with the person who wrote the driver for the hardware encoder in IRC, you learn that there's no way to make the driver handle it properly, at least not yet. Therefore, you extract the picture parameter set produced by the encoder, make some changes to have it include information about how the picture should be cropped by the decoder, and splice the modified PPS into the buffer for the encoded frame.
That is what I've been doing at work (with some modifications; the above example is derived from working with two separate hardware video encoders). Even if it was possible to obtain exact documentation on how the encoders worked, sitting down and thinking really hard about how to describe the problem mathematically before writing the code would've been really hard, but because nothing really is documented, I will go so far as to claim it's impossible.
Most business problems can't be solved with a formula, or with a thousand formulas, or with a million formulas, especially if the problem is “our organization is an illogical clusterfuck” or “we don't even know what we should be doing”.
> I consider the fact that you can't code your solution in this style until you have done so as a feature
It's a feature of programming discipline and intellectual honesty, not of functional programming.
> As opposed to imperative style which (...)
Functional programming doesn't prevent you from implementing “half done solutions with hidden bugs, not covering all corner cases”.
It can be helpful to look for generality and abstraction when solving problems. But it can be unhelpful to assume that general solutions and common abstractions must exist to an extent that makes a given solution more robust and efficient than a patchwork of partial solutions that explicitly handle corner cases.
I don't find myself scribbling down mathematical equations when doing things like chaining promises in javascript or transforming sequences with LINQ or marking fields as const. I realise functional programming can go in very different directions to things less mundane, but the general principle of avoiding side effects has been a huge win for me when programming in any language, even C++. It's just easier to debug or write something when you can fix some constraints, ie "this method is referentially transparent".
Those are pretty low hanging fruit, but they help a lot.
But the article (and my criticism) is about the rather high hanging fruit of mathematical purity - of which "side-effect free" programming is just a side-effect (pardon the pun).
Did I mention network failures? Also, as beautiful as side-effect-free programming is, any non-trivial program will have side-effects outside the program itself. And your beautiful functional code must account for that.
For them, your argument is in favor of avoiding functional languages. Too math-like! A step-by-step procedure might be simpler to understand.
Chinese looks complicated, unless you know Chinese.
The ultimate criteria to judge a language should be how efficiently (concisely) it expresses the concepts of the subject matter.
At most, I will concede you: Goodness_of_language = conciseness × parsing_speed_of_an_experienced_user
In the example from this article, all developers I've ever worked with would understand what the "good old C" example is doing within seconds. Most would understand the Python example. Most of them still working in the industry today would be able to figure out what the functional python code was doing without much excessive effort - it would take them longer, which is bad, but if they were working in it every day I'm willing to believe that they'd parse simple examples like that readily enough.
None of them would understand APL. J is even worse. I stared at the J code for a full minute without any comprehension of the language syntax or what was being accomplished. That code isn't readable or maintainable. It is bad code.
But why does code need to be readable to someone who doesn't know the language? Yes, it's true that Python and some other languages are pretty readable even to people who haven't written any of that language, and that is a nice feature in many contexts... nevertheless, if I'm hiring a Python programmer, I'm either going to hire someone with experience in Python, or I'm going to accept that they're not going to hit the ground running and be massively productive right out of the gate... so how is it really such a loss to write in J and accept that non-J programmers won't understand my code?
Writing terse readable code is great. Writing code that is so terse that it doesn't have variable names is an antipattern.
They all more or less follow "C-like syntax".
In fact compare APL or J to any of the Tiobe 20, and you'll see they are wildly different in look and feel.
And for the laughs, I've been taught java first, and read a lots of C code, and I had a very bad understanding of what was going on. APL or similar don't cause that confusion. Actually, after years of FP, I now get C a lot more. To quite Hickey, it's not easy but it's simple. There's a set of primitives and operations, a clear system. It fits some peoples brain. For others, imperative state machine are just what they like.
Some might say that the ultimate criteria to judge a language should be how quickly an inexperienced user starts to be fluent in it.
I studied math (so the syntax isn't really an issue), but I think functional programming is completely inappropriate for general purpose programming tasks (e.g. writing a website, writing an iPhone app, etc.).
The ultimate criterion to judge a formal language (such as a programming language) is how easily anyone (regardless of expertise level!) can express their ideas in a precise and unambiguous fashion in it.
It isn't clear that this criterion favors any known programming style.
Edit: grammar
Of course leaning more math isn't nearly as hard as Chinese (for the non-Chinese), but this analogy isn't working in your favor.
I trained as a classical musician and when I picked up the guitar, I was too lazy to learn the notation and learnt the songs by ear and feel instead (this may be why my guitar is collecting dust in a corner). Conversely I know a lot of "amateur" musicians, some pretty advanced, who have learnt songs by ear and can't decipher musical notation of any kind, or understand any music theory.
Chinese looks complicated even to the Chinese, hence Simplified Chinese arriving in the 1950s (and according to Wikipedia, they've started another round of simplification in 2013). Without an alphabet, there are thousands of characters to learn (8,000 in the 2013 version). Ask any Western-born Chinese how fluent they feel and how easily they can read it...
But it's not complicated. One of the great things about mathematical expressions is that they are simple. But two of the other great things are that they are concise and unambiguous, and it's those two that unfortunately lead many students to think they are complicated.
A step by step procedure is exactly a mathematical process. Mathematical notation is just a shorthand for some procedures so you don't have to spell out every single step every time. You'd be well served by learning them!
In programming it's often the case that a shorter program is preferred to a longer.
In the productive world, an easier to understand program is better, even if that implies more code.
Variable names like "TotalAmount" are better than "x".
Also, a 10 line procedural code is better than a one-line function, just because is easier to understand, and easier to change in the right way when the specification change. (it will change 15 times before implementation)
More code in terms of code size in bytes isn't always more complex. For example, sometimes one wants to emphasize the similarities and analogies and spends extra bytes to show that. But that in my experience is rare - and while it could be beneficial for a novice or for a substantially unfamiliar code, for good code if soon becomes burdensome.
TotalAmount instead of x would probably drive crazy most mathematicians. Or most APL programmers. And probably most people demonstrating code on a whiteboard. Even in more mainstream languages like Clojure or Haskell a lot of variables are rather short; it's unfortunate that C++ with history of Hungarian notation, Java or C# have a habit to use long names.
There is a reason why mathematical notation exist. F = gmM / (R^2) is often preferable to "force is proportional to both masses and to inverse of square of the distance with proportionality coefficient gamma".
My advice for you is to try develop a program in REPL fashion. Standard J package from Jsoftware has a good environment for that - you can launch a computation expressed in a line, see the result and if you aren't satisfied just copy the line, make changes and launch it again. Pretty convenient in comparison to classical save-compile-run-checkResults loop of, say, C++ development. One line is certainly easier to change 15 times before implementation when specification change.
It can correlate with complexity positively and negatively.
Which one explanation below is easier to understand?
Photosynthesis? Or 'process used by plants and other organisms to convert light energy into chemical energy that can later be released to fuel the organisms' activities'?
You could argue that Photo+Synthesis might be enough to imply the essential underlying mechanism of the process itself, but I think more people would prefer the latter just for clarity.
It's a bummer to pose that question as though there isn't an answer, when the answer is well known and very important.
The motivation of avoiding side effects is because we humans make more mistakes when there are side effects. Functional programming is about making a piece of code not depend on anything outside itself. When you do that, it's easier to get right, and harder to screw up the code. In a word, it's safer.
> I’ll present you a view which, at least to me, makes that set of features seem coherent; and functional programming, like a deep and beautiful paradigm: Functional programming attempts to reduce programs to equations.
We also make more mistakes when we make things too abstract, remove specificity. Write code that is too concise, and you lose meaning, context, and readability.
The "functional python" is the best looking code in the article to my eyes. It explains what it does to the reader, semantically. The APL & J examples aren't very enticing to me. I'm curious about them, but I wouldn't want to work in a codebase where if you don't know what the name of the function means, you can't figure out what it does.
Making code mathier is good occasionally, when working on mathy problems, but generally I'd rather have words and const statements in my functional programs than a concise expression with single letter variables.
Is it possible that APL and J fail this test only because you (and I) don't know how to read APL or J? I think there are some languages that end up being difficult to read due to failings of the language, but I don't know enough to know if APL or J is among them.
Python is, perhaps, more readable than most languages, but I've seen people learning Python have a hard time with many of its choices, so we really have to consider whether we find some version of the code more readable only because we've read a lot more of that kind of code.
Most of the time "easy to use" just means "what I'm used to", and I don't think programming languages are an exception to that.
Yes and no. :) Its a great point; familiarity and practice is a big part of comfort and productivity, absolutely.
I love playing code golf in Python. You can make Python very concise, and very unreadable. But I don't write professional code that way, and I definitely don't have a goal of making production code as concise as possible. Conciseness is good for very small projects, for the 5 minutes while I'm working in the code. Conciseness is awesome for Project Euler. Too concise, and when I come back to my own code days/weeks/months later, I can't understand it at all. Anyone who's coded for long enough on big projects will know the feeling of coming back to clean, well commented, semantically explicit code years after writing it, only to have no clue what it's doing or how it works.
I also know how to read math, and math (typically... almost always) has this very problem. The conciseness is good for proofs and multi-step algebra, you need it while deriving things and working with the math. Once you codify it into a process, the conciseness is a big obstacle to understanding and code safety. I make lots of mistakes with math because it's too abstract. The mistakes are usually easy to fix because it's only a line or two at a time, but it'd be unworkable and unreadable if I had many thousands of lines of math the same way I have with regular code.
A mathematician's definition of a sorted list is different to a programmer's at least in part because a mathematician is interested in a sorted list and a programmer is interested in sorting a list. This difference is intrinsically invariant of the programming paradigm (though not of the context).
This issue isn't so much that this post didn't convince me that there is an underlying, semantically relevant sense in which the APL approach is more mathematical, but that it didn't even seem to notice that this was a thing that needed to be argued.
Programming is _always_ math; was it Dijkstra who said "programming is one of the hardest branches of applied mathematics"? It's also engineering - as in "engineers have to do some math occasionally, even if they don't realize that".
I often ask on the interviews to produce an expression returning sorted 3-element list using some simple primitives. I've found that with chosen primitives you still can model different approaches - and even though in all cases the result is an expression ("sorted list is..."), the structure of that expression can model imperative style ("first do this, then this...")
But unlike what this article suggests, these traits are not essential to functional programming. As long as you stick to functions which given values return other values, and your language has support for passing functions to other functions as well, you can reap most of the benefits of functional programming.
How about (.) (.) (.) (.) (.) (.)
I probably share your opinion on which is more readable :D
It does, we only need to change the syntax (semantics will stay the same), and the underlying mathematical structure will flourish before your eyes.
Hold on. Does this guy think "mathematical" just means "expressed as a string of symbols instead of words"?
The "function" in "functional programming" and the "function" in "avoidance or banning of functions with side-effects" are two different words, by the way! FP is named after the math word, not the programming language word, which means different things in different languages but usually means a kind of subroutine. FP means programming in a style that emphasizes composition of mathematical functions, and in an imperative language this generally amounts to writing your subroutines in a certain way. It just so happens that many imperative languages call subroutines "functions."
is the same as
(+/ % #) &. ^.
because adverb (/ in this case) binds with + before three verbs +/ , % and # are grouped into the fork. So some parentheses aren't necessary. It would be cleaner to show three expressions which share the common part as similar in letters as well.
It's obvious how to change the imperative code: `if (x[i] == 0) return 0;` What would this optimization look like in APL or J?
If you replace any of those numbers with 0, it will all collapse to 0 - perhaps not what you want. But it's not hard to write such a program:
geometric_mean1 =: (+/ % #) &. (^."1) @: (0&>.)
geometric_mean1 2 4 8
4
geometric_mean1 _2 4 8
0
geometric_mean2 =: (+/ % #) &. (^."1) @: (>. -)
geometric_mean2 _2 4 8
4
geometric_mean2 2 4 8
4
geometric_mean =: (+/ % #) &. (^."1)
Logic programming is close to this. In functional programming, left side of equation always consists of dependent variable and nothing more. Logic programming allows to put any variables on both sides of equation.
How do I learn/get/run APL or J? How do I even type the APL code given? What does the J code even mean?
As for the content...
"makes code super concise (less room for bugs)"
Unfortunately, while this sounds great, it does not logically follow. And while geometric mean is well-defined and understood, most functions I'm probably going to write in whichever language are going to be more complicated and less well-defined. So, how does APL/J/pure-functional help/make my non-trivial code more beautiful/concise/bug-free/provable?
They are unicode characters. Type them the way you would type any unicode character. If you have not yet found a practical way of doing it, there are plugins for editors to help you type APL characters with no hassle.
>How to learn APL?
There's a Dyalog APL manual which is quite clear, Google it. Also, you can use the GNU APL interpreter. I would recommend learning APL first, only by reading the Dyalog manual and writing programs with pencil an paper, then learn J in detail.
> How do I learn J?
Jsoftware provides a free interpreter. They also have a sleek interpreter app for Android. I would recommend to download this app and go through the project Euler problem set. Jsoftware provides all the documentation you will need in their webpage
http://code.jsoftware.com/wiki/System/Installation
> what good will J do for me?
The best would be to learn it and see it for yourself. Sadly, you find yourself in Paul Graham's Blob Paradox.I can't get you out.
> The best would be to learn it and see it for yourself.
I've found the statement on the front page of jsoftware.com to be rather accurate:
"If you are interested in programming solutions to challenging data processing problems, then the time you invest in learning J will be well spent."
And "data processing problems" could be understood pretty widely. I've had a case when I first spent about 45 minutes producing a prototype (a program which would generate realistic plans for floor of an office building) using J, then converted that prototype in functionally equivalent (no embellishments) program in C# spending a couple of hours. Certainly working on a prototype took several failed attempts, yet it was still faster than doing the same in C# after the algorithm was already clear.
Equations in lambda calculus do seem to have a "preferred direction" specified by the evaluation rules, though referential transparency lets you go "in the other direction" in your head, when reasoning about your program.
Perhaps the programs ~ equations idea would apply even better to logic programming, or term rewriting?
Functional programming is a way of organizing code as chains of functions...
I want to understand how FP replaces OOP in a similarly concise example. I have a sense that it involves a reconsideration of why OOP is useful, and solving that issue with a widely new approach, but I am not there yet myself. I’d love to see something like these Python comparisons but involving the demolition of an OOP implementation.
np.exp(np.mean(np.log(a[a>0])))
> reduce(lambda b, b: ab, 1, positives)
Should be
> reduce(lambda a, b: ab, 1, positives)
> Just in case all that wasn’t enough to convince you about the superiority of the functional gospel, take this threat: FP will displace OOP, so start thinking along our lines our you will eventually lose your job
OOP, especially with a focus on immutability, is not that far away from FP. It's not like, say physics where two domains can be conceptually quite different. OOP and FP are just two programming tools meaning that their variation is quite limited by the domain they are in.