Don’t Be Scared Of Functional Programming
smashingmagazine.com
smashingmagazine.com
Functional programming is great and feels right, I am amazed how knowledge of fp affects my js, C# and Python up to this day. But from a social perspective: Loose the smugness, and then we'll talk fp.
It takes me a few seconds to rewrite foldl from rote understanding: foldl f z (x:xs) = foldl f (f x) xs, because it takes a function, a 'zero', a list of head and tail, and calls itself again with the same function, a zero 'going up' (so accumulating) and a list 'going down' (being exhausted). So as you see this is an almost a visual way to understand it, which is natural to those that already are familiar with these abstractions, but is meaningless to those who don't.
If someone then says "how dare you ask that" then they have the wrong approach to things. Socrates asked every question and thought they were all valid. By asking the basics you learn the abstraction as if you're proving it to yourself, and that's more powerful.
So it boils down to familiarity, is what I'm trying to say. I usually come across as more passionate than arrogant about it. If you stick to it, it will make sense. It took me 4 years of Haskell exploration for it all to click really hard and now I'm trying to shorten other people's paths. In my opinion the gap is around mathematical abstractions, how they're created, what do they mean, how they actually relate to reality, and what work do they accomplish for us that we then don't need to do anymore.
I think this type of explanation falls into what the parent post is talking about. That is a technical accurate but very dry way to describe a construct that simply calls the function it's given for every element in a collection and returns the accumulation of the results. Granted you can dive into all sorts of discussions about what that _really_ means and what power it gives you, but save that for much later, it isn't introductory material.
An analogy to how functional programming often is taught would be if I tried to teach a new programming language by going over the BNF grammar first without context instead of starting with how the language is actually used in practice. It may be more 'technically correct' but it's a terribly ineffective way to learn for most people, even those who know and understand BNF. People are often more interested in what they can do or create with this new thing, not the nitty gritty details of how it works mathematically even if they intend to get into the details later.
Having said that, for an example of how I would actually explain some FP to someone, search this page for "DomainList" and I have an explanation there.
I agree with you 100%; my belief is that mathematics and its notation often gets introduced too early, before students have had a chance to build intuition.
I'm not saying that actual smugness doesn't exist, but almost all the time it's actually just projection from someone who's used to being in the know now struggling to understand something new.
Remember when you explained to someone using a language lower on the power spectrum (VB or PHP maybe) why they should use all the cool stuff that you can do in Ruby or Python, how much more productive they would be, how much more fun it was to work in? You thought you were helping them and showing your excitement, you wish someone had told you this a long time ago!
Unless you are a world class educator they were confused by some parts, threatened that their investment in their current language was losing value and worried they might not be able to function in this new world and therefore protected their ego by seeing you as smug.
Second: The blub paradox (at least as posed by Paul Graham) is wrong. To see why, just look at Lisp and Haskell. Users of both languages are sure that they're at the top of the power curve. They're sure that when they look at the other language, they're looking down, and they can tell you precisely why ("it doesn't even have macros" and "it doesn't even have a decent type system", respectively). But they can't both be looking down at each other - unless languages cannot actually all be ordered along a single axis called "power".
In fact, you have to ask "power for what"? Once you do, you might start to see language power more as a tree than as a single line. You can really only compare power between languages on the same branch. Pick the language that gets you the furthest in the direction that your problem lies.
Those two abstractions are several times more powerful than the overly constrained abstractions imposed by OOP. The power of lisp is that it is homoiconic, and it asks what it means to write programs that write programs. The power of Haskell is that it is algebraic, and it asks what it means to execute a mathematical proof.
Both therefore have the power to really abstract away from things in any way you want. They do not constrain the programmer with an abstraction that is too high and not composable, as are (mathematical) functions and macros (which are composable with hygiene). The prevailing paradigm offers poor substitutes for composability and can simulate very poorly abstractions that are lower than Objects (thereby needing design patterns to make up for starting so high in the abstraction tower).
Program power is not one dimensional, and not two dimensional. There are more "best" languages than Lisp and Haskell. There are more categories that matter than just programming in lists and programming in functions.
What I mean is the current mainstream languages are not the best of any category, like Haskell is up there for functional programming and lisp is up there for s-expressions. C++ is not the best for controlling details (see Rust, D, etc), in fact it's quite a broken language. I doubt Fortran is better at matrices than APL (in fact APL is arguably the best in the matrix category) and so on.
That's the problem. These languages that are best in their categories are many many times better than the mainstream languages of today, which aren't the best in any category they represent.
That's my point. It absolutely doesn't seem that way to me, because I know Haskell and Lisp.
I'm also personal friends with PHP developers who I used to work with and they have exactly the same attitude towards anyone talking about Ruby or Python or literally anything else.
They are angry at the bullshit that these smug language jerks keep throwing their way about how "bad" their perfectly good language is.
Then, there are other FP languages, which you can easily learn in a few days/weeks, very nice and natural, e.g. OCaml, or SML, or maybe Scheme(Clojure?).
Programming in PHP must have a similar effect.
It's too mainstream for me now, I'll have to switch to reactive programming.
[1] https://github.com/CrossEye/ramda
More info on the topic:
The book is also very good: https://leanpub.com/javascript-allonge
Personally, since I work in storage which is where we have to store all the "state" that the FP folks have defined into Somebody Else's Problem, I have little tolerance for that particular flavor of the month.
Still, even if you store your internal state, that's hardly what functional programming means when opposing state.
The answer is something along the lines of: the compiler is usually very different from most, it uses graph reduction and rewriting techniques, garbage collectors optimized for block reusage for all the heap churn that all those function frames would generate if implemented naively (that's why recursion ends up being as fast as a loop); this is my cursory understanding of it. But yes, I would suggest Lisp in Small Pieces and Appel's compiler books.
For practical purposes what is important is that FP is inviting you to program in this world where it is not important to talk about how those things end up getting implemented on the machine. In fact it lets you quickly escape that world whenever possible by allowing you to alias the String type to FirstName for instance; so you can talk about the problem you're trying to solve, and not the computer that will run it. That's the point of declarative programming and FP is declarative.
It's the same in Prolog, I'd think. I'm sure they don't care about how these things are actually being created and destroyed on the machine; the power of Prolog's declarative style is precisely that it allows you to never talk about the machine if you don't want to. That ends up being more general than the usual abstractions of the day that still model themselves after the turing machine.
There's no state to speak of. FP builds the blueprint of the program (one level removed from the way OOP and procedural see things) and gives it to the computer to run it. Now we make the execution order implicit (it's now implicitly threaded through the functions' caller-callee relationships) and we lose the ability to do state (because state depends on ordering things, which can be represented by the semicolon in curly-brace languages). We also lose the ability to do one thing after the other. But although it sounds like a bad thing, it's actually a really good thing, because apparently most bugs live inside semicolons.
Declarative programming is a gradient. You can be exposed to a bunch of tedious details in FP and you can abstract over them in OOP. Saying FP is somehow magically declarative (as if that were a thing) leads to profound disappointment when complexity is inevitably encountered.
> It's the same in Prolog, I'd think.
"Cut" it out :) Also see taming space leaks in Haskell.
> There's no state to speak of. FP builds the blueprint of the program (one level removed from the way OOP and procedural see things) and gives it to the computer to run it.
The primary difference between pure FP and OOP is identity: there is no identity to speak of, things don't have names or addresses, they are just values whose equality is determined purely by structure. The inability to name things in FP is a huge drawback, because things actually have names in the real world.
State doesn't depend on ordering. It depends on time; imperative and even functional languages often convolute the two concepts, but not always (see real FRP, not fake FRP ala Rx). Functional languages eschew state are merely eschewing the notion of time, which makes equational reasoning hard; it takes something like FRP to bring time back into the fold by making it explicit. Still, its quite hard to program interactive apps in pure FP given the intrinsic aversion to time.
You say this as if functional programmers somehow make your life harder... It's not like they're decrying the existence of persistent storage or trying to put you out of business :)
Also, functional programming isn't about offloading state, it's about removing it from programs altogether (in the sense of "state" being something that isn't explicitly passed through function arguments).
In reality, someone can take advantage of FP with absolutely no understanding of the mathematical underpinnings. I think that's what the author was trying to get across.
- Don't understand it - Are afraid of it - Find it too complicated
Just because I speak French and English fluently does not mean I can speak Japanese. My wife is Japanese, and speaks it fluently. We've been together 19 years. I've tried to learn. it just does not make sense. It would be nice, because I have relatives there, and I could go live there, legally, but it just doesn't make sense in my brain.
What can I do?
Functional programming is the same. Every couple of years I take a look, and I recoil in confusion.
It also doesn't help that people make fun of me for seeming dumb.
I want to write a program that reads one text file, and outputs another.
Let's say the first file contains a list of emails, one email per line.
The second file should contain the list of email domains in the first file, one line per email domain.
Procedurally, I would simply declare a list, read the file of emails, line by line, split each one on the @ symbol, and if the part on the right was not already on the list, I would add it to the list.
At the end, I would write the list to a new file, one line per item in the list.
How would one do that using functional programming?
We define what we have first (after all this is mathematics):
EmailList = a list of Email
Email = a string that goes "something" then "@" then "domain"
Now what we want:
DomainList, no duplicates.
Ok so no we're set up. We want a function that takes us from an EmailList to DomainList, as defined above. There are no objects so we simply say the relationships between things. I will introduce the tools we need:
map = takes a function and a list and applies that function to every element of the list, returning a new list.
We use map over EmailList with the function extractDomain which we will write later (it's trivial to do without mutation so I won't show here).
So now we have a list of domains, but there might be duplicates.
So now I have DomainList(with duplicates) and I want DomainList(without duplicates), so I want a function from DomainList to DomainList.
Well, what's the difference between what I have and what I want? What I have has too much, and I want less (ie. the list I have has too many domains (duplicates) and I want fewer (unique)).
filter = takes a list and a function. That function it takes, that you will supply to filter, we'll call f. Function f takes one list element (so filter will call f for every element) and it must return a boolean to mean whether that element should go into the new list, or if it should not go into the list.
So we could use filter to go from a DomainList with too much, to a DomainList with fewer things. However each time our function f is called, we don't know if this function has already been called before with that same domain; and because we don't have state to change a variable to keep a tally, it seems impossible.
So let's approach the problem in a different way. One thing we could do is sort the list, then group adjacent similar items, then map over the list taking the first element of each item. But we won't do that.
We need something more generic than filter. Let's look at fold.
fold = takes a list, a function f, and a starting point z (that we call 'zero'). For each element of the list, it calls your supplied function f with the current element and the zero; but then, it would be useless because it would be the same zero all the time. So what fold does, to be useful, is that whatever the result of calling your function f with a list item and the zero is, THAT value (so the value the call to yourFunctionF(list_item, zero) returns) will be the next 'zero'. So now we have a little machine called fold that can thread something we have through its inner mechanism and play along with us.
Using that we can say the zero is the list of elements we've seen so far.
So now our DomainList -> DomainList function will be a fold. That fold will take a the DomainList we have, and a zero that will be the empty list (to start with!) and a function f. That function f will simply check if the item is inside zero. If it is, then simply return zero (because that will be the zero for the next call to f!); if it's not, then return a new zero with that item added to it.
Now we've achieved the goal without any mutation (ie. without us talking/concerning ourselves with any mutation. of course there's lots of mutation going on in RAM but the point is not to eliminate that!)
Does that help?
It's like I asked for a tuna sandwich and you explained how to build the engine of the fishing boat to go catch the tuna off the coast of Baja.
I feel like I need a masters in computer science to understand what you wrote.
let me see:
# Python:
domains = []
emails = open("file_with_emails.txt","r").readlines()
for email in emails:
____domain = email.split("@")[1]
____if domain not in domains:
________domains.append(domain)
out = open("file_with_domains.txt","w")
for domain in domains:
____out.write(domain)
out.close()
input:
joe@msn.com
mike@aol.com
tim@aol.com
sam@hotmail.com
output:
msn.com
aol.com
hotmail.com
Simple no?
For instance, what does = mean between domain and email.split ? Who is equal to who? That question is a question that students sometimes have. You have to explain that Time is implicit in the code, and that an equal sign works by making the thing on the left equal to what the thing on the right was just before this line. So now there's a concept of Time and that's how things start to get very complicated, because it is implicit in your code.
Then, what does ".append" mean? I mean, you call append, but now you don't assign the result to any variable? And what would it mean if you said domains = domains.append(domain), then we really need to have a talk about the hidden Time.
So it only seems simple because it's familiar to you; so in fact I believe it's not simple at all.
But this I believe it's simple, and my fishing boat engine above would've benefited from this:
https://www.fpcomplete.com/tutorial-preview/4217/4k7kq6jGUE
If you look at that code (you can also click play to run it) and then look at the explanation again, could you please tell me what doesn't seem simple?
Edit: let's not forget your Python example uses a map, disguised as a for loop. That for is a keyword, but backstage it's just map where email is the element, emails is the list, and everything you write inside that scope is the function that gets passed the element each time (except in your case the function (i.e. the loop body) can also see the whole list, which is prone to bugs).
You're right though that my using python language features like lists and file open and loops is like function, but I understand those, so I can use them.
Mmmm, really? A map? Like Google maps? Like a paper map? unfamiliar terminology.
Seriously, I learned how to use for loops in 1984. Been using them ever since. Maybe that's the problem. Too old to learn new tricks.
Oh, but the moral of the story is that because the computer understands those maps, then I can simply tell it what I have (point A, origin) and what I want (point B, destination) without telling it how to do it. Then the computer goes and writes that code for me! The same code you're now still writing manually. Because you don't want to learn a new "trick". :)
No, a map like the Python built-in function "map". Though, really, the for loop in your Python sample isn't a map done as a for loop, its a reduce done as a for loop. (Again, like the Python built-in "reduce".)
Reduce (also frequently called "fold") turns a list of items into a single item (of maybe the same type or maybe a different type). It does this based on a function that tells you how to update that single item as it looks at each list item in turn.
For instance:
def fn(a,b):
return str(b) + a + str(b)
reduce(fn, [1,2,3], "0")
Start with "0" (we call this the accumulator value)
Read the first element of the list (1)
Pass "0" and 1 to fn
Turn 1 into "1"
Stick it on the front and back of "0"
Return "101".
Use "101" as our new accumulator value.
Read the next element of the list (2)
Pass "101" and 2 to fn
Turn 2 into "2"
Stick it on the front and back of "101"
Return "21012".
Use "21012" as our new accumulator value.
Read the next element of the list (3)
Pass "21012" and 3 to fn
Turn 3 into "3"
Stick it on the front and back of "21012"
Return "3210123".
Use "3210123" as our new accumulator value.
Notice that we've reached the end of the list, return our latest accumulator value.You can think of the body of a for loop in python as the body of a function. In that case, a for loop is like the built-in map function, which applies that function to every element of the list. So convert your for loop into:
def body(email):
domain = email.split("@")[1]
if domain not in domains:
domains.append(domain)
domains = []
map(body, emails)
def body(domains, email):
domain = email.split("@")[1]
if domain not in domains:
return domains + [domain]
else:
return domains
domains = reduce(body, emails, [])
def reduce(func, l, currentResult):
for item in list:
currentResult = func(currentResult, item)
return currentResult
def reduce(func, l, currentValue):
if len(l) == 0:
return currentValue
else:
return reduce(func, l[1:], func(currentValue, l[0]))
domains = set( email.split('@')[-1] for email in open(path) )
I can't do the mental gymnastics to think about the code that way.
When I think about the purpose of the code, the result of the code, before the code has been written, I do this by "compiling and running" in my head. Of course, there's no actual compiling. There's just thinking.
When I reason about the code, either code I'm going to write, code I wrote, or code other people wrote, I execute the code in my head.
I'm currently building a web-scraping system, to look information on partner web site. I'm using python 3, selenium, pyodbc to sql server. I'm also developing the same in C# (3.5) using the webbrowser component in systems.forms. It's got a lot of moving parts, lots of languages involved. I read the documentation for the tools, then I assemble in my head, essentially "running the code" in my head, and then, once all the edge cases are dealt with (are there iframes, popups, ajax async calls, flash and pdfs) then I design on paper, with boxes, and then, later, later, if it all works in my head, then I code it.
Can't do that if I can't reason in my head about what the program is going to do, exactly.
Functional Programming, for me, does not fit in my mental model, so I can't use it, regardless of actual implementation.
http://qt-project.org/doc/qt-4.8/widgets-analogclock.html
When you could instead say just what you mean in functional programming:
http://elm-lang.org/edit/examples/Intermediate/Clock.elm
The most important skill for a programmer is to be able to learn new things. Especially when they're there to help you.
addEmail = lambda |doms,email| doms.union([email.split("@")[1]])
domains = reduce(set(),addEmail,open("file_with_emails.txt","r")))
open("file_with_domains.txt","w").writelines(domains)
And, more how I'd really write it in Python (which is also functional, though perhaps less obviously so, and leverages Python set comprehensions):
domains = { email.split("@")[1] for email in open("file_with_emails.txt","r") }
open("file_with_domains.txt","w").writelines(domains)That is actually a really good compromise and a style I tend to stick to when coding Ruby/Rails.
(ns my.namespace
(:require [clojure.string :as str]))
(->> (str/split (slurp "/home/deadghost/fake-emails") #"\n")
(map #(second (str/split % #"@")))
set
(str/join "\n")
(spit "/home/deadghost/new-file"))
Clear as mud?
say ~ m/ '@' (.*) / for lines
Note that lists (`lines` returns a list) are lazy by default in P6. So the above code will start producing results immediately and continue without exhausting RAM even if the input is infinite.
The `/ ... /` bit above is a "regex". It means match the symbol '@' and return all the following characters of that line. P6 regexes are far more powerful than P5 regexes. P6 regexes can work together to comprise arbitrarily complex parsing grammars. The main P6 compiler, Rakudo, parses input source code using a P6 grammar.
There are of course many more ways:
say lines.map: { m/ '@' /; $/.postmatch ~ "\n" }
The fun part is that many languages end up aiming for it. Good multithreaded/reentrant code is almost functional. I just saw a ruby rant asking to avoid mutation as much as possible. Same goes for PHP PSR-x ...
Functional programming suffers from acute mathematitis, the people behind it are fond of iterating too fast around their newfound concepts and it end up as cryptic as possible.
Then I had a class where we learned Haskell with one of the best professors in the department. He basically taught me all of the great things about functional programming and now I try to use concepts from that class in whatever language I'm programming in.
Learning functional programming is paradigm-shifting and gives you a much broader set of tools to work with, but if it's not taught well it won't help you.
OOP just feel more eloquent to look at personally.
Struct.function(arg1, arg2, arg3);
function(struct, arg1, arg2, arg3);
fn close<T>(&:<T>){
if(<T>::isType(tcpConnection)){
tcpConnectionClose(<T>);
}else if(<T>::isType(serialConnection)){
serialConnectionClose(<T>);
}else{
//throw exception
}
}
tcpConnectionClose(*);
http://clojure.org/multimethods
I encountered this recently but I'll try to give a (bad) explanation.
You'll call `function(thing, arg1, arg2, arg3);`. Another function will run on the arguments and return a dispatch value. For example, it will check what `thing` is and return `struct`. The `struct` version of that function is then run on the args and gives you your value.
In this way you can define several `close` functions based on dispatch value instead of one monolith nested if/else `close`.
I'm on the opposite side of the fence, OOP has never ever appealed to me. "Why would anyone want to use this crazy mess?" kind of thing. I'm sure I can learn to appreciate it with time but it's not a native paradigm to my mind.
I only have experience with lisps when it comes to functional programming but the reason I enjoy it is that it's simple. You have functions and you have data. Functions transform data to more data and the two aren't tightly bound. If you pass a function a value it will always return the same result if you pass it the same value.
THE clojure video: http://www.infoq.com/presentations/Simple-Made-Easy (I don't think he even says the word clojure in the entire hour presentation.)
> letting the compiler figure out which one to call when and obviating the runtime exception entirely.
That is how FP eliminates a whole category of bugs. You have to have "experience", which some love to boast, to know that it's better to have an else clause there and throw an exception.
If you think that's basic knowledge you'd be surprised with how little do "experienced" programmers that get paid very handsomely actually know; I've met 30-year of experience programmers that do not believe that the final-else-then-exception pattern is a good idea, because they think "crashing the program is never good". So what ends up happening is of course the program ends up crashing anyway (supposing you have passed an invalid connection in that case) but now the stack trace is potentially much deeper than it needs to be, and therefore misleading. All stack traces lose their value collectively because now we can't tell the good ones from the bad. That can make debugging harder than it needs to be.
But instead in FP, because it's a fundamentally different way to program, that situation doesn't even arise in the first place, obviating a solution. In FP things are modeled differently from OO, it's not just "procedures without an object jacket". The advantage here is the compiler verifies the "object" selection for you (which "class"/type of connection) and can provide some mathematical guarantees. Those guarantees are what metonymically "eliminate bugs".
So we can rely less on the programmer and more on the compiler. You know, like it's meant to be. After all we are sitting in front of a computer, so we should just let it do its thing without telling it how. Turns out autonomy in coming up with a solution is good both for the programmer and for the computer (and then again for the programmer).
FP has nothing to do with procedural programming at all, it is fundamentally different and diametrically opposed to both procedural and object oriented programming, which are then both in the same group away from FP.
You should look up some tutorials on FP if you want to understand what it means in practice.
Well, that's not helping. But I agree that devs should learn FP, and I now write in a much more functional style than I used to. I resisted for a long time, and that was a mistake.
I personally find this view a little overbearing.
In "lay programmer's" terms, functions as first-class objects can often come down to being able to pass functions to other functions or function composition. Most of us learned about function composition in Algebra 2... So that's pretty straightforward even at its worst.
Eliminating side effects is equally straightforward in that all it means is that any variables outside the scope of a given function are not changed by the function.
def a(b):
m = 0
return b(m)
def a(b):
m = b(1)
return m+7
compose : (a -> b) -> (b -> c) -> (a -> c)
a `compose` b = \x -> b(a(x))
partial3 : (Integer -> Integer -> Integer) -> (Integer -> Integer)
partial3 f = f 3
map : [a] -> (a -> b) -> [b]
map [] f = []
map (x:xs) f = (f x):(map xs f)
compose :: (b -> c) -> (a -> b) -> a -> c
compose f g x = f (g x)
You will know if there's state in a language if you can build a function that returns different results given the same values over time; IORef does not allow you to do that.
As to your broader claim here, I while I understand what you're getting at (and probably wouldn't have bothered responding, but for the above) I think it's most precise to say that IORef is an encoding of state. This does not have to mean that state gets updated by means of side effects.