Stop writing lambda expressions in Python
treyhunner.com
treyhunner.com
> 1. The operation you’re doing is trivial: the function doesn’t deserve a name
Sure.
> 2. Having a lambda expression makes your code more understandable than the function names you can think of
This is just vague. Locally defining a function right where you need it makes code much more understandable to me more often than the wacky names people often come up with. Naming things is hard!
> 3. You’re pretty sure there’s not already a function that does what you’re looking for
So the reader of the code has to know what that built-in function does. How is that much different than expecting the reader to know what a lambda is?
> 4. Everyone on your team understands lambda expressions and you’ve all agreed to use them
Or just use them and have the members of your team better themselves by learning a useful language feature.
---
In my experience, names lie--functions don't. If a developer isn't going to spend the time to write a nice comment, they aren't going to use a comprehensible naming convention. In trivial cases where naming is easy, the lambda is also usually very easy to parse.
This only might be valid if people on the team weren't programmers, but rather researchers that had to use python. In that case, the code would have to be very easy to understand. But in that case, they should probably learn what a lambda function is. It's really not that hard
import operator
sort(values, key=operator.itemgetter(2))
sort(values, key=lambda x: x[2]) from operator import itemgetter
sort(values, key=itemgetter(2))
To me it depends on what I'm writing. If it's a library where performance will matter then it's the itemgetter approach. If it's a script it will be the lambda.
For this reason, I don't really mind it. It's a sort of cue to the reader about the style of programming that will follow.
To be clear, what Python does need is multiline, inline anonymous functions. In JavaScript/ES2015, the fat arrow function syntax is just remarkably powerful in its ability to simplify code. I would go so far as to say that the ES2015 fat arrow syntax completely changed the way my programs are written, increasing power and reducing complexity. ES2015 fat arrows I think are possible the best programming language feature I know.
async programming in JavaScript benefits massively from the beautifully terse and powerful anonymous functions in ES2015. Python now has powerful async programming, but no equivalent to the ES2015 terse inline function anonymous function syntax.
I raised this issue elsewhere once and someone said "there will never be multiline, inline anonymous functions in Python because it would require brackets". If so then it is a great pity that Python is simply not capable of including a language feature that is IMO absolutely critical.
Also, as mentioned elsewhere in this thread, using the word "lambda" to describe anonymous functions is a really really bad decision. Lambda sounds very deep, very computer sciencey and very complex. Probably they should just be called anonymous functions in keeping with the rest of the industry, which also is not a great name but is much better than "Lambda" functions.
Beginners might think along these lines: "Lambda .... lambda calculus? I don't know calculus. I'm outta here."
Names matter.
Which is pretty much what you get in Python.
But most Lisps are pretty rough-and-ready about these sorts of things. IIRC, Scheme requires the body of the lambda to be a single expression, doesn't it? And then you just cheat your way out of it with (begin ...).
No, it doesn't require it.
R1RS does not permit it. And: In the the R1RS syntax BNF there is a (DEFINE (<identifier> <identifier list>) <form>) with one form. But later in the document it is a '<form list>'... Then the LAMBDA syntax has a BODY, but it is only a FORM...
But they have some important difference:
1) Anonymous delegates (introduced in C# 2.0) allow you to elide the formal parameters when defining a method of a given type if the body of the method doesn't use those parameters. Even though this construct is basically obsolete, I still use it just for this feature.
2) Lambda expressions can be automatically transformed into expression trees, which means that you can use them to construct (awesomely powerful) fexprs[1]. But, they can't contain statements, which means you can't use them for any kind of imperative code. These are the most equivalent to Python's lambda as far as that limitation is concerned.
3) Statement-bodied Lambdas can contain statements (as the name implies), but the language won't convert them into expression trees for (you can do it manually – tedious but not at all hard). At the time that the lambda expression feature was introduced (as a member of the constellation of features that made up LINQ), the framework didn't even contain expression classes that corresponded to most of the imperative constructs, but that changed with the advent of C# 4.0 which implemented a meta-object protocol[2].
Another implementation detail (last time I checked, which has been a while) is that anonymous delegates get turned into instances of System.Multicast delegate, whereas a lambda expression either gets turned into a class with a method that corresponds to the lambda expression (and fields that correspond to closed over variables), or into an expression tree, depending on the class of the variable to which it's being assigned.
When you name and define a function some other place and use it elsewhere, you have increased the overall complexity. What is this named function? Where is it used? Is it accessed by other bits of code? Should it be? All these questions come up when you see a named function. It is in fact less readable and requires more code to define a named function elsewhere and then call it. Inline terse anonymous functions are more straightforward, self explanatory and simple.
It is often the case that you know an anonymous function will only ever be used once, in this particular bit of code so it is much more clean, readable and understandable if the anonymous function lives right there.
Especially valuable in reducing the complexity of async programming.
It's hard to make a strong case just in words, but once you really understand the power of the fat arrow syntax for anonymous inline functions then you use it more and more and it becomes second nature and the programs you write have a completely different style, oriented towards neatly positioned inline anonymous functions everywhere that typically can be read and understood at a glance as you skip through the code.
There is simply no other way to write a function that absolutely cannot be called by some other bit of code. That's the problem with functions that are defined outside their usage context - you may know for sure that the function will only ever be called from this one point in your code, but if you have to declare it as a named function elsewhere then there is always the possibility that in the future someone will come along and make use of that function - it is more complex in the immediate term and leaves open the door for increasingly code complexity in the longer term. Terse inline anonymous functions such as the fat arrow syntax solve this perfectly. They are so clear and easy to understand that it's actually rather beautiful.
This is how terse you can make a JavaScript fat arrow function:
x => x
message => console.log('new message: ', message)
> It's hard to make a strong case just in words, but once you really understand the power of the fat arrow syntax for anonymous inline functions then you use it more and more and it becomes second nature and the programs you write have a completely different style, oriented towards neatly positioned inline anonymous functions everywhere that typically can be read and understood at a glance as you skip through the code.
This is exactly why I think Python has such limited anonymous functions: because it does not want to promote that style.
> There is simply no other way to write a function that absolutely cannot be called by some other bit of code.
Python also believes that you do not need to defend against this possibility, as long as you make it clear to other developers that they should not call your "private function". This is also why the language allows monkey patching anything at will. If someone wants to ignore good sense, let them.
>>because it does not want to promote that style.
Why would Python not want to promote a more readable, simple, powerful, reliable and maintainable programming style?
Can you reference anything to back this assertion?
Indeed naming my functions resulted in more testable and clearer code.
Can you explain more about how async/await obviates the need for fat arrows?
You don't need callbacks with async/await. No callbacks, no need for fat arrow.
await (async function () {
//Do a thing
}());
await request_some_data()
await (async () => {
// just make the async request inline
})()Otherwise, this is just restating the same arguments people already made above.
There's no practical reason why
(() => {
}());
await (async () => {
}());
99% of the time in places where you might use a multi-line anonymous function you're already inside another function
So when you define your named function in Python, it is only available within the scope of the current method call and there's no danger of it being abused by other developers.
The lack of true private vars in python is a separate issue, and one of the best things about the language (because library authors inevitably get carried away and mark things as private which don't need to be - then you're stuck in copy & paste land... I did loads of ActionScript back in the day and this was super frustrating)
Python has apparently never worked in a large, corporate environment with a multinational team.
Yes, in theory you can just not call private methods. And in my own personal code I don't care much about restricting variable access, because I trust myself. However, it is a tremendous amount of work to get a company culture to start respecting privates if they aren't already. I've had so many phone calls trying to explain why "yes, you can technically call this function, but in reality you can't, and I don't care that your deadline is tomorrow, you still can't. And yes, I know that technically you're on a separate team and not under my jurisdiction, but you can't just remove me from the code review and call all of my library's private functions anyway."
It's taking the path of most resistance. True privates are very helpful on large teams.
It makes no sense at all to break up the wonderful conceptual flow of functional components like map, fmap, filters, folds, monadic operations, with sudden harsh definition of a function that takes your brain out of the context of what was flowing and into the context of what is the function.
I want to see something like
val someStat = myData.groupBy(thingExtractor)
.foldLeft(baselineValue)(statAccumulator)
I want to read code like an essay. I don't want to have to jump to a function definition to figure out what `thingExtractor` does. To me, that's like taking a book and rearranging the chapters in alphabetical order. No, put them in chronological order. Don't define a function three "chapters" ago in your code and then expect me to remember it if you're only going to use it once. I wouldn't read Ikea instructions that way, why should I read code that way?
But again, personal preference. And you don't have to choose either/or -- I love that Javascript's anonymous and named functions are so similar because it means you can switch between the two styles with basically zero cognitive overhead.
I will say, having worked in large enterprise before, I have seen bugs come out of people not inlining code and instead attaching it to classes or leaving it otherwise accessible. Specifically, people will use functions that are intended to be method-specific in other places. Even if those functions are well-written and don't have side-effects (which is often not the case), the original author still doesn't know that other people are now depending on them as an API.
So later on the code gets refactored and those methods get changed or removed, and suddenly you have a broken build. That's something you can partially mitigate by making a function private (although not entirely, because I've also seen it happen in code within the same class/scope). Python doesn't have true privates, but you can also mitigate that problem by just using code reviews to force people to respect `_`. In practice, I often found that it was easier to make the function anonymous, so everyone knew 100% that it was only being used in exactly one place.
Many bugs have to do with utilizing closures to access variables needed in the function body, which then make refactoring harder and make modifying unit tests harder.
This can be even worse in languages like Python where there is a distinction between early binding and late binding and you have to be aware if by closure you’re using a reference name that may be associated with different data during its lifetime, or a name whose value won’t change, because a change to the underlying value could make a difference in what is bound inside the anonymous function at different times when it’s called. The classic example is trying to define functions in a loop where functions use the loop variables by closure. Then being surprised when every one of the functions has a reference to only the final value of the loop variable.
Even in statically typed languages, this makes things much harder to reason about than they should be. On the other hand, making an anonymous function that accepts many arguments for all the data it would try to access by closure is stupid: those arguments need to be documented, and it’s just so much cleaner and maintainable to do that with a regular function definition, which also makes it much clearer what all the conditions are for calling the function.
What’s worse is that these things can be deeply welded into some coding context, like using a flatMap over some TypedPipe in Scala / Scalding, and can result in needing to make in-line functions that are hundreds of lines long with arbitrarily complex function bodies, which then become tied into assumptions about the runtime context you’re embedded in, and then nobody can figure out how to refactor it into a standalone function, so it just grows by attrition to an inline lambda over years and is extremely fragile. Change something seemingly unrelated about the outer context it’s defined in, and suddenly you get unexpected, cryptic compiler errors complaining something’s wrong with the TypedPipe, and you have to dig deeper to understand why it’s related to the anonymous function.
I would say many of the most serious closure-related bugs and bugs related to unrefactorable yet undocumented dependence on an enclosing context that I’ve seen have been largely a direct result of the programming style of in-lining anonymous functions inside functional programming constructs.
I sympathize with your claim of “not reading an essay” especially because people can be prone to try to use functional programming or overloading the Python data model and operator syntax with cutesy bullshit that they try to pass off as expressiveness.
But I think there’s a middle ground where you think about it not like an essay, but just basic modularity and separation of concerns, and write functions separately except when they are very short and really trivial.
I often have to fight to get people not to expose huge amounts of state whenever they build a module, so by far the most common bugs that I see are people being too loose with turning things that really shouldn't be modules into very fragile, cumbersome modules that depend on being used only in specific (undocumented) places with specific (undocumented) setup.
If I was in the opposite situation, and everyone I worked with already inlined code all the time, then probably most of the bugs I'd see would be related to people reusing variables, abusing hoisting by making spaghetti references to variables that are defined later in the function, etc... and in that case I could definitely see myself agreeing with you.
I have on occasion wanted the ability to define an anonymous function that didn't inherit variables from the scope that it was defined in. So I'll give you that - I would love for the ability to make an anonymous function that only has access to variables that are explicitly passed in. If I could isolate variables going into a closure as easily as I can isolate variables going out, I suspect a lot of the problems you're talking about would be easy to solve.
I’m not sure how we could objectively decide if either of these two approaches is definitively better, but in the specific, restricted case of a framework like Scalding, I’d strongly wager that avoiding in-lining ends up better in the long run. Those cases also have little connection to the weak module design issue you brought up, since it’s usually a module with de facto map reduce boiler plate and then just a few isolated places with any actual implementation, and when those parts are expressed as huge in-lined anonymous functions inside Scalding data type wrappers, I know right away it’s a bad code smell.
The correct approach might just be the opposite of whatever you and your team's predilection is. In your case, you're saying that the teams you work with are using inline functions instead of following a restricted framework, in part because they're using languages that encourage them to just slap a bunch of nested code in instead of writing out the extra boilerplate.
Well, they probably already know to be careful about module design -- so if you encourage them to inline less code, odds are pretty good you won't suddenly wake up in the morning with a codebase with a hundred classes and a bunch of obscure private/public methods named `setupEntityExtractorForCollisionPart3`.
On the other hand, if your team is coming from a Java background and half of them are starting from the position that lambdas are just witchcraft, then it's probably not a bad idea to get them over that fear.
In a setting where everyone is inlining most of their code, probably the tests that are coming out are all integration tests, so... yeah, bias towards creating units so you can unit test. In the opposite situation, I'm actually just trying to get people to stop testing private methods and leaking implementation details into their tests. So I would love if people were testing with a little less granularity.
I know that my initial reaction to you listing off the problems you've run into with people building anonymous functions that couldn't be refactored was just, "yeah, but why the heck would anyone make that mistake? How hard is it to organize the variables in one function?" So I assume that other people might listen to my complaints about improper code reuse and think, "yeah, but why the heck would anyone ever just reuse a method in a class without checking the documentation first?" So my takeaway from that is, "different people struggle with different things."
var x = function(x){ return x };- it can never be named
- it inherits `this` lexically from its parent scope (which imo is the sane default)
[0] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
you just have to visualize it as a tree
I don't understand. It's harder to flatten the tree: you have to unflatten it in your mind afterwards to understand what's happening. It's easier to just visualize, say, this lisp function as a tree directly.
(defun good-enough-p (guess x)
(format t "~% Guess =~7,4f Guess^2 = ~7,4f Error= ~7,4f" guess
(* guess guess) (abs (- (* guess guess) x)))
(< (abs (- (* guess guess) x)) .001))They can read the value of local variables (including arguments) in the parent function, and write to them (using the "nonlocal" statement). If one nested function assigns a new value to a captured variable then, of course, this change is seen by all other nested functions and the enclosing function. Most importantly, if the lifetime of the nested function object is longer than the runtime of the enclosing function (most commonly because you return the nested function object from the enclosing function) then the lifetime of the enclosing function's local variables is extended appropriately.
This applies to both nested functions defined using a "def" statement and nested functions defined using a "lambda" expression (except you can't assign to variables in a lambda expression). I say this because you don't say what you mean by "they", and it's a common misconception that only lambda expressions capture enclosing variables.
The only disadvantage of this is that, because name lookup is done at execution time rather than parsing time, all the enclosing variables must be captured and their lifetimes extended even if the nested function doesn't use any of them. In this sense, nested functions are too closurey in Python! This is not really so bad because if you don't need variable capture then you should simply not be using a nested function in the first place; this is best anyway because it means you are sharing one function object between different uses.
Lambda expressions look in python like a sore spot, because Python stresses so much statements, and procedural programming.
IIRC, recently we've seen passing a paper in hacker news, that let beotians write programs, and only ~10% IIRC (or less) of their uterances were procedural statements!
Are there any resources explaining this which you would recommend?
Statements can also be combined, but in a separate way. You have control flow blocks, and inside of those you have one serial list of statements. Control flow blocks and statements can only go in other control flow blocks, and not in expressions.
If you’re coming from languages like C or Python, that might not seem like a huge deal. Why would you want “x = 5” or “while y < 10:” in an expression?
Languages that focus more on expressions have surprisingly simple ways of expressing some things, though.
In Rust, “if” statements are fully expressions, so you can assign them:
let x = if y > 10 {
let a = readLine().parse();
let b = readLine().parse();
a + b
} else {
y * 2
}
if result is successful {
result.value
} else {
return result.error
}
If you wanna get really fancy, languages like Haskell and Elm don’t have side effects, so things like “x = 5” don’t exist. There are only expressions. You can create values that represent doing something (kind of like Redux actions), and there are lots of tools for combining those action values. Because it’s all expressions, you can combine them arbitrarily. If you have a common case for how you combine your effects, you can just make a plain old function for it, because it’s all just expressions.
JavaScript has things like TypeScript (and Babel in general) that add features that are "missing" from the language. Is there anything similar for Python and anonymous functions?
The braces also makes it easier to find the start and end of a anonymous function even if it's just a one liner, compared to pythons where its only delimited by : and ,
the fact that creating a function with lambda versus the normal way is different is bonkers. for example, in f# (and other sane languages), the following are identical:
let test1 = fun x -> x * x
let test2 x = x * x
x:int -> int
also, you can't pass operators as functions in python? in f#, you simply wrap them in parentheses.
let test3 f x y = f x y
> test3 (*) 2 3 ;;
val it : int = 6
edit: also, this guy's thoughts on map and filter, in my opinion, show the python community's backwoods thinking regarding functional programming. yes, it seems generator expressions are nice (they are called sequence expressions in f#), but map and filter, even for his simple examples are much more clear. they communicate better what is actually happening. the fact that he never uses map and filter at all and then goes on to basically say map and filter aren't even needed in python says a lot.
It seems like a language designed for people who really have no interest in craft of programming, and just use it as a means to an end (note I'm not saying there's anything wrong with this, but if your career is in programming, and you keep that attitude, I'd suggest you find another one or move to management, you're not gonna be happy).
And finding new more effective ways to do common things, new tricks so to speak, is important in keeping the mind engaged.
The tools might not be as important as the end product, but for the person who is always using those tools, being able to improve and customize them to their liking is extremely important.
I guess the difference is a programming language is less a tool than an end product in itself, that needs to be able to be understood and modified by other people. As opposed to say, a plumbers preferred toolset, of which no trace will be left when the next person comes to do the job.
It's kind of funny how much that echos the past criticisms of Java - a language lacking in all kinds of features that frustrate advanced programmers. Yet Python's popularity was in many ways a reaction against Java - perhaps for other reasons.
That's what python is doing and why the two are different.
(I don't think a language should track that sort of thing, but that's a different story).
val test1 : x:int -> int
val test2 : x:int -> int
Does that mean that test1 == test2 will return true?
Personally, I have found it very useful for debugging that function objects in Python carry their name. That makes it much easier to find out where something is coming from.
--------------------
Edit: Apparently, there are some people who do like Python the language, and some people who dislike Python the ecosystem. If you want to find a thing, just say it doesn't exist, and it will find you.
but yes, there are lots of libraries. i think people need to take a stance and simply pull those libraries into other languages (which many are already) or simply realize that most are just wrappers around some other language library, which can be easily done in other languages as well.
Don't get me wrong, there's a huge number of packages available and a lot of them are really quality and I have used the language a lot, but good gosh is the packaging a deployment a fucking disaster.
Environments are a 3rd party bolt on, that relies on hacking around with environment variables. There's no distinction between dev dependencies and actual, application dependencies. The packaging/deployment tool has gone through a bunch of different formats/styles and when it comes to deploying your code, you have nothing better than writing a bash script that automates your environment creation, activation and running of the packaging tool (you then just hope that Pip doesn't fail on some bizarre edge).
I beg to differ. I like Python the language. I also like Python the ecosystem.
But the point is, even with f# syntax, most of the blog post would still apply (needless function calls, non-trivial functions, multiline-unpacking and so on)
I do not agree with some of the premises of this post, but saying that changing lambda syntax will somehow help is just not true.
i wasn't really suggesting changing the syntax would help (although the confusion here is more semantic than syntactic). i was pointing out that in other languages, like in f#, things are much more clear than in python. the section mentioning the name thing is still not clear to me (in python) in terms of what exactly the differences are, and that was my point. that exists a lot in python.
Here are some reasons why I like it:
It's cleaner to read because there aren't curly braces and semicolons cluttering up the code.
It's cleaner to write because the expressions are simpler and look like something readable instead of line noise.
It has a REPL, so it's easy to experiment.
It doesn't force me to write boilerplate code, such as types for every variable declaration or everything having to be a class or huge amounts of scaffolding just to write "Hello, world!".
I'm sure I could think of more, but those are what come to mind on the spur of the moment.
for example, in all three languages above, expressions are much easier to understand because they always return a value. this makes code easier to reason about. this is not true in python.
the languages i mentioned do have static typing but they have type inference. so you get the best of both worlds. you do need to type annotate at some points to clarify things, but this isn't a problem as it documents to both the user and compiler.
given your above reasons, you owe it to yourself to learn an ml. there is the programming languages course on coursera by dan grossman, which uses sml in the first part, and the ocaml mooc just started where you still have time to register and complete it.
https://www.coursera.org/learn/programming-languages
https://www.fun-mooc.fr/courses/course-v1:parisdiderot+56002...
F# has curly braces. Also, as you say, it requires some type annotations.
I see that F# has a REPL, but it seems bolted on as an afterthought instead of being an integral part of the language. (Also, it's not clear whether the REPL is available in Mono on Linux. I am allergic to Windows and anything built by Microsoft.)
> expressions are much easier to understand because they always return a value. this makes code easier to reason about. this is not true in python
Huh? Python expressions always return a value.
Perhaps what you mean is that Python also has statements, which do not return a value, whereas these languages do not--everything is an expression, as in Lisp?
> you owe it to yourself to learn an ml
I don't owe it to myself to learn anything unless I decide to. I wasn't trying to convince anyone else to use Python; I was just giving reasons why I like Python. Please extend me the same courtesy when you talk about things you like about your favorite languages.
> I see that F# has a REPL, but it seems bolted on as an afterthought instead of being an integral part of the language. (Also, it's not clear whether the REPL is available in Mono on Linux. I am allergic to Windows and anything built by Microsoft.)
well that first sentence is simply not true, and i don't know what gave you that impression. f#'s repl, f# interactive, is available on linux through mono and will soon be available through .net core. i can't help you being needlessly allergic to a major operating system or company. f# came out of microsoft research.
> Huh? Python expressions always return a value.
they do? take:
def test(x):
"something"
test(2)
> I don't owe it to myself to learn anything unless I decide to. I wasn't trying to convince anyone else to use Python; I was just giving reasons why I like Python. Please extend me the same courtesy when you talk about things you like about your favorite languages.
i don't know why you're upset. i am not for sure where i was discourteous. i didn't mean anything by my statement other than to check out the languages and courses (the coursera one is very good). if those are your reasons for python, i just thought you would honestly like an ML-based language. i just used "owe it to yourself" as an expression, which apparently came out wrong through text. of course you don't owe anyone anything. i was just making a suggestion.
True
>>> None
>>> print(None)
None
test(2) is None
peter@localhost:~$ python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def test(x):
... "something"
...
>>> test(2) is None
True
peter@localhost:~$ python
Python 2.7.12 (default, Dec 4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> def test(x):
... "something"
...
>>> test(2) is None
True
However, there is another even more direct experiment you could have run at the Python REPL to see what Python expressions return: just type the expression directly at the REPL and see what it prints back at you. In Python 3:
peter@ToshibaSatellite:~$ python3
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> "something"
'something'
peter@ToshibaSatellite:~$ python
Python 2.7.12 (default, Dec 4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> "something"
'something'
> i guess that is true in python3 but it is not in python 2.7.
No, it's always been there.
>>> def f():
... pass
...
>>> f() is None
TrueYes, and thanks for acknowledging that. But then the phrase "owe it to yourself" didn't really communicate your intent correctly, which is why I objected to it.
I think the GP's point was that functional languages give you similar benefits, but with more uniform and disciplined behavior of many language features. Why FP is not more widely used compared to imperative language is another topic though.
I love JS because it's flexible and doesn't lock me into a paradigm. I can be as strict as I want to be. In the end, I think it depends on how someone thinks about problems. I tend to bend my thinking depending on the type of problem and how I might optimally solve said problem. When I need something else (often for performance), I'll circle around at that point for that part.
Can you give a link? I'd be interested to read it.
http://www.gigamonkeys.com/code-quarterly/2011/hal-abelson/
the paragraph he says it in is:
> It’s just a different course. We could have done it in Scheme and I sort of wish we had. And for random reasons we didn’t. But the thing that ties it together—if you had fifteen seconds to describe the whole course—it’s still about abstraction and modularity. The beginning of that course is not very different from the beginning of 6.001 other than you do it in Python. But at that point, where you’re not building interpreters yet, you can pretty much do this stuff in Python. You have to contend with Python’s broken notion of lambda but other than that it’s not really that different.
he mentions it as an aside really, but the overall interview itself is very good. abelson is a treat to listen to.
Agreed. I only mentioned Scheme specifically because Abelson did in what was quoted.
as explained in the interview, the course changed to python because the course itself changed. they basically killed off the old course and created new courses built around their new degree program structure. the new course hits a lot of different aspects of electrical, computer, and software engineering, many of which heavily use existing libraries, so they seemed to pick python for this.
in my opinion, they could have easily written libraries in another language, so i suspect there were some politics at MIT that lead to the decision to use python.
def binds it to a name at the same time, which is remembered in the object (which AFAIK isn't really used for anything except maybe debugging tools), lambda returns an anonymous one. The resulting object can be passed around in both cases.
the point was in something like f#, lambda creates a function which can then be bound to a name if you want. if you do so, this is identical to the "normal" function definition because that is really just syntactic sugar for creating lambda functions and then binding it to a name.
Because lambdas don't have function names.
the def syntax merely sets the name property on the function object too, since it has a name to set available. As said in other comments, nice for interactive use, irrelevant otherwise.
>>> normalize_case = lambda s: s.casefold()
>>> normalize_case
<function <lambda> at 0x034B04B0>
>>> normalize_case.__qualname__ = "normalize_case"
>>> normalize_case
<function normalize_case at 0x034B04B0>
>>> def normalize_case(s):
return s.casefold()
>>> normalize_case
<function normalize_case at 0x00594420>
>>> normalize_case.__qualname__
'normalize_case'however, this still goes in the bucket of why i personally dislike python. this just seems sloppy and overly complicated.
for example, i tested this out myself.
> test = lambda x : x*x
> test
=> <function <lambda> at 0x7ff6221d76e0>
> test.__qualname__
Traceback (most recent call last):
File "python", line 1, in <module>
AttributeError: 'function' object has no attribute '__qualname__'
> test.__qualname__ = "test"
> test
=> <function <lambda> at 0x7ff6221d76e0>
> def test2(x):
> return x*x
> test2.__qualname__
Traceback (most recent call last):
File "python", line 1, in <module>
AttributeError: 'function' object has no attribute '__qualname__'
> test2.func_name
=> 'test2'
> test.func_name
=> '<lambda>'
yet, somehow, people call python "simple".
As far as I know, the __qualname__ property was introduced in Python3.3, so obviously you wouldn't see it in any Python 2 version.
Perhaps it would help if you didn't think of "lambda" as defining a lambda function, but something more prosaic like "defexpr", and that Python doesn't have lambda functions at all?
As to your experiments, you have found one of the many differences between Python 2 and Python 3.
After researching F# for about 10 minutes, perhaps you might think of Python 2 as being like OCaml and Python 3 being like F# - they are two instances of the same language family, with a mutually compatible language subset, but many incompatibilities.
python's version bifurcation problem is its problem, not mine. the problem exists.
You've bitten onto a more or less internal detail (I've TA'ed a bunch of undergrad courses in Python, and it's firmly in the land of things students don't know if they don't discover it on their own) and use that and seemingly quite little grasp of how Python works to argue that it isn't simple, while of course languages you are more familiar with are better.
Similarly, Python basically keeps track of "proper" names for proper functions, but can't/won't do so for lambdas.
The lambda syntax isn't that different from the syntax for defining functions, unless you're referring to the fact that it's different than anonymous functions in other languages.
I don't find the inability to pass around operators to be a hindrance, but the fact that getting into python requires understanding all of these dunder or magic methods is another story.
However, on map, filter, & reduce, I agree with you completely.
One of the worst aspects of python is the extensive use of bad practices, and ignoring functional programming principles is one of the top offenses.
they weren't intended to be great or mind blowing. they were just examples to make the point that in a good language, it's clear what is going on.
the operator thing is an extension of that. in f#, the "infixness" is just a syntax thing. the semantics is that they are functions and can be treated as such. it seems operators are something weird in python.
Lambda, map, reduce, and filter were part of the Python 1.0 release. The actual commit date was Tue Oct 26 17:58:25 1993 +0000:
commit 12d12c5faf4d770160b7975b54e8f9b12694e012
Author: Guido van Rossum <guido@python.org>
Date: Tue Oct 26 17:58:25 1993 +0000
* compile.[ch]: support for lambda()
* PROTO.h, mymalloc.h: added #ifdefs for TURBOC and GNUC.
* allobjects.h: added #include "rangeobject.h"
* Grammar: added lambda_input; relaxed syntax for exec.
* bltinmodule.c: added bagof, map, reduce, lambda, xrange.
* tupleobject.[ch]: added resizetuple().
* rangeobject.[ch]: new object type to speed up range operations
(not convinced this is needed!!!)
Van Rossum described that history at https://python-history.blogspot.com/2009/04/origins-of-pytho... . I don't see anything which sounds like "throwing a bone", much less being "explicitly against functional programming" when that discussion occurred.
My understanding is that he wasn't opposed to their inclusion initially but became negative towards them only years later.
Yep. Had a coworker like that recently. Loved “simple” code, chock full of mutable data everywhere, where everything was written out a good 3 or 4 times longer than I would have liked to see it.
I call that sort of “simple, explicit” (repetitive) code “My Summer Vacation” code, after a bit in an old Cheech and Chong skit where the highschool kids have to read their essays to the class.
As someone who has had a lifetime of on-off relationships with attempting to become a competent programmer, (BASIC, assembler, C, Pascal, Visual Basic, C++ and others that I can no longer remember), Python is the only language that has helped me become anywhere near useful or proficient as a programmer. I've managed to build a couple of -genuinely useful- applications (one of which is being used daily in my girlfriend's school), which hasn't been the case for anything else.
I've found that Python is simple enough to learn, and hasn't really been any kind of a limitation for me - this may be because of me, not the language - OK, I appreciate that I'm not going to be writing 3D games or real-time audio apps in it, but for what I've been trying to do (solve real-world problems), it's been amazing. I even programmed a 'play sound effects when you press a MIDI controller' system which I used at a show with 20,000 people attending this summer, replacing a massively over-complex (and expensive) audio environment with about 30 lines of Python combined with two libraries I spent an afternoon learning.
That, for me, is why I like Python.
TIL 3D games and real-time audio apps are not "real-world problems".
There never was a goal in Python development to support all programming paradigms.
How do you distinguish between "design flaw" and "different design goal than what you want"?
EDIT: (Van Rossum's own explanation of why he thinks lambda's design is okay: https://www.artima.com/weblogs/viewpost.jsp?thread=147358 )
The phrase, by the way, is "one obvious way".
See also: map and filter being the only 2 other functions implemented and both of them being less performant than their imperative equivalents; outright denial of any attempt at TCO because "Guido doesn't like recursion".
No, don’t ignore problems of TCO by saying “guys don’t like functional programimg”. Guido cleary states why he doesn’t like it: [1]. Even though TCO is one of the ES6 standard, TCO also is largely denied by JS implementors (Firefox, Chrome, and Edge devs) because of the similar reasons. As a result, Safari is the only browser that supports TCO. [2][3] And Rust team is not eager to introduce TCO [4].
[1]: http://neopythonic.blogspot.com/2009/04/tail-recursion-elimi...
[2]: https://www.chromestatus.com/feature/5516876633341952
We will sometimes optimize for TCO, that issue is that we can’t guarantee it just yet.
Anonymous functions are vital to any modern high level language.
I think most people get hung up on the word. If the keyword was changed from “lambda” to “fun” for example I think it wouldn’t be as obtuse to more intermediate developers.
To be honest I think the lambda does Python a service vs a disservice. It helps to illustrate that everything in Python is an object. Once you realize you can treat functions similarly to how you treat things like integers and strings your mind opens up to functional style. Assigning a function to a variable the same way you do with a string, for instance, helps to reinforce this concept.
I don’t think you’re wrong, but I am also not experienced enough to know why this would be the case. What is the virtue of having an anonymous function? I get that it makes code somewhat easier to read and write if you’re only using the function once and it’s short, but that seems like an edge case. What am I missing?
You are about to embark on a great adventure. I wish you luck on your quest!
Compared to JavaScript it didn't evolve it's syntax much. Of course JS was much worse off to start with.
The `__magic__` is about avoiding naming conflicts while still being explicit. It's a choice between magic or reserving a lot of names.
About self, it's so strange to define a method with 2 arguments and call it with only one. Not only it's strange, it feels the opposite of being explicit. Most other languages manage to do without that self (Java, Ruby).
I run into another oddity today. Yesterday if forgot about
a = 0
if not a:
print("like in C")I like 'em because you can do "stupid python tricks" like Church numerals: https://en.wikipedia.org/wiki/Church_encoding
plus = lambda m: lambda n : lambda f: lambda x: m(f)(n(f)(x))
succ = lambda n: lambda f: lambda x: f(n(f)(x))
mult = lambda m: lambda n: lambda f: m(n(f))
exp = lambda m: lambda n: n(m)
c0 = lambda f: lambda x: x
c1 = lambda f: lambda x: f(x)
c2 = lambda f: lambda x: f(f(x))
c3 = lambda f: lambda x: f(f(f(x)))
c4 = plus(c1)(c3)
c5 = plus(c2)(c3)
c6 = plus(c1)(c5)
c7 = succ(c6)
c8 = succ(c7)
c9 = c3(succ)(c6)
c10 = mult(c2)(c5)
c11 = succ(c10)
c12 = mult(c3)(c4)
c64 = c3(c4)
c256 = exp(c2)(c8)I mean it as a self-deprecating way to decry the quality of the post without being too harsh. The author of the blog post had good intentions, but in my opinion he should have put it off for another decade or so, to gain perspective.
>No true Scotsman or appeal to purity is an informal fallacy in which one attempts to protect a universal generalization from counterexamples by changing the definition in an ad hoc fashion to exclude the counterexample.
edit: ok he didn't mean it that way!
* FizzBuzz with no control-flow keywords: https://github.com/ubernostrum/interviewer-hell/blob/master/...
* Detecting if a number is a perfect square as a one-line lambda: https://github.com/ubernostrum/interviewer-hell/blob/master/...
I can't find it now but some daredevil wrote a Lisp interpreter in a single Python expression. A work of art.
I much prefer languages that go all the way and give you full integration like Ruby, ES6 and Groovy. For example, in Groovy closures just blend completely into the language:
[1,2,3,4,5].grep { it > 3 }.collect { it * 5 }
map(lambda y: y * 5, filter(lambda x: x > 3, [1,2,3,4,5]))
[x*5 for x in [1,2,3,4,5] if x > 3]
You wrote elsewhere that
> I won't miss Grails because I think that, a bit like Gradle, it is an antipattern use of Groovy - needlessly applying its dynamic features where they are not even required
Groovy was designed as a dynamic language to complement Java. Its static compilation was tacked on much later, whereas Kotlin and Scala, like Java, were designed to be statically typed from the ground up, and they, unlike Groovy, also run on other platforms besides the JVM. If you're not going to use Groovy's dynamic features, you might as well use Java, Kotlin, or Scala.
Reminds me of a story.
A few years ago Peter Norvig (Google) posted a "Show HN" posts with one of his amazing notebook excursions where he went through nice little problem, building up a library of functions to make a solution. I think this has happened multiple times.
One such time, one bit of code used a lambda expression, and I was new to Python, and wondered in a possibly-slightly-whiney comment (as I recall) whether it was the most readable way to do things.
Within an hour or so, his post was silently updated and the lambda keyword was gone. I don't remember the nature of the rest of the change, whether it was just the one block of code or more, but the code was more readable to me, at least as a newbie, at the time. Not sure my comment had anything to do with it, but I was impressed that he did make the change in any case. I have never worked with him, but my respect for him grew with that tiny little incident.
BTW in case anyone feels the urge to explain lambda expressions to me now, that's OK, no need, but thanks anyway.
def length_and_alphabetical(string):
"""Return sort key: length first, then case-normalized string."""
return (len(string), string.casefold())
https://docs.python.org/3/library/operator.html
In particular 'attrgettr' and 'itemgettr' are very useful functions to map over objects and sequences.
const points = [[[1, 2], 'red'], [[3, 4], 'green']]
const points_by_color = points
.sort(([point, color]) => color)
This tutorial is not an argument against lambdas, but an argument against abusing Python lambdas.
> lambda expressions are an odd and unfamiliar syntax to many Python programmers
Well these python programmers now have an opportunity to learn this unfamiliar syntax. You don't avoid using parts of a programming language just because it's unfamiliar! To new Python programmers, a lot of the syntax is unfamiliar at first, especially if coming from a curly-brace language or new to programming in general.
While I like many of the ideas in the article (especially the use of standard library functions that already do what your lambda was trying to do), it seems to make some cases against using lambda where the justification isn't entirely valid. :/
A good rule of thumb is given by the google python coding guidelines [1]:
"Okay to use them for one-liners. If the code inside the lambda function is any longer than 60-80 chars, it's probably better to define it as a regular (nested) function."
[1] https://github.com/google/styleguide/blob/gh-pages/pyguide.m...
It isn't, it's just a different way to approach programming.
As for the article, it isn't really a case against lambdas themselves, just against misusing them. It certainly didn't make a strong enough case for me to stop using them.
While in some functional languages (Haskell) and array programming languages (J), tacit or point-free style is often more idiomatic than lambda expressions (or equivalent).
There's something funny about that.
1. It is usually taken too far (indeed I think flip is probably too far and (.).(.) is certainly too far. I’m also not a big fan of eg fmap.fmap)
2. (.) is the wrong way round. Code reads much better if it has type (a -> b) -> (b -> c) -> (a -> c)
In APL or J was I don’t have enough experience to say other than that I find it difficult to read but the puzzle is amusing
I personally have a hard time understanding almost any point-free Haskell code and deciphering a line of J might take me an hour. But as long as it's readable to the intended audience it's alright. Of course it isn't sometimes.
I have never seen a code that wasn't made clearer when moving from a three-liner non-lambda somewhere upper in the code, to a one-liner lambda used at the right place
Lambdas are in so many languages than it seems worth it to spend 10 minutes learning what they are.
Theres a workaround using default arg value to capture by val but its ugly and easy to forget and hard to "see" in review that its missing.
This is as much an issue of loops not generating "fresh" instances of their index vars as it is a problem with lambdas, to be fair, but it still sucks, esp in mathematical domains. Julia used to suffer from this but fixed it, I believe the same is true for C# too.
polys = []
for i in range(0,4):
polys.append(lambda x: x**i)
polys[0](2)
> 8
polys[1](2)
> 8
polys[2](2)
> 8 polys = [lambda x: x**i for i in range(0,4)]
for i in range(0,4):
polys.append(lambda x, i=i: x**i)
polys[0](2)
> 1
polys[1](2)
> 2
polys[2](2)
> 4
polys[3](2)
> 8 colors = ["Goldenrod", "Purple", "Salmon", "Turquoise", "Cyan"]
normalized_colors = map(str.casefold, colors)Is this a thing? Are Lambdas "Unpythonic?" or is the writer of the article totally off base?
Though it didn't happen in the end: https://softwareengineering.stackexchange.com/a/252162/29892...
I've come down to the view that _defining a function using `def` syntax adds structure to a program_, even when it's not desired. It's multi-line, includes an indent and a keyword. It look more like creating a `Class` than a `list` for example.
As a result, in order to define a function _without_ adding structure people fall back on lambdas.
I think the best resolution would be to embolden lambdas with some of the benefits of functions - e.g. multiple lines
First line says:
An eta conversion (also written η-conversion) is adding or dropping of abstraction over a function.
For example, the following two values are equivalent under η-conversion:
\x -> abs x
and
abs sorted_numbers = sorted(numbers, key=abs)