"Real" anonymous functions for Python | Hacker News Reader

"Real" anonymous functions for Python

lwn.net

36 pointshackandthink54 comments

Unshamed plug of my project lambdex, which I worked out years ago to bring multilined lambda expressions in Python.

Nice :) Today it's possible to do a lot of it with vanilla Python (sortof):

   something(lambda x: [
      x2 := x + 10, # define variables

      [ print(f"Number: {i})
        for i in range(x) ], # loop

      [ 0 for i in range(x)
           if print(f"Number: {i}") or True
           if print(f"Second line: {i}") ], # loop if you prefer "normal code order"

      0 if x > 0 else {}[f'{n} should be positive'], # conditions and asserts (throws KeyError, but includes the error message)

      x2 * 2 # return
   ][-1])

note: I'm not responsible for coworkers injuring you after committing this

> in an idiomatic manner

Come on. No. It has the benefit of working, but definitely not being idiomatic.

Anonymous functions and object destructuring were strong influences in me moving much more towards typescript than python after many many years of python being my primary language.

Anonymous functions completely change the experience and style of programming and python is diminished for not having them.

Pythonistas I have found dismiss the importance of anonymous functions and object destructuring. I assume this is because they don’t have the experience of these things therefore do not know their value. Pythonistas point to this or that python feature saying “see, python can do javascript style destructuring” or they point to lambda functions. Again this can only mean they don’t have experience if the real thing.

There also seems to be a bit of “not invented here syndrome”, after all how could javascript (yuck!) do anything better than python?

Javascript anonymous functions and object destructuring are two of the most useful tools there are in programming and python misses out.

Ideally we can make progress on this.

I saw people occasionally do weird things to bypass this, like `let x = 1 in f(x)` manually transpiled to `(lambda x: f(x))(1)` or thank god we have walrus now, so `(x := 1, f(x))` (as mentioned in the original link) is a workaround that is somewhat actually readable.

There really isn't a syntax problem. All you have to do is, assuming you're in whitespace-oblivious mode due to being inside `()` or something, switch back to indentation-based lexing after the `:` of a `def` or `class`, and end once it dedents or you reach (but do not consume) a close paren (of course, after the close paren you might end up enabling indentation mode again).

I'm not sure what style guides would declare the correct formatting to be, but this would not confusing to the parser at least:

  some_function(def f(a):
    return a # comma must be on the next line unless you add parentheses
   , def g(b):
    return b)

Note that `lambda:` and `slice[:]` are different uses of the colon, and `wal := rus` is a different token.

Indeed. Nim also has significant whitespace, and your example translated:

    proc some_function(a, b: proc(a: int): int) =
      echo a(1)
      echo b(2)
    some_function(proc(a: int): int =
      return a
    , proc(b: int): int =
      return b)

is valid Nim and compiles just fine.

(Also, "1, 2" is not valid Nim, so unlike with Python you can just put the comma right after `a' without syntactic ambiguity.)

Following the aside ... requiring parentheses around tuples is generally best for readability. I can only think of two cases that are more readable without them:

  (x, y) = (y, x)
  some_array[(x, y)]

Languages generally already special-case both the LHS of assignments and the inside of indexing brackets, and special-casing the RHS of assignments seems like a reasonable extension.

To amplify your "comma right after", the call can even be simply:

    some_function(proc(a: int): int = a, proc(b: int): int = b)

And Nim lets you put ()s in a great many places to resolve problems. I find its syntax quite a bit nicer than Python, actually.

Why do people want anonymous functions so much? Is it just because naming things is hard?

Classically trained math focused types tend to be especially bad at naming things. Which, I guess, is fine when developing the infernal equation. but it sure does not do anyone any favors when they have to understand the thing.

personally, in python, I refuse to use lambda functions on ethical grounds, all my functions are named. And then you should see my javascript, all those names would probably make any real JS dev sick.

What?

Anonymous functions are amazing for: * Iterators (map, reduce, filter) * Callbacks (ui actions) * Threads/async tasks

The reason I don't name them isn't because I can't think of a name, it's because: * There's no point, they'll only be referred to once * Introducing a separate function block breaks the flow of the code and makes it harder to read (especially for iterators!)

Why do people want anonymous functions so much?

Because in many cases a function is tightly associated with a particular context, and making it a separate named function is actually harmful for understanding.

personally, in python, I refuse to use lambda functions on ethical grounds

Do you really find this:

  def get_name(x):
    return x.name

  sorted_by_name = sorted(people, key=get_name)

more readable than:

  sorted_by_name = sorted(people, key=lambda x: x.name)

Yes actually, it’s significantly more readable.

It's debatable if it's more readable or not, for sure it's shorter.

The first is more readable and easy to follow but takes more effort to track down the original fuction

Unless you’re using a modern IDE… Then you just command click, and it takes you right to the function.

the "lambda" keyword is a problem. Compare to say, groovy:

    sorted_by_name = people.sort { it.name }

I’ve worked on several large and prominent projects; most of which experienced a large layoff of engineers in the middle of development. As the team who regularly takes over laid off team’s work with 1 engineer to the previous 10. The top things that slows our team down is “legacy” node applications (it seems like every time you turn around, whatever JavaScript or node version you used is out of date) and figuring out why an anonymous function exists and what it does.

Also due to the multiple style choices and options in JavaScript the readability is trash.

Are you seriously going to name "obvious" functions like `[x + 1 for x in xs]`?

I agree - I also don’t use lambdas in Python, because they make the code significantly less readable or maintainable down the line.

Because coming from other languages like Rust, Kotlin, and ES6, going back to Python feels archaic. A huge reason for that comes down to lambda syntax.

> I refuse to use lambda functions on ethical grounds, all my functions are named

This is embarrassingly more literal than I would like, but I in large part refuse to use Python on something similar to "ethical" grounds because it doesn't support inline / multiline functions (and a couple of other similar issues).

After using typescript/ruby/groovy/scala/kotlin it's just so fused into my way of thinking about programming that each time I have to pull out an incidental function from its context and it reduces the clarity of the code, I grind my teeth. I'm guessing you just won't understand this unless you immerse yourself in another language and its idioms.

Limiting lambdas to a single expression makes functional programming in Python quite painful , along with the lack of a "let" block for temporary variables and the removal (!!!) or argument unpacking which I am still mad about.

Syntax is definitely a problem, but I wonder if "looking Pythonic" needs to be such a huge roadblock.

comprehensions are (a) expressions and (b) bind

Yes, I've used list comprehension for temporary variables and chaining expressions together. But you can't tell me that's in any way "Pythonic" :)

True; a while ago I sent someone a script written in my idiolect and he replied that I really ought to know where Common Lisp was, if I wanted it.

Transpiling to comprehensions is, however, a great way to get access to arbitrary python libraries from experimental languages.

I didn’t quite understand what the “real” anonymous functions are compared to lambda. Is it just that the lambdas aren’t multilines?

Having grown up with Python lambdas, maybe I’m blind to their limitations, but I didn’t feel like there was any real argument as to why not make a real function when you need more than a lambda.

The major killer feature of scheme-style lambdas is that you don't have to name them. Naming stuff is hard, and you usually get it wrong. If the thing you want to do is a couple lines of code, and you can just drop it in as a lambda and not have to think of a meaningful name, it's easier to keep flow-state. If things expand to the point where glancing at the code doesn't tell you what the function does, that is the time to break it out into a named function.

Just call it foo. Not a problem.

You can still do anything you want with just Python lambdas technically speaking right? Like, Python lambdas can express any lambda calculus expression, so as long as you're not writing something with side effects, you can use an arbitrarily complex Python lambda. Though I guess the resulting code wouldn't exactly be Pythonic.

Language design is really about what you could do, but about how to efficiently express certain things.

> Language design is really about what you could do, but about how to efficiently express certain things.

That's missing a "not", right (probably after "really")?

Not the person you're replying to, but I assume they meant "rarely" instead of "really". I make this typo all the time myself!

That makes sense. Thanks!

Depends on what you mean by "technically".

Computationally, python lambda is fully capable of whatever a regular python function can do, including side-effects, (you can call foo.set_bar()).

The limit is syntactical: a lambda has to be one single expression. So, no assignment statement, no multiple lines, etc.

I tend to use inner functions (nested def) when I need something more than a simple lambda, but don't want to come up with a good name or pollute the class / module namespace

It works okay, but I do wish there was a better way

> It works okay, but I do wish there was a better way

What's wrong with it?

nothing really, probably just me, but it always felt a bit ugly to nest function definitions, plus it can separate the (often single-use) function from the place it is being used in

Can you set a breakpoint inside a lambda function without resorting to using multi line trickery using braces that essentially does what people say lambdas should not do?

Not the OP, but I program professionally in Python for the past 5 years and i have never set a breakpoint. Assuming they don't work in lambdas (sounds like a tooling issue), is this really such a big problem?

Depending on what you're doing this can be really annoying.

>Not the OP, but I program professionally in Python for the past 5 years and i have never set a breakpoint.

This means you are either one of the best or one of the worst programmers. People in the middle have bugs in their program and can use a debugger to find them.

>sounds like a tooling issue

The tool, in this case, being the interpreter. Which is exactly the point.

No, but I also can’t set one in the middle of a chain of methods.

In Python it’s quite common to have to refactor in order to debug.

One could argue: given that so much useful software has been written with Python after so many years without anonymous functions (beyond lambda expressions), perhaps GvR had some wisdom in never upgrading the lambda expression beyond single-line expressions.[1] It does mean that Python programs tend to be more procedural / imperative / object-oriented in style rather than functional, but maybe that's not such a bad thing. You can still write some nice quasi-functional code that is quite readable.[2] And I say this as someone who has a lot of fun playing with functional programming, and even once wrote a blog post (based on a technical talk) comparing Python and Clojure.[3]

[1]: https://python-history.blogspot.com/2009/04/origins-of-pytho...

[2]: https://github.com/amontalenti/elements-of-python-style?tab=...

[3]: https://amontalenti.com/2014/11/02/clojonic

How about allowing non-indentation-defined code blocks? Say, by wrapping a block of code with {}?

That's already been taken care of - all you need to do is:

  from __future__ import braces

Excellent, Python is officially my favourite language now!

I am not 100% certain, but I suspect you haven't tried it yet. Or you might like your languages to incorporate humour?

They might not have imported antigravity yet, either.

There are already multiline lambda functions available by using the walrus operator:

foo = lambda x: ( z:=x+1, zz:=z*2, zz+7 )[-1]

it defines a tuple and returns the last element. List comprehensions, walrus operator and lambda make a functional language out of python.

As written, your comment isn't demonstrating multiline lambdas. You need to prefix each of the code lines with two spaces so you get a code block. Otherwise it's treating it as a single paragraph and concatenating it all together.

What you currently have:

  foo = lambda x: ( z:=x+1, zz:=z*2, zz+7 )[-1]

What you seem to want:

  foo = lambda x: (
    z:=x+1,
    zz:=z*2,
    zz+7
    )[-1]

>For more complex computation, named functions, which can be limited to the local scope to avoid any name-collision problems, are the way to go, as a 2006 blog post from Guido van Rossum stated. In his opinion, there is no Pythonic way to embed a multi-statement construct in the middle of an expression.

Well, there's no Pythonic way because Guido didn't let one be.

>The problem, in a nutshell, is that Python blocks are delineated using white space, not braces or keywords like begin and end, so any expression used to create a function with multiple statements would need a way to incorporate white space—or it will not look like Python at all

All kinds of other syntax made it to the language in the last 18 years, but an "end" delimiter for multi-line closures cannot?

>The problem, though, as Paul Moore pointed out, is that there is a process that needs to be followed so that the core developers will reconsider a longstanding decision of this nature; someone needs to explain why the objections raised before are no longer valid

Well, the objections raised before" are all subjective opinions about how this is unpythonic, so good luck refuting feelings.

You're either for it, or against it, and if you're for it and use subjective opinions as objections, then the other side can't really do anything, even if they state objective facts (like how developers are by now well familiar with them, how they avoid polluting the local namespace, how they're useful to setup APIs that provide the mechanism and you pass in the policy at the call site, and so on).

>But without a solution to the syntax issue, the whole argument is not worth having.

Well, there can't be a solution if the problem "we don't like how it looks" - the actual practical problem can certainly be solved.

> There is, of course, lambda, "but its usefulness is quite limited and its syntax is frankly a bit cumbersome".

I found them useful enough to implement a BLC interpreter [1]. Equally useful as lambda in perl, ruby, and javascript.

[1] https://rosettacode.org/wiki/Universal_Lambda_Machine#Python

mmmmh, I wonder how ruby is doing this

Reply on news.ycombinator.com