Hacker News Reader

Comment by seertaak | Hacker News Reader

Functional programming in Python is severely hampererd by Python's lambda keyword, which - simply put - sucks.

I like the way the Functional Programming HOWTO in the Python docs (https://docs.python.org/3/howto/functional.html#small-functi...) puts it:

  Fredrik Lundh once suggested the following set of rules for refactoring uses of lambda:

  1. Write a lambda function.
  2. Write a comment explaining what the heck that lambda does.
  3. Study the comment for a while, and think of a name that captures the essence of the comment.
  4. Convert the lambda to a def statement, using that name.
  5. Remove the comment.

Yeah, Python would really benefit from multi-line anonymous functions.

There was a limitation in Python where spacing and indentation gets ignored between parentheses, which makes it impossible to pass a multi-line lambda as an argument to a method or function. However, given the new parser, that limitation might be able to be mitigated.

But you can define functions inside of functions (recursively).

  def function1():
      print ("hello outer")
      def function2():
          print ("hello inner")
      function2()

  function1()

Output:

  hello outer
  hello inner

But you can’t do that (or try/except) in an expression such as a list comprehension or similar.

Yes, because gross.

Nor, IMO, should you want to.

Why not? Many other languages have closures and even Python has lambdas. Multi-line closures work well for these languages including in map/reduce/etc or comprehensions where applicable. Seems like your preference is overfit to Python’s limitations.

Because you can do it in a function. It's not a limitation when you can do it. Having to define a function that's easier to read, and testable as an independent unit, is not a problem.

I don’t think this limitation is likely to change - the only ways to allow whitespace-sensitive statements inside an argument list are all really ugly.

You can do multiple lines, just not multiple statements:

    >>> f = lambda x: x * \
    ...  20
    >>>
    >>> f
    <function <lambda> at 0x7f8b69f2c6a8>
    >>> f(2)
    40

In fact, no statements at all - just a single expression.

well, you could pack a lot of other expressions into that expression, especially with 3.8/3.9, e.g.

x,y = 1,2

q = list(map(lambda t: (tx := t*x, ty := t*y, tx+ty)[-1], [1, 2, 3]))

Walrus operator: not even once.

walrus, actually, being an expression, it's a perfect fit for lambdas

Agreed. I suspect that most uses of python decorators become moot with proper multi-line anonymous functions. I assume some would argue that decorators create more readable code but they seemed like a syntax hack to me.

I would argue that multi-line anonymous functions are unpythonic. Exhibit A is this line from PEP 20:

  Readability counts.

I'm currently about a month into learning a legacy codebase that was written in a functional language. If I could single out one thoroughly egregious practice that has made this code far, far more difficult to read and understand than it should have been, it's multi-line anonymous functions. In general, if a function is doing something complicated enough to need multiple lines of code, it's doing something complicated enough to merit an explicit name.

> if a function is doing something complicated enough to need multiple lines of code, it's doing something complicated enough to merit an explicit name.

  def foo_and_bar(x):
    foo(x)
    bar(x)

whew! good thing i named that

IME this limitation just leads to throwaway names like

  process(x)
  go(x)
  do_foo(x) # in foo(x)
  cb/callback/fn

because a lot of things just don't have sensible names! just like how it'd suck to have to come up with a name for every loop body

(although there was some book advocating for "replace every loop body with a named function" so some people enjoy that i guess...)

I see you chopping the first two words off that quote. ;)

In this specific case, that single instance of a single pattern is such a throw-away that it doesn't deserve a name, but the pattern itself is easy enough to name. So I'd skip the single-purpose function and create a combinator.

  def do_each(*args):
    def helper(x):
      for fn in args:
        fn(x)
    return helper

and then, when I need to do both foo and bar, I don't even need a lambda.

  map(do_each(foo, bar), some_sequence)

That's a fairly specific case, though. Moving back to the general, I would say that a function that does more than one thing, but can't easily be named, is a code smell.

Of course, every general rule has its exceptions. But I'm not so keen on the idea of optimizing one's coding style for the exceptional cases. Going back to PEP 20, "Special cases aren't special enough to break the rules."

(I realize mapping a function that returns nothing is terrible, but I'm feeling too lazy to think of a better example.)

i like a nice combinator as much as the next person! but consider this: if python already had multiline lambdas, would you be arguing for using a narrow-purpose combinator instead of `lambda x: foo(x); bar(x)`?

[this kind of reminds me of Go's generics mess, where workarounds for lack of generics are "just how you write Go and that's the language's philosophy"... until generics land and suddenly they won't be]

Probably. I think the former is more readable and, at the use site, more concise.

Regardless, I don't think the hypothetical is super useful, because its unstated major premise is, "But what if we, for the sake of argument, ignore all the other good reasons why Python doesn't have them?"

My favorite programming language is functional, and has significant whitespace and multiline anonymous functions. While it is my favorite, I do have to concede that the Python language maintainers' worries about the syntactic implications of multiline lambdas in a whitespace language are accurate.

(I could quote the zen of python some more here, too. Lines 5 and 6.)

Assignment expressions were "unpythonic" based on PEP 20 until they weren't. Modern languages have multi-line anonymous functions, and developer expectations have changed in the 17 years since PEP 20 was published.

the thing with python is that it's "indentation-based" -- it's difficult to see "where does this anonymous function return or end?"

maybe it's time to allow optional brackets for marking where a function starts and ends? eg. a { ... } (or anything else -- even {| ... |} will do)

If you really want though, you can write a web server in a single python lambda:

    (lambda flask:
        (lambda app:
            (app,
            app.route('/')(
                lambda: 'Hello World!'
            )
            )[0]
        )(flask.Flask('__name__')).run()
    )(__import__('flask'))

See https://gist.github.com/e000/1023982 for more horrible examples

I hate it, but it is clever.

Functional programming in Python is severely hampered by the fact that Python is statement-oriented rather than expression-oriented.

The lambda limitations are a product of this plus the particular whitespace rules Python has.

This has really not been true since python grew a ternary operator, syntactically you can write functional code perfectly fine in python -- the only significant syntactic shortcoming is lack of pattern matching.

What actually hampers functional expression is semantics and mostly two-fold:

1. functions are terribly slow and always grow stack, so cannot replace iterative constructs.

2. Although python actually comes with a fair amount of functional data structures out of the box (str, bytes, tuple, frozenset, namedtuple, ...) none of them can be "updated" efficiently.

There are some other things (exception as opposed to sum-type based error handling), but fixing the above would be enough to write functional code pretty unhampered, I think.

> Although python actually comes with a fair amount of functional data structures out of the box (str, bytes, tuple, frozenset, namedtuple, …) none of them can be “updated” efficiently.

If they can’t be updated efficiently, they are just immutable than particularly functional.

Of course I agree that calling such data structures immutable is more precise, but it's still in contrast to both imperative programming and imperative data structures. In particular every single functional language (including the purest of the pure) fairly heavily use immutable array based structures in one way or other. So if functional programming requires only using data structures with structural sharing, there are no functional programming languages.

Pattern matching is on the roadmap for Python 3.10[1].

[1] https://www.python.org/dev/peps/pep-0622/

Not really.

Python's lambda does pretty much what you would expect from Scheme. It creates a callable the binds arguments to parameters in a lexically scoped namespace and then evaluates the body of the lambda in that namespace. And with an open parenthesis, you can write multiline lambdas and indent it however you want.

All the usual wizardry is possible:

  >>> (
      lambda n: (lambda fact: fact(n, fact))(
          lambda n, inner: 1 if n == 0 else (n * inner(n - 1, inner))
      )
  )(5)
  120

Also, Python has rough equivalents to some special forms in Scheme:

  (if testexp posexp negexp)  ⟶  (posexp if testexp else negexp)
  (cond (p1 e1) (p2 e2) (else e3)  ⟶  (e1 if p1 else e2 if p2 else e2)
  (begin e1 e2 e2)  ⟶  [e1, e2, e3][-1]
  (and e1 e2 e3)  ⟶  (e1 and e2 and e3)
  (or e1 e2 e2)  ⟶  (e1 or e2 or e3)

Some statements do have a functional form but aren't well known:

  class keyword  ⟶ type(name, bases, namespace)
  import keyword ⟶ __import__(name, globals, locals, fromlist)

You have map(), filter(), partial(), and reduce(). The operator module provides function equivalents for most operators. Also, the itertools were directly based on their equivalents in functional languages or array manipulation languages.

That said, Python does lack some essential tooling that you would really miss:

  - There is no way to create new special forms.
  - Some important special forms are missing: let, let*, and letrec
  - The language spec precludes tail call optimization.
  - Some Python statements lack functional equivalents: try/except, with-statement, and assert-statement

Actually there is a let and let* equivalent as well:

  (let* ((v1 e1) (v2 e2)) e3)  ⟶ ((v1:=e1), (v2:=e2), e3)[-1]

Dr. Racket example:

    > (let* ((x (+ 3 4)) (y (* x x))) (+ y (- x) 8))
    50

Python equivalent:

    >>> ((x:=3+4), (y:=x**2), y-x+8)[-1]
    50

Yes, with walrus expression, lambda gains a lot of flexibility

Except :

- python scoping is not really lexical (and it captures variables, not the values they contain) - python is statement based, so lambdas are less powerful than in scheme.

So not really the same thing.

Given that Python allows unicode characters in source, I'm surprised it doesn't allow lambda to be replaced by the λ character.

I'm surprised that you're surprised.

For those, like me, who get excited by this note that there are restrictions on the Unicode categories that are allowed, see the supported characters¹ and gory details². It is often enough to write math-y code in a usable way, but occasionally you'll find you can't use the character you want.

¹ https://docs.python.org/3/reference/lexical_analysis.html#id...

² https://www.python.org/dev/peps/pep-3131/

That would be quite unpythonic. Down that road lies Raku.

EDIT: To be clear, I like Raku. But Python has its own distinct aesthetic.

Well you can't do `l = lambda` to replace it either, it's not really anything to do with allowing full unicode identifiers.

Also by the lack of persistent data structures with structural sharing. That forces you to take an expensive copy whenever you want a modified version of some data, or bash it in place.

Isn't it a true functional way?

I could definitely be wrong but I think most functional data structures aren’t fully copied. Instead, they’ll utilize something like a pointer to the data that stayed the same so only the part that changes is “copied”.

There are smarter ways to deal with data structures in the functional setting than copying the entire structure over and over again.

See https://en.wikipedia.org/wiki/Purely_functional_data_structu... and Okasaki’s book Purely Functional Data Structures.

There's Pyrsistent[1], which provides persistent data structures.

[1] https://github.com/tobgu/pyrsistent

I imagine it is problematic that Python is already very slow, and then to pile up idioms with performance downsides on top of that is kind of rough.

Functional programming doesn’t have an inherent performance downside, but it depends on specific optimizations in compilers and runtimes to make it fast. Python doesn’t really optimize anything from what I know.

This is one of the biggest reasons I really like (the python superset) Coconut: https://github.com/evhub/coconut

It looks interesting, but I’m not entirely sold. All the examples are about math functions, which is kind of stupid to implement in Python.

How does this benefit “plumbing”?

My biggest problem with Python is that I either have to write

list(function1(*function2(len(obj.method())))

Or try to avoid all the variables by using classes... which is fine... if it wasn’t for self taking up roughly a third of the word count.

Can you elaborate on this? What is wrong with Python's `lambda`?

It can only have one line for example.

no, not correct

Python’s `lambda` can only contain a single expression.

There’s no good way to add support for full anonymous functions to Python’s grammar. One of the rules that makes significant-whitespace work elegantly is that statements can contain expressions, but never vice-versa.

Plenty of languages with significant-whitespace have multiline anonymous functions, like Haskell, Standard ML, Ocaml, etc. Maybe it's no possible in Python for a student reason, but the reason is not that the syntax is bad on indentation.

The syntaxes of those languages are fundamentally different from Python’s. They don’t have an “expressions may not contain statements” rule - they don’t even have statements.

It sounds like you agree with me: the reason Python does not have multiline lambdas is not that it has significant-whitespace, but other decisions including the “expressions may not contain statements” rule.

Nitpick : neither ocaml nor SML have significant whitespace.

You're right! It's been too long since I used them. I think I was probably remembering F#'s light syntax.

since lambda is for simple and short anonymous functions most of the time why do I need type the whole word each time? can they also do what javascript does(or similar):

    x => x * 2

instead of

    lambda x : x * 2

Until very recently, Python’s grammar was strictly LL(1), so the parser couldn’t handle, for example, `(x, y) => x * y`.

Perhaps with the move to a PEG parser, this syntax could now be supported?

I like:

    λx: x*2

Like others have said:

- Can only have one line

- Can only use expressions, not statements. E.g. `print`s, loops, conditionals are out.

- Overall just kinda clunky

Here's an SO post about lambdas where the answer is "Use def instead." https://stackoverflow.com/questions/14843777/how-to-write-py...

The lambda calculus is Turing complete, so in theory, Python’s lambdas should suffice...

I mean... You're right. They suffice. They're just less friendly (i.e. useful) than lambdas in other languages.

Which of course says nothing about usability and readability.

Turing completeness is completely unrelated to whether or not anonymous functions are easy to use in Python.

print() is a function now and you can use it with lambdas.

Even in Python 2, you can do something like this:

    println = lambda s: sys.stdout.write(s + '\n')

Not that it really makes things much better, but, at least it shows you can do it.

Exactly. Kind of an own-goal too, from none other than ex-BDFL. While not totally obvious, it's not impossible to widen the syntax to allow multi-line lambdas, you just need to ditch the stack-based lexer-integrated whitespace sensitivity behaviour.

This works for conditionals:

    In [4]: f = lambda x: "Yes" if x else "No"

    In [5]: f(True)
    Out[5]: 'Yes'

    In [6]: f(False)
    Out[6]: 'No'

It's Python's version of ternary operators, so not sure if that counts as a "true" conditional; but it is one.

Loops don't work, but list comprehensions do, and they are definitely the way to go here. Multi-line loops deserve a `def`.

> Can only use expressions, not statements. E.g. `print`s, loops, conditionals are out.

print is a function (and thus can be used in expressions)

python has conditional expressions (<true-val> if <cond> else <false-val>)

loops are a limitation, though comprehensions, map(), functools.reduce(), and the itertools module can allow lots of looping functionality in an expression.

The use of lambda is becoming somewhat of a smell in Python in general. PSF's own black code formatter will complain about using it and pretty much always says to just use a def instead

Also the recursion limit.

And no tail-call optimization.

Intentional to improve tracebacks. There's trade-offs with every design decision.

Reply on news.ycombinator.com