Hacker News Reader

You can achieve the same thing in python by using two lambdas (which is actually what you're doing in JS with map):

   >>> powers_of_x = [(lambda i: (lambda x: x**i))(i) for i in range(10)]
   >>> [f(2) for f in powers_of_x]
   [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]

Or you can use default parameter values to use a single lambda (though this means it can be overridden, it's not semantically equivalent to the js implementation)

   >>> powers_of_x = [lambda x, i=i: x**i for i in range(10)]
   >>> [f(2) for f in powers_of_x]
   [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]

your python snippet is equivalent to the following JS:

   >>> function* comprehension(iterable){
   >>>   let i;
   >>>   for(let j of iterable){
   >>>     i = j;
   >>>     yield x => x**i;
   >>>   }
   >>> }
   >>> powersOfX = [...comprehension([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])];
   >>> powersOfX.map(f => f(2))
   [512, 512, 512, 512, 512, 512, 512, 512, 512, 512]

And Python's comprehensions are actually a shorthand for writing the above generator with this syntax: yieldExpression for i in iterable. The semantics are consistent with the regular for..in

   >>> def comprehension(iterable):
   >>>     for i in iterable:
   >>>         yield lambda x: x**i
   >>> [*comprehension(range(10))]

is equivalent to

   >>> [lambda x: x**i for i in range(10)]

This is all a consequence of python's late binding.

Amusingly, the "default parameter" version relies on a feature that also gives rise to another WTPython entry: https://github.com/satwikkansal/wtfpython/blob/master/README...

That's a different issue around default arguments that are mutable, rather than default arguments generally.

The lambda case here is more like a default argument of a constant integer (immutable) than a default that is a list, which can be altered directly by the function it's a default of.

The underlying feature that both rely on is that default arguments are set on function declaration instead of function call. Mutable/immutable only affects whether or not you hit the bug.

Parenthetical comprehensions (generator expressions) are equivalent to your expanded function, but list comprehensions are not. `(x for x in range(10))` creates a generator, but using square brackets converts it to a list, including allocating the necessary memory. If you’re trying to do yield and streaming, the parens will serve you better.

You will notice that I explicitly unpacked the generator object in the brackets below the generator function by prefixing it with an asterisk.

I agree totally with that, it's an ugly trap in python, although mitigated by the fact the language doesn't encourage this kind of paradigm so you rarely encounter it.

But more absurd than creating a global variable if you forget "var/let/const" ? More absurd than "this" being schizophrenic ? More absurd than having no namespace for 20 years ?

That's pushing it.

After a long time using both languages, I’d argue that non-sane equality checks in js is the biggest source of bugs.

Unfortunately, this is not true. Although the equality operators are a favorite bikeshedding topic, the largest source of errors in JS code bases are type errors. Using a "non-sane" equality check, such as those with type coercion would actually mask or alleviate these source of bugs you mention.

I was thinking more of the case where the types match but objects and arrays with the same contents are considered different. I’ve watched every member of my team get stung by it again and again - and then have to create workarounds to get past it.

I don’t agree and this behavior shouldn’t be surprising. The alternative would be to walk the container and compare the value of each element, which could be horrible.

Quite the contrary, it's very beautiful and useful:

    >>> (1, 2) == (1, 2)
    True
    >>> (2, 1) == (1, 2)
    False

If you need identity for perf:

    >>> (1, 2) is (1, 2)
    False

If you just need the type check:

   >>> isinstance((1, 2), type((1, 2)))
   True

It's very explicit, practical, and you can set the scale of practicality vs performances where you want. Plus: no implicit weird type conversion, only one equality comparison operator, and no hidden rules.

I think it's sane.

I’m not sure if you’re for or against here. Walking the container is exactly what you have to do, and it is horrible. More importantly, if it’s not your library, you don’t get any choice on how the equality check is implemented.

In my opinion, if standard tooling is able to identify the foot-gun, it’s not too terrible of a foot-gun. I don’t think standard python tooling would catch the above issues, but standard JS tooling would identify a missing variable declaration. As for the other things you mention (this, namespaces), I never said JS was perfect! Just that JS got a lot of shit for its scoping, fixed the problem, and now has sane scopes. Py has scoping with a lot of the same problems JS used to have, but for some reason didn’t fix them, and thus to this day is riddled with these foot-guns.

> I don’t think standard python tooling would catch the above issues,

Of course it does.

    $ echo "
    > powers_of_x = [lambda x: x^i for i in range(10)]
    > [f(2) for f in powers_of_x]
    > " > test.py
    $ pylint test.py
    ...
    test.py:2:27: W0640: Cell variable i defined in loop (cell-var-from-loop)
    ...

Pylint is the most powerful linter for Python is well integrated in VSCode or PyCharm.

When concerns are raised about JS, people are quick to answer you are not supposed to use it without modern tooling (Webpack + babel + typescript + eslint + lowdash...), although many devs still do.

Now, I use and teach a lot of JS, and I enjoy many of its modern features. I especially like the object destructuring, it's even better than Python's unpacking.

But objectively, you can get a very decent experience in Python without any kind of tooling. And yet, there is a lot of it if you want to up your game: pylint, black, mypy, jedi, pytest, poetry, etc.

That’s good to hear. As I said, I don’t use python much anymore.

Are they really foot-guns? The lambda variable reference thing has appeared more in forums comparing languages than I’ve seen it used at all in python and I’ve spent a lot of time working in large open source python projects with thousands of contributors.

Even if you got the binding right, your powers of 2 would be shot down for being unpythonic anyway due to lambdas being shunned when there are cleaner ways to write things.

powers_of_x makes no sense as a variable name for a list of functions that you have to call in order with the same number to actually get the powers of that number. I get that it’s just a contrived example, but most can be turned into someone more apparent to the reader.

    def powers_of_x(x, limit=10):
        for i in range(limit):
            yield x^i

Less memory, no anonymous functions, more flexibility, testable...

Modules avoid the variable issue because ‘use strict’ is applied by default for them.

IMO JS as a language is in a much better place than Python currently. I end up frustrated a fair amount by arcane errors in Python due to less expressive constructs or some wtf issues like unexpected variable scoping within Python modules.

This is most probably because you are trying to code in Python like you code in JS: this always leads to frustration.

I started to have fun coding in JS the day I accepted it was not Python and that I had to structure and style my code differently.

E.G: in Python you will use iteration a lot. A lot of a lot. But not so many callbacks. The reverse is true in JS.

Iterators and callbacks aren’t comparable. You don’t use them for the same thing.

What do you think each(), map() and filter() do ?

But even without that, I'm not saying they are comparable. I'm saying the same API will use explicit anonymous callbacks in JS and something else in Python (decorators, subclassing, protocols, generators...). I'm saying that the same API will use __iter__ in Python and something else in JS (type conversion, proxy object, explicit method call...).

E.G, this is a Python pattern you'll find in contextlib or in pytest fixtures:

    @somekindofregistration
    def foo():
        print('code that runs before')
        try:
            yield
        except Stuff:
            print('Error handling')
        print('code that runs after')

This uses Python iteration mechanism to run code at 3 different times in a life cycle.

While in JS, you would pass 3 callbacks.

Map, filter, and fold are higher order functions, functions which take functions as parameters.

You are misusing the word callback. A callback is a function passed to another thread that will maybe be invoked later as a response (like it calls you back).

https://en.m.wikipedia.org/wiki/Callback_(computer_programmi...

That's not what a callback is. A callback is just a function passed to an outer function that the outer function may call.

Even MDN uses the term callback to describe e.g. the argument to 'map': https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

In contrast, Lodash goes out of its way not to use the term "callback", presumably to avoid confusion since it's a place where many functions-that-take-a-function-argument do not call the provided function, e.g. https://lodash.com/docs/#curry

> I agree totally with that, it's an ugly trap in python, although mitigated by the fact the language doesn't encourage this kind of paradigm so you rarely encounter it.

This. I've made this mistake exactly once in the last 7 years, but that one time almost caused a security issue.

This situation is mentioned in the docs:

https://docs.python.org/3.8/faq/programming.html#why-do-lamb...

"This happens because i is not local to the lambdas, but is defined in the outer scope, and it is accessed when the lambda is called — not when it is defined. At the end of the loop, the value of i is 4, so all the functions now return 42, i.e. 16."

These sort of scoping gotchas are pretty common across all programming languages and are a great argument for unit testing. As usual the answer is be more explicit what you're asking the language to do:

>>> powers_of_x = [lambda x, i=i: x*i for i in range(10)]

>>> [f(2) for f in powers_of_x]

[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]

Note the positional argument with default "i=i."

More discussion:

https://stackoverflow.com/questions/452610/how-do-i-create-a...

Isn't there something wrong if you have to explicitly state i=i?

Any feature that is a "great argument for unit testing" is a poor feature.

There is nothing bad in general with list comprehension there. Problem is lambda which uses variable from outer scope. In Python, comparing to JS, using variables from outer scope is not so common. I prefer more obvious constructions and in case of problems with scope I can even create class to protect value that I need to manage.

In a decade of python use I can probably count on one hand the number of times lambda has been a good solution to a problem I had. Many times I end up re-writing them as regular functions for clarity sake.

What's the symptom and what's the cause? Python's lambdas aren't very good, so they're frequently not a very good solution...

I don't find this at all unexpected. In an imperative language I expect a loop to be implemented by mutating a common variable.

Do you perhaps think that because you come from a background in low-level languages?

Imperative loop, maybe. But list comprehensions are a sublanguage imported from Haskell and looking like Haskell, so it's somewhat natural to be surprised by mutable variables here.

Lua and Perl use fresh variables for each iteration, instead of mutating a shared copy.

>My biggest complaint about Python is that it somehow doesn’t get flak for having the same (if not worse) scoping as JS, which gets endless hate for its function-scoped variables. (So much so that block scoped variables are the new normal in JS, but not in Py!)

It's less of a problem in Python because it has built-in module support. A global variable is only global in the module, unless you import it with "from foo import *", which lazy programmers do but is discouraged by style guides.

You forgot the i=i in your lambda definition. Then there exists a local variable i and everything works as you wanted.

Maybe I’m misunderstanding, but I’d argue that arrow syntax in js is equally special syntax as lambda function. They’re both native ways of making anonymous functions. Python’s version is more clunky, but no more special

Aren't Python's lambdas expressions while JS's arrow functions are not?

Arrow functions are expressions in JS.

Surely they have to considered expressions to be assignable to a variable?

Interesting, I don’t use python day-to-day, so I had assumed that the delayed capturing of the iteration variable was a lambda thing. Looks like this is still broken:

for i in range(10):

   def mul(x):

     return i^x
   powers_of_x.append(mul)

That to me coming from a JS background is totally wild

It’s actually due to python late binding. The variables are looked up at call time, rather than function define time. One could argue that your example shouldn’t work anymore in python3 since variables in list comprehensions go out of scope when they finish now. I haven’t tried it myself. I fill it under, “things I never need to do” :-)

It's not "python late binding", closures behave this way in any language with mutable bindings[0]. The entire point of a closure is to close over its lexical context, if the lexical context is mutable and mutated before the invocation of the closure, the closure is going to reflect that change.

    for(var i=0; i<10; ++i) {
        powersOfX.push(x => x^i);
    }

will behave the same way because it does the same thing, so will e.g.

    for i := 0; i<10; i++ {
        powers_of_x = append(powers_of_x, func(x int) int { return x ^ i })
    }

in Go.

One of the confounding factors is that many languages have block-scoped for loops especially when using iterators instead of low-level C-style loops; that's also the case in Javascript when using `let` bindings (and is in fact one of the major reason to use `let` instead of `var` if that's possible).

The underlying concern is still there[1], but this very common failure case is avoided: rather than update a single binding, each iteration creates a brand new binding for the closure to close over.

Alternatively, a common mitigation technique is to emulate that using e.g. immediately invoked function expressions.

In Python you can also use the "default parameter" trick to shadow the closed-over binding (though most of the time this is used for performance reasons) without the overhead (both syntactic and runtime) of lambdas in lambdas in lambdas:

    for i in range(10):
        powers_of_x.append(lambda x, i=i: x^i)

[0] excluding the special case of low-level languages with capture clauses, as well as languages like Java where closing over mutable bindings is specifically forbidden (a lambda or anonymous class can only close over `final` bindings)

[1] and remains a regular issue in async code fighting over closed-over context

C++:

   std::vector<std::function<int(int)>> powers_of_x;
   for (int i = 0; i < 10; ++1)
       power_of_x.push_back([=](int x) { return std::pow(x, i); });

works as one would intuitively expect (a different i is captured for each iteration). The issue is with those languages that conflate values with references.

…

> excluding the special case of low-level languages with capture clauses

Of course it works, you're specifically capturing `i` by value. It's not exactly surprising that doing things completely differently yields a different result.

To be fair I missed the note at the end of your commend on my first read. Still, in the specific C++ example, no other capture clause is valid.

Also, python could have chosen a slightly different closure semantics and preserved sanity: instead of closing over the binding itself, it could close over each object reference separately (exactly in the same way the default parameter hack works).

You're right. I remembered the reason after reading your sibling comment and almost deleted my comment to avoid the confusion. Thanks for going into the details.

Works intuitively in golang:

https://play.golang.org/p/pn2jBTaTNS8

  package main
  
  import (
   "fmt"
  )
  
  // return a^n
  func Power(a, n int) int {
   var i, result int
   result = 1
   for i = 0; i < n; i++ {
    result *= a
   }
   return result
  }
  
  func main() {
   fmt.Println("Hello, playground")
   x := 2
   var powers_of_x []int
   for i := 0; i < 10; i++ {
    powers_of_x = append(powers_of_x, func(x int) int { return Power(x, i) }(x))
   }
   fmt.Println(powers_of_x)
  }

  //Output:
  //Hello, playground
  //[1 2 4 8 16 32 64 128 256 512]

I know nothing about go, but it seems to me in your example powers_of_x is a list of n integers, while in the example being discussed is an list of n functions (which can be used to compute the n'th power).

I provided a Go example demonstrating that it does not. Your version

1. uses a slice of integers which completely misses the point

2. uses an IIFE which completely misses the point

The point is to look at what happens when you close over the iteration variable.

Why would it be "totally wild" coming from a JS background when the corresponding JS code:

    for(var i=0; i<10; ++i) {
        powers_of_x.push((x) => i^x);
    }

behaves the exact same way for the same reason?

By definition, a closures closes over its definition context, it doesn't capture the values at definition time but instead keeps referring to said definition context. If the definition context is mutable and modified, the closure reflects that change when it's finally invoked.

"Excess syntactic sugar causes cancer of the semicolon."

GJS: Gerry Sussman attributes this quote to Alan Perlis.

_Structure and Interpretation of Computer Programs_: https://mitpress.mit.edu/sites/default/files/sicp/full-text/...

The real WTF here is that you are getting a power output from thee ^ operator, when it actually does xor. Indeed, I've never seen JavaScript mix up operators :)

/s

HN isn’t able to render two adjacent asterisk, so I had to make do.

If they're surrounded by spaces it goes ok...

    x ** y

Even more fundamental is the nonlocal/global stuff required in order to avoid declaring your variables. Many people are surprised by what this snippet does:

    a = 0
    def f():
        print(a)
        a = 1
    f()
    f()

All it does is produce an error: "UnboundLocalError: local variable 'a' referenced before assignment". The second f() call makes no difference.

Is that what you expected from such a simple piece of code?

Well it's f that has the problem, rather obviously, and the other three lines are unrelated. So I don't see how this is some especially simple way of triggering that error.

The other three lines are not unrelated. If you just take the first three lies (and the last one), it's fine:

    a = 0
    def f():
        print(a)
    f()

Maybe this is all obvious to you, but I'll bet you can't name any other language which behaves this way. Compare the original Python to JavaScript (or Lua, Perl, Tcl, Ruby, C, Scheme, Rust, Clojure, or ...):

    a = 0
    function f() {
        console.log(a)
        a = 1
    }
    f()
    f()

This one behaves how I think most people would initially expect the Python snippet to behave. First it prints 0, then it prints 1. (Of course JavaScript has it's flaws too...) If you don't believe me, ask your coworkers and friends what they think it does before running it.

My only real point is that Python conflates variable declaration and variable assignment in a way which initially seems like a friendly time-saver, but which ends up being pretty subtle and confusing until you've learned its quirks. All of that just to avoid declaring your variables (in some fictional version of Python):

    var a = 0
    def f():
        print(a)
        a = 1
    f()
    f()

Here it would be clear which are declarations + initializations, and which are only assignments. And for just the cost of typing the word "var", the compiler could tell you when you've made typos in your variable names. As an added bonus, you could get rid of the "global" and "nonlocal" keywords.

js also doesn't behave as you want:

  var a = 9;   
  function foo() {
      console.log(a);
      var a = 12;
  }
  foo();

  $node foo.js
  undefined

The same goes with ruby.

JavaScript is far from perfect, but at least it's clear you declared two variables.

You’re sidestepping the problem. The problem is that Python uses the same syntax for variable declaration and variable assignment, and this can lead to unexpected behavior. That’s far from normal in the programming world, making it more unexpected.

Of course I'm not a good judge, I have been programming Python since 1.4 and my colleagues also all have Python experience.

But at least I like the idea that it gives an error message when confronted with ambiguity. That way it doesn't do something you didn't expect silently. Sadly it only gives it at runtime, not at compile time.

I'm not sure what your point is about Python 1.4. Maybe it's that you're past the point where these things trip you up.

However, there are plenty of cases where Python won't give an error message too. Make a typo somewhere in the middle of your function, and it'll quietly introduce a new variable instead of letting you know. A rarely used branch of an if-statement could hide this indefinitely. This could also be avoided (or at least mitigated) by requiring variables to be declared explicitly.

Yes! So much that! I can't believe that it's okay in python to declare/initialize a variable within a loop or if statement, and that it goes to live on afterwards. JS fixed that issue years ago with let and const.