Hacker News Reader

>> If you are used to call backs, promises, tick() calls or closures, it will be natural to reach for them a lot, and be disappointed at their state in the Python ecosystem.

It's a real disappointment that Python has no answer for JavaScript fat arrow.

The fat arrow is the one construct that completely changes the structure of all the programs I write - it makes things extremely terse and I use it in nearly every function/method in some way.

I'm a big fan of Python's whitespace structure but it seems to be an obstacle to creating single line functions that are both powerful and get multiple things done, which sometimes makes a program easier to read and more understandable. Sure there's lambda functions but I find them clumsy and hard to work with compared to the beauty of the fat arrow.

The rare case where you do need a fat arrow, it is indeed elegant once misses from Python.

But I also know by experience that people from other languages are reaching for anonymous functions way too often instead of using the tools that are suitable for the use case in python.

E.G: if you use a lot of map() and filter() in python, you are doing it wrong.

> If you are used to call backs, promises, tick() calls or closures, it will be natural to reach for them a lot, and be disappointed at their state in the Python ecosystem.

Maybe - but at the same time, generators are side-effectful and quite general. That means, when someone uses a generator, now I have to understand what happens by looking at the code. If someone uses a promise or a closure/lambda, then the scope of what can happen is much more limited.

It's like looping vs. mapping. You can use "for ... in ..." syntax or you can do "map(..., ...)" but I generally prefer the latter (minus the syntax overhead in python), simply because I easily see at a glance what the intend is.

Generators are such a game-changer for clean separation of data traversal from data processing. It's my favorite trick in python and I miss it dearly in other languages.

In languages without generators it's easy to write code that traverses some data structure in hard-coded way and does something configurable on every node, but it's hard to write code that traverses a data structure in configurable way and does some hardcoded thing on every element.

In Python it's trivial both ways and even both at once.

> it's hard to write code that traverses a data structure in configurable way and does some hardcoded thing on every element

Wouldn’t this just be a function that operates on an iterator? I suppose generators make the creation of lazy iterators easier, but generally the solution for languages without generators is to have your traversal build a list. So you lose the laziness but the code remains simple. Then map your per-node processing to the list.

> generally the solution for languages without generators is to have your traversal build a list

Yes, but then you move from O(1) to O(n) in memory usage. And if you want to optimize it you have to restructure your code, when in python lists and generators are drop-in replacements.

Also, producing a list doesn't work very well as a substitute for infinite generators.

Until you want to walk the generator twice -- stateful generators are a design flaw, in my opinion.

The C++20 coroutines has the potential of being this and much more. As with many things C++, its adoption will be very slow (they are supported by latest compiler versions, but there is no support in the standard libraries yet).

> It's my favorite trick in python and I miss it dearly in other languages.

Which other languages? Generator support is common:

https://en.m.wikipedia.org/wiki/Generator_(computer_programm...

This list really sort of stretches the imagination on what language level generator support looks like.

Certainly in Java you can use the `Iterator` interface to get generator like behavior. But that's both a whole lot more complex and more verbose. Primarily because you have to coordinate the `hasNext` method with the `next` method.

With python and yield syntax, that naturally falls out without a bunch of extra code juggling.

Sure, some of those (C, Java, C++ templates) are pretty weak, but C#, Ruby, ES6, and lots of the other language or library implementations are not. Its far from a unique Python feature.

Java methods like Stream#filter and Stream#flatMap are easier than rolling your own iterator; you can even ask that work be done in parallel.

"In Haskell, with its lazy evaluation model, everything is a generator"

How absent is generator support in languages in general? As far as I know, even PHP has had yield for something like a decade, and JS for longer (at least as far back as 2006, when Yield Prolog got discussed on LTU[0]). Or is there something about the Python approach to generators that's distinctive?

[0] http://lambda-the-ultimate.org/node/1732

The Ecmascript 4 proposal included the `yield` keyword, but the proposal as a whole was abandoned in 2008. That’s why the version names jump from ES3 to ES5. Some of the features then returned in later versions, including generators with ES6 in 2015.

Language tends to converge on good features, luckily.

Although some generator implementations are not as complete as python ones, and the ecosystem generally donc have much support for it

E.g : Very few languages have an equivalent to itertools.

This sounds interesting. An example would really be helpful, I have a vague idea of what you're talking about but have a hard time concretizing it.

You have (psuedo)code like this:

    def print_every_file_in(root_path):
       for path in children(root_path):
           if is_file(path):
              print("file: " + path + ", size:" + get_size(path))
           else:
              print_every_file_in(root_path)

If you want to extract the printing part it's trivial in any language:

    def print_file(path):
       print("file: " + path + ", size:" + get_size(path))

    def print_every_file_in(root_path):
       for path in children(root_path):
           if is_file(path):
              print_file(path)
           else:
              print_every_file_in(path)

And you can easily parametrize what function to call on each node.

But in Python it's equally trivial to extract the traversal:

     def every_file_in_path(root_path):
       for path in children(root_path):
           if is_file(path):
              yield path
           else:
              yield from every_file_in_path(path)

     def print_every_file_in(root_path):
        for file in every_file_in_path(root_path):
           print("file: " + path + ", size:" + get_size(path))

or even extract both:

     def print_every_file_in(root_path):
        for file in every_file_in_path(root_path):
           print_file(file)

which for me is a very clean way to implement this and it's nontrivial to do in languages without generators.

It’s also trivial and easy in Haskell — you just need an instance of `Foldable` or `Traversable` on your collection, and then you can fold or traverse it in a configurable way. Or for recursive structures, use https://hackage.haskell.org/package/recursion-schemes. Or even just pass a traversal function as an argument for maximum flexibility.

A big problem with many of these features is that they almost always end up getting used in ways that makes the code more convoluted when there are simpler alternatives.

needing to store state - use a class

needing lazy evalutaion - use a class (though this is a common enough use case to be a primitive, e.g. Kotlin)

decorators - if you pass decorated methods around -> now you have to remember the context the methods hold. Instead, refactor your code to be more direct so that you don't need decorators.

type hints - they are not fully enforceable at compile time, unless everything in your codebase has type hints. This is never the case and type hints can easily lull you into a false sense of security.

functools module - generally I'm ok with this but sonetimes you have to be aware of when you need to deep copy vs when you don't etc.

I don't know many languages, do decorators show up in any others? I _love_ them in Python.

Yes. In fact, E.G: C# and JS have them, although they don't have exactly the same features.

edit, sorry I've meant generators. Incidentally, it's also true for js decorators.

How do Python generators differ from JavaScript generators?

Yield from comes to mind.

Isn't JS yield* the same as yield from in Python

Unless I'm mistaken, it propagate pulling, but not pushing with send(). I haven't checked though.

Decorators are still a pending proposal in ECMAScript, so they would only be available to transpilers. That makes them non-standard until the proposal is approved, where the implementation could change.

In C#:

    static IEnumerable<int> Fibonacci()  
    {  
        int first = 0;  
        int second = 1;  
      
        yield return first;  
        yield return second;  
      
        while (true)  
        {  
            (first, second) = (second, second + first);  
            yield return second;  
        }  
    }

Interesting use of tuples I had never considered before, a lot of state is packed in that `(first, second)...` block! Pretty fun though.

Emacs Lisp* has a system called "advice" that's maybe a bit clumsier than Python's, but is also more powerful. The relationship between decorator and decorated [is configurable][0]. For example, the decorated function could be called conditionally on the result of applying the decorator to the arguments, something like:

   def _combined(args):
     return decorator(args) and original(args)

It can also be applied retroactively, not just at the decorated function's definition. This is extremely handy in the context of configuring your text editor, but I admit I wouldn't like it in a program I was working on with other people. (Basically a form of monkeypatching.)

* Granted not really used for general-purpose programming like Python.

[0]:https://www.gnu.org/software/emacs/manual/html_node/elisp/Ad...

Emacs Lisp has macros which are a lot more elegant (not to mention powerful) than Python decorators which comparatively, are nothing more than a kludge.

Depends on what you mean. I often hear claims that you can use decorators in language X, which are formally true, since decorator is just a function accepting and returning another function, so any language that has anonymous functions (which is a lot of languages by now) "has decorators". But for me this is kind of false marketing, because there's nothing impressive about that, and the killer feature of Python's decorators is the syntactic sugar, which makes them so easy to use. And in this sense not so many languages "have decorators". You can do something similar with annotations in PHP, but I've never actually done this in a real project, because that feels like abusing the language.

One thing that really annoys me about PySpark is Python's lack of tuple unpacking in lambdas. It turns what should be relatively readable pipelines of transformations into an unreadable mess.

Alas, Python 2 had tuple unpacking, but they removed it in Python 3 :(