Hacker News Reader

I find myself reaching for reduce fairly often, still. What's the list-comprehension equivalent?

    map(fun, iter) --> [ fun(x) for x in iter ]

    filter(fun, iter) --> [ x for x in iter if fun(x) ]

    reduce(bifun, iter, initial) --> ???

edit: I'll also add that this is made worse by the fact that itertools lacks a sliding window function.

There's not a great `reduce` replacement with comprehensions. It'd probably be something closer to:

  last = initial
  for value in iter:
      last = byfun(last, value)

However, I don't find myself writing that frequently.

You forgot the walrus operator!

  accum = initial
  series = [accum := bifun(accum, value) for value in seq]
  # result = series[-1]

To be honest, this is the first time I've used the walrus operator.

All hail the walrus!

(Seriously, I like that.)

This is very nice. Replaces reduce() for me. Now to upgrade to Python 3.8! (stock Python on my cloud machine is still 3.7)

doesn't it use a lot of memory since we're only interested in the last value?

You can do the same thing with a generator.

    x = 0
    deque((x := x + 1 for _ in range(10)), maxlen=0)
    print(x)
    >> 10

Using deque to consume generators is an old trick documented in the itertools recipes: https://docs.python.org/3/library/itertools.html#itertools-r...

> this is made worse by the fact that itertools lacks a sliding window function

We have `itertools.pairwise()` for making a sliding window of pairs, but yes, there is no function that accepts an arbitrary number of elements.

It is not in the standard library, but I quite like more-itertools: https://more-itertools.readthedocs.io/en/stable/api.html#win...

I find it interesting that you use reduce so often. Most python functions that handle two inputs handle it generically by handling a list of inputs, so passing in a full list exploded as args is sufficient and reduce isn't needed. All of pythons built-ins at least work this way like sum(), print(), etc. As such, I have not ever reached for reduce. Sometimes a comprehension inside a function call though.

What types of situations do you use reduce for?

It's used to replace all looping. A combination of map and reduce is an alternative to loops.

Also you don't need map. You can use list comprehension to replace map. But list comprehensions can't replace reduce.

Technically reduce can also replace map. The only primitive you need to do all your iterations is technically just reduce.

If you want to get more elegant than you don't even need reduce. You can eliminate all redundant primitives and use functions themselves for iteration. It's Just recursion.

If you use reduce to replace looping, your coworkers probably hate you. Don't write Clojure in Python.

It's called functional programming.

Coworkers don't hate it it's more readable in general to split between map and reduce and forego looping.

It's less readable to use reduce for all looping even though technically you can.

Looks like it's been implemented since python 3.8 : https://stackoverflow.com/a/55890523 With the walrus operator :=

> ...and knows why you should probably used comprehensions instead.

This. The author does not seem to know that (1) reduce is not a built-in function anymore, and (2) map and filter were also planned to be removed from Python 3 [1]:

> About 12 years ago, Python aquired lambda, reduce(), filter() and map(), courtesy of (I believe) a Lisp hacker who missed them and submitted working patches. But, despite of the PR value, I think these features should be cut from Python 3000.

> Update: lambda, filter and map will stay (the latter two with small changes, returning iterators instead of lists). Only reduce will be removed from the 3.0 standard library. You can import it from functools.

[1] https://www.artima.com/weblogs/viewpost.jsp?thread=98196

I have only been writing python since Guido wrote that comment, but I have to say I don’t agree with him. I find the map/filter syntax to be very readable and intuitive. List comprehension is ok but to me becomes nearly incomprehensible (see what I did there?) where nesting is involved.

It’s easy to chain map and filter statements without losing readability. I think I might be in the minority though.

It's a "when in Rome..." situation.

Theres nothing wrong with using them but what you are going to find across the community are comprehensions. They concisely bring the data container and the iteration together with no fuss.

If you need to map and reduce and filter and want to build a chain and/or have reusable predicate functions then go for it.

I always encourage my teams to reach for the ecosystem usage before doing things because another language does it different (map/filter/reduce being the best option in JS 90% of the time).

But maybe you work at a functional shop and everyone Gets It(tm) and then it is the norm.

They both seem self evident, though I prefer the functional syntax. I don't write enough python that I don't have to look stuff up each time it comes up. C++ and Rust are my normal tools, and they are both fine having a couple ways to do something. Is the "one way to do something" matra still a guiding principle for Python? Having multiple ways means when I guess at syntax I have more chances to be right.

There also seem to be really stong opinions about what is idiomatic. I find writing python without lots of typehints and a good IDE to be infuriating, but I've been told type hints aren't pythonic.

To me, the concept of “pythonic” has lost credibility for all the reasons you mention. It used to be that “duck typing” was a point of pride, something everybody learned. Now if you’re not type hinting, I think most people consider it bad practice.

There’s a bajillion ways to do everything. We use print() now, but not raise(). f-strings or .format()? Maybe c-style string formatting? How about dict formatting? Is adding strings ever permissible? Didn’t we used to use single quotes? I thought that was pythonic too.

Pythonic doesn’t exist. What’s popular is all there is. I say let’s bring map, filter and reduce into fashion.

> I've extended Python with C for funsies, but have never, not once, needed to do it at work.

I used to do it much more in my earlier days of python 2 in around the mid-00s. It was more often the case that there was some C library I wanted to interface back then.

It’s still useful in performance situations (high iteration counts, library writing, etc). But I find myself doing it much less often now.

> I've written Python professionally for 20+ years. I've extended Python with C for funsies, but have never, not once, needed to do it at work.

Well, different folks have different experiences. One of the first Python-related things I had to do for my job was implementing a Python API for a custom C library that controlled some ancient hardware.

Sure! That's totally a thing, and lots of people do that.

But I've done tons of stuff in Flask. I wouldn't say you need to "Have a good understanding of Blueprints" to be a Python expert. Or "proficiency subclassing yaml.YAMLObject". Or "encyclopedic knowledge of Requests verbs". Or "strong opinions on Poetry vs Pipenv". Or "can use Ansible to configure a fleet of servers".

Those are all common Python subjects in the areas I poke around a lot. You can be an expert data engineer without knowing any of those things. Just like you can be an expert backend engineer without having used Pandas a lot.

>You can be an expert data engineer without knowing any of those thing

Yes, but then you're exactly that, an expert data engineer, not an expert python programmer. Extensibility, in particular C interoperability is a core aspect of the language and it's to a non-trivial extent designed around it. Much of Python's utility rests on the fact that libraries exist who make use of this feature, especially in data science!

You can be an expert software architect in Lisp and write many programs without resorting to macros, but if you want to call yourself an expert Lisper you'll need to know your macros.

I get what you're saying, but I disagree. You can write an awful lot of expert-level Python code without learning C. After all, the result of that interoperability is something that looks and acts like a Python object. It's not like users of that code have to know about C. They just write Python like usual, and it's magically (from their POV) faster than if it were also written in Python.

As far as the data engineer bit, that's in the context of this article, which asserts that you need to know libraries predominately used by data engineers in order to be an expert programmer. I disagree, because Python is no more "about" data science than it's "about" web services or platform development or infrastructure management.

> Python is no more "about" data science than it's "about" web services or platform development or infrastructure management.

Amen, brother. It's incredibly annoying how every new niche that Python accreted over the years, is full of people who firmly believe Python exists only for them. At one point it was LISP refugees, then markup junkies (like me), then sysadmins, then 3d artists (!), then webapp jockeys, and now it's datascience and machine learning. Every kid thinks the world is built for him, but pushing such limited worldviews can only box the community into a corner - just look at Ruby to see what happens then.

> I bet there are lots of people who are experts in C who've never written a line in the GCC or Clang codebases.

Eh, but any proficient C programmer should be able to inspect the generated assembly and determine if they like the results or not.

Circular references are not an issue, the GC is more than just reference counting.