Hacker News Reader

Coming from Perl, I used to want this badly, but then I thought that there's absolutely nothing wrong with

    m = re.match(...)
    if m is not None:
        pass

Now, I wonder what you meant saying that single-line version is less error-prone, because I don't think so. I believe they're exactly the same in this regard, except for a bizarre case when someone would bastardize the code by putting some irrelevant lines between the assignment and the comparison, obfuscating the logic.

Also, as I tend to use `if m is not None:` rather than just `if m:` (because I typically don't want to run all that subtle `__nonzero__` magic), it would - subjectively - look less pretty in the one-line version: `if (m := re.match(...)) is not None:`.

Use in the generator expressions to create aliases/shortcuts and avoid repetition is great, though. I love this use case.

The issue, in my opinion, is when you want something like this:

    while m := f(many, args):
        # do stuff with m

Now, if you're writing this in Python 3.7, you often end up with some code duplication:

    m = f(many, args)
    while m:
        # do stuff with m
        m = f(many, args) # duplicate

Or something like this:

    while True:
        m = f(many, args)
        if not m:
            break
        # do stuff with m

Personally, I consider the last version to be the most elegant, but still something of an anti-pattern. So I welcome this new operator.

I literally wrote your last example twice last week to hash files in 4k chunks. Those 4 lines would be reduced to 1 with the walrus operator. I welcome it, as well.

Edit: 1 line, not 2 (excluding the "do stuff")

This comes up in recursive descent implementations as well https://gist.github.com/cellularmitosis/53913ad229bad5d0a3bb...

Why not something like this:

  def f_iter(many, args):
      while True:
          m = f(many, args)
          if m:
              yield m
          else:
              raise StopIteration
  ...
  for m in f_iter(many, args):
      # do stuff with m

This way you’re isolating all the initialization logic, error handling, etc. And you can focus on your domain logic in your client code.

And less reasonably, something similar for regular expressions:

    for m in filter([re.match(...)], bool):
        ...

;-)

so i should write a whole other function that iterates over a list instead of being happy that := exists?

Yes! No pain, no gain!

Instead of raising StopIteration you could also just return from the function. You could also make it a bit shorter if you swap the if branches.

  def f_iter(many, args):
      while True:
          m = f(many, args)
          if not m:
              return
          yield m

Instead of raising StopIteration you have to return from the function, otherwise you'll just get a RuntimeError starting with Python 3.7.

You're right. The [changelog] for Python 3.7 states:

> [bpo-32670]: Enforce [PEP 479] for all code. This means that manually raising a StopIteration exception from a generator is prohibited for all code, regardless of whether ‘from __future__ import generator_stop’ was used or not.

[changelog]: https://docs.python.org/3.7/whatsnew/changelog.html#id115

[bpo-32670]: https://bugs.python.org/issue32670

[PEP 479]: https://www.python.org/dev/peps/pep-0479/

Python has built-in facilities for that, there is no need to write wrapper by yourself.

   from functools import partial
   with open('mydata.db', 'rb') as f:
       for block in iter(partial(f.read, 64), b''):
           process_block(block)

That last one isn't an anti-pattern. That's the pattern concluded best for do-while in the discussion of PEP-315.

https://www.python.org/dev/peps/pep-0315/#notice

The last version clearly exposes that you have an infinite loop. This is something hidden by the `while(evaluate)` expression.

Also if it comes to pure form I would have preferred `while (evaluate) as x:` out of establishing a parallel with existing syntax, but that's not very important.

On the contrary: the loop isn’t fundamentally infinite, it’s just an artefact of the language that you have to write it that way. The walrus lets you put the termination where it belongs.

I don't understand that the walrus has any significance with regards to the `while` loop termination. it's the nature of the evaluated expression that makes a difference, more precisely, whether it will return something that is cast to False.

In that sense, while loops over expressions always require knowledge of the expression to understand the semantic. On the other hand, something like:

    while True:
       with x as y:
           print(y)

would not require understanding of x to understand the intent. Therefore it is not semantically equivalent to:

    while x as y:
        print(y)

Note that in both cases, `:=` is strictly equivalent to `as` as it is defined for `with`. That is a matter of personal preference.

If you only have one or two regex like that then there is no difference in readability. If you have ten in a row though then suddenly your function no longer fits on the screen and it’s more difficult to understand what’s going on.

I mean, what you really want in this case is Perl's =~ operator, as well as the implicit $_ variable.

Think about if you match multiple patterns:

    for line in f:
      if (m:= pat1.search(line)) is not None:
        ... do stuff ..
      elif (m:= pat2.search(line)) is not None:
        ... do other stuff ..
      elif (m:= pat3.search(line)) is not None:
         ... do something else ..

In older Python that's:

    for line in f:
      m = pat1.search(line)
      if m is not None:
        ... do stuff ..
      else:
        m = pat2.search(line)
        if m is not None:
          ... do other stuff ..
        else:
           m = pat3.search(line)
           if m is not None:
             ... do something else ..

I think the newer makes it clear that it's supposed to be a simple elif chain, where all branches following the same structure.

There are other ways to structure it, but the alternatives I can think of also have their own cumbersome complexities.

That's... lovely. I propose making this the standard example of why you might want to use the walrus operator.

If actions are different, I tend to put them in functions (independent testable pieces, yay!) and iterate over a mapping:

    LINE_ACTIONS = (
        (re.compile("pattern 1"), do_stuff),
        (re.compile("pattern 2"), do_other_stuff),
        (re.compile("pattern 3"), do_something_else),
    )
    ...
    for pattern, action in LINE_ACTIONS:
        m = pattern.search(line)
        if m is not None:
            action(m)
            break

Or even use a method registry pattern that would auto-populate LINE_ACTIONS by just declaring the fuctions:

    @action("pattern 1")
    def do_stuff(m):
       ...

Alternatively, I might just break the processing into a function:

    def _process(line):
        m = pat1.search(line)
        if m is not None:
            ... do stuff ...
            return

        m = pat2.search(line)
        if m is not None:
            ... do stuff ...
            return

        m = pat3.search(line)
        if m is not None:
            ... do stuff ...
            return

    _process(line)

Of course, this depends on the purpose. Could be completely inadequate in some situations.

Indeed, I've also used this pattern.

I've found that I don't like using it. As you write, it's "inadequate in some situations", which I consider as part of the cumbersome complexities I mentioned.

For examples, do_stuff() and do_other_stuff() may need to share variables, and do_something_else() might need the line number to report an error while the others don't.

This can be handled with shared state/nonlocal, and by passing in more parameters to the generic handler API, but these add complexity.

Or, different parts of the file may have different line dispatch processing (eg, a header block followed by a data block followed by a footer block) where one of the handlers must indicate the transition to a different processor.

Also, function dispatch in CPython is slow. While the regex tests are also slow, it can also be important to consider the (using a hypothetical number) 5% overhead for dispatching over inline code.

IMHO the whole `re.match` is a design flaw in stdlib.

re.match() should always return a match object, but instead .group(1) will return None. Then we can write one-liners easier without the walrus operator.

Sometimes you just check for a match (e.g.of entire string) and define no groups.

The problem is not in re.match but in None being a relatively poor Maybe / Optional (still ways better than C-style `null` of many other languages).

> Sometimes you just check for a match

re.match vs re.search

re.search is different. re.match matches from the beginning of the string, while re.search matches from anywhere in the string.

Could just have been

    if re.match(...).matched:

But yeah, agree that Optional is what I want here.

    if let Some(m) := re.match(...)
        m.group(1)...

    m = re.match(...)
    m.group(1) # TypeError because I haven't unwrapped the Optional

In current Python, that last line sometimes throws a runtime error.

I think you mean .group(0) as .group(1) can already return None:

    >>> import re
    >>> pat = re.compile("(A+)|(B+)")
    >>> m = pat.search("HUBBLE")
    >>> m.group(1) is None
    True
    >>> m.group(2)
    'BB'
    >>> m.group(0)
    'BB'

I don't see how using your proposed change would help.

I mean re.search() should always return a MatchObject, dont return None.

Looking at your wish for exploratory evaluations in for-comprehensions, I was reminded of how Clojure does this. I found it quite elegant.

The inside of the for-braces allow for keyword-based sub-clauses: There's

- ":when" which will suppress output for elements not matching a filter expression,

- ":while" which will stop when elements stop matching a filter, and

- ":let" which will let you bind some new values in mid-loop.

I rarely need these features, but when I do I find them really really helpful. Maybe someday someone will consider doing something similar in Python.

Your second use case is actually already covered in Python, assuming you have some kind of sentinel value you can break on:

  for s in iter(network_service.read, ''):
      process(s)

I think there are a lot of use-cases for some kind of dedicated "if-with-outputs" syntax, where you have some extra variables available inside the if-block if the condition matched. Such a syntax could cover a lot of the problems of double execution but also prevent hard to understand code like the walrus operator.

... I've got no idea how the syntax could look though.

How would this hypothetical syntax differ from the walrus operator? I.e. what is hard to understand with the walrus operator that wouldn't be with this new hypothetical syntax and why?

I think it could avoid cases where you "abuse" the walrus operator and make code harder to understand than necessary. E.g. things like:

  if (value := func()) and another_condition:
    ...

Or:

  if another_condition or (value := func()):
    ...

Or:

  x=1
  my_list = [x := x+1, x := x-1]

etc.

Ah ok, so you'd essentially allow it in fewer contexts? I would definitely have supported that, although I do think combining walrus with other boolean expressions in if and while is super useful and not particularly hard to understand.

It comes up somewhat rarely but it is useful at times. But of those rare times, it is exceedingly rare to require any more complexity than you've shown above, such as having two assignments in one statement.

That was the reasoning given for using ":=" instead of "as", to allow more complexity. I still think it was a mistake.

> while s := network_service.read(): > process(s)

Watch out with this one though. In this case it's good, but if you receive numbers from such function, it will break loop not just on None but on 0 as well. Easy to forget.

In many cases you'll need to do:

    while (s:= ....) is not None: