How PEP-572 would change the standard library | Hacker News Reader

How PEP-572 would change the standard library

github.com

38 pointsrbanffy65 comments

It's going to look quite alien at first, but this is a great demonstration of how this will actually help the language. The regexp examples are a common one always felt overly verbose when I do that. The risk is that it makes it very easy to write overly terse code, I don't think I'm the only dev who's ever written a "clever" list comprehension only to come back to it later and struggle to get my head round it.

Seeing this in action, I think I kind of hate it. One of the basic aspects of idiomatic Python that I like the most is the fact that doing multiple things at once is discouraged. Multiple assignment (x,y = a,b) is possible but not typically done except where it conveys special intent.

This seems to be replacing

    foo = check_foo()
    if foo:
        do_foo()

with

    if (foo := check_foo()):
        do_foo()

which goes against that idea.

but at the same time, having foo exist only inside the block that actually makes use of it seems a good idea

This can replace that, sure, but I think it's more meant to be replacing things like:

    foo = check_foo()
    while foo:
        do_foo()
        foo = check_foo()

With

    while foo := check_foo():
        do_foo()

which is a case that hasn't had an elegant solution in the past. Another, if example, where you want to call a function based on the result of a check, but don't want to make that check more than once:

    foo = check_foo()
    if foo:
        do_foo(foo)
    else:
        other_foo = check_other_foo()
        if other_foo:
            do_other_foo(other_foo)
        else:
            # Continue

with

    if foo := check_foo():
        do_foo(foo)
    elif other_foo := check_other_foo()
        do_other_foo(other_foo)
    elif .....:

Yes, all of this has been possible and there are other ways to do it, but this expresses the problem very concisely and has less nesting at the cost of a slightly more complicated syntax. I find these rewritten examples easier to parse than the originals.

Pulling things out of comprehensions is pretty neat too.

Other than a few cases (e.g. these) I don't see this suited or intended for extremely wide usage, but no worse than any other complex idiom that people can overuse.

I agree the check_foo() duplication was bad, but was this workaround really so bad?

    while 1:
        foo = check_foo()
        if not foo: break
        do_foo(foo)

Its only downside was taking 2 extra lines, not actual understandability.

It makes the intent of the loop less clear, since now the condition is tucked away somewhere in the middle of the body, even though it's really a precondition.

I'm torn on this, because although it's easy to say that for a toy example like this, when these "preconditions" evolve over time, you get things like:

    while 1:
        now = time()
        if now > prev_time + UPDATE_TIME: report_progress()
        foo = check_foo()
        if not foo: break
        if foo < 0: print_error(); break
        do_foo(foo)
        prev_time = now

Now, was it really dictating a precondition? Or was it actually dictating a postcondition? Or both or neither? Would you really want these responses to dictate the locations of the condition in the syntax? Because it seems to me that forcing yourself to put these conditions in the loop condition just makes the code extremely ugly and difficult to evolve (requiring you to duplicate code or have very convoluted expressions with assignments).

The biggest thing this gives, IMO, is the ability to filter on a computed value in a comprehension, and then use that computed value as the result. It's something that's needed quite often if you use comprehensions heavily, and there's no good workaround for it currently (you can use a nested "for" over a 1-element tuple to create a named binding, but it's much uglier, has more overhead, and readability is basically non-existent).

I agree that's a good feature, but feel it's hard to make comprehension examples that look elegant enough to avoid obvious sniping about how the feature makes it "unreadable".

    foo = check_foo()
    if foo:
        do_foo()

Why would you ever write that?

Just

    if check_foo():
        do_foo()

What if you need to use the value of 'foo' more than once?

Then I would make that clear from the example

    foo = check_foo()
    if foo:
        do_foo(foo)

PEP-572 has a very good section that talks about how people love to bike-shed about examples. Examples that are too simple are ridiculed with "I wouldn't do it that way", and examples that are too complicated are ridiculed with "that's far too convoluted to be a common use-case".

I think this comment falls along this type of bike-shedding.

Obviously we disagree on where the example lands. My original post was in good faith.

"Why would you need a new construct for something that is already solved?"

To me, adding the variable assigned before the if-statement, inside, or after the if-statement would have made it clear why one might write the original code that way, and why a new type of assignment could be helpful.

Looking at the downvotes, that is apparently not a common confusion.

> which goes against that idea.

Then remove 'for' as it iterates and checks for a condition at the same time

Remove list comprehensions, as they do multiple things in one single statement

Remove map/filter/zip as they iterate and apply a function

Remove the '+=' operator (though x = x + 1 still do more than one thing at the same time)

I think it's the re package's fault. It's badly designed.

re.match should return a match object no matter what, and .group() should return strings, empty string if non were matched.

We don't even need anything remotely :=

What about regular expressions that match the empty string?

There might be ways the regular expression library could be designed better, but making re.match return a match when there's no match is not an improvement.

No match is also a match.

The problem here is that it's bad design design if a function returns multiple different types of result. Generally there should only be one type of result to make handling reliable. It if's neccessary, you can still test for an empty result, making it the optional case that it is.

and what's the use case for that?

The main problem here is that you can't chain re.match().group() because None has no .group()

Why do you want to match empty string with a group?

The use case for a regular expression that matches the empty string is detecting whether something starts with an optional pattern. E.g. '(foo)?' to match the optional prefix 'foo'. FWIW if you really want re.match().group() to always return a string, you can just wrap all your regular expressions with '(...)?'.

The reason why someone might want to be able to distinguish between "no match" and "the match is empty" is writing code that takes any arbitrary regular expression and uses it as a filter. Then someone else could pass in an expression that matches empty and everything works together nicely.

Do you have any arguments to justify that? It sound very reasonable to only return a match object when you have a match. Otherwise, you give more responsability to the Match class (knowing when it is None and handling many special cases) while you could just use None.

This looks awful. It was an upside of Python that it had simple syntax, so that it had less cognitive overload. Without at least adding proper scoping and variable declarations to at least add functionality, having two assignment syntaxes seems worse than not having it at all.

This is so ugly, especially the C-like parens. I wish they had gone with the pythonic as syntax:

   if rawdata.count("\n", i, j) as nlines:
      self.lineno = self.lineno + nlines
      ...

I agree this one is very elegant, but the downside is that you'll still need the parens if you use it as a subexpression, which you most likely will if you want to distinguish between, say, 0 and None.

Using 'as' is prettier, but a pain to read, as your eyes have to go all the way to the end of the line

You can still use the old way if you think this new one is ugly, but to be honest I think it's going to be an improvement.

I don't understan what you mean that you have to "go all the way to the end".

with

    if my_data := my_function():
        do_stuff(my_data)

the left-hand side of the assignment is completely irrelevant to the if statement. Here you have to go "all the way to the end" just to see what that clause of the if-statement is.

With the as-syntax, the if-statement and the clause are placed together.

However, I do find the if-as sentence kinda weird, when trying to understand it using normal english.

> the left-hand side of the assignment is completely irrelevant to the if statement

I guess is subjective? To me, that's the first thing I want to know about an assignment -- what is being assigned to? Especially if you try to keep your programs reasonably functional, then it's in fact the thing that matters, since the right-hand side would be side-effect-free.

Another way I see it is that an assignment expression is like solving a problem: first you'd ask "what did they solve for?", then you ask "how did they solve for it?" -- not the other way around.

Just go back to the original example

> if rawdata.count("\n", i, j) as nlines:

I have to read the beginning of the line, then jump to find the as (which is not very obvious in the expression) then find the var name.

> if nlines := rawdata.count("\n", i, j):

Here I do a continual motion from the beginning of the line up to the start of the condition

> the left-hand side of the assignment is completely irrelevant to the if statement

It is relevant to the subsequent block of code, I need to know what 'nlines' mean.

I agree. This makes me think about the elegant "attached as" syntax [1] of Eiffel:

if attached rawdata.count("\n", i, j) as nlines then ....

[1] https://www.eiffel.org/doc/eiffel/Void-safety%3A%20Backgroun...

The parentheses are actually not required: https://mail.python.org/pipermail/python-dev/2018-July/15433...

Why not "as" syntax: https://mail.python.org/pipermail/python-dev/2018-July/15433...

(I'm just pointing to the given explanations for the interested. I'm not necessarily agreeing with them.)

Note that this doesn't actually encompass all the possible usages of the PEP-572 operator in the standard library, of which there appear to be 312 more candidates (and a further 100 in the rest of CPython):

  root@8d88e1e678b0:/cpython# astpath -A 1 "//Assign[(targets/Name/@id = following-sibling::*[1][name(.) = 'If']/test/Name/@id)]" > pep-572.txt
  root@8d88e1e678b0:/cpython# grep "Lib" pep-572.txt | head -20
  ./Lib/argparse.py:284   >        help = self._root_section.format_help()
  ./Lib/argparse.py:285            if help:
  ./Lib/argparse.py:2275  >        a = [action for action in positionals
  ./Lib/argparse.py:2276                if action.nargs in [PARSER, REMAINDER]]
  ./Lib/cmd.py:300        >                    doc=getattr(self, 'do_' + arg).__doc__
  ./Lib/cmd.py:301                             if doc:
  ./Lib/cmd.py:356        >        nonstrings = [i for i in range(len(list))
  ./Lib/cmd.py:357                                 if not isinstance(list[i], str)]
  ./Lib/compileall.py:123 >        mo = rx.search(fullname)
  ./Lib/compileall.py:124          if mo:
  ./Lib/dis.py:386        >    show_lineno = linestarts is not None
  ./Lib/dis.py:387             if show_lineno:
  ./Lib/dis.py:403        >        new_source_line = (show_lineno and
  ./Lib/dis.py:404                                    instr.starts_line is not None and
  ./Lib/enum.py:86        >                already = set(value) & set(self._member_names)
  ./Lib/enum.py:87                         if already:
  ./Lib/enum.py:153       >        invalid_names = set(enum_members) & {'mro', }
  ./Lib/enum.py:154                if invalid_names:
  ./Lib/enum.py:848       >    negative = value < 0
  ./Lib/enum.py:849            # issue29167: wrap accesses to _value2member_map_ in a list to avoid race
  root@8d88e1e678b0:/cpython# grep -E "\.py\:[^w]+>" pep-572.txt | wc -l
  412
  root@8d88e1e678b0:/cpython# grep -E "/Lib/.+\.py\:[^w]+>" pep-572.txt | wc -l
  312

For some reason the opposition on this matter is really vocal. I’ll make the controversial but extremely low risk prediction that this feature won’t be the death of Python and that most will come to find it relatively harmless like fstrings once the pundits are done milking it.

I think a large part of it is that if you've read thousands of lines of python code and have all the syntax in your head, it's painful and annoying to suddenly be reading python code that is and looks different. I found this commit really grating to read somehow but I reckon in 3 months it'll be natural.

Very nice, although I would hate to be the one to review that patch. A large number of small "fiddly" changes, it's easy to be afraid that copy/paste occurred and that some error snuck in.

That patch is not meant to be applied. It has been created to demonstrate the difference that the new syntax would make on code actually used in production.

As someone who is not very good with grammars of programming languages, what is wrong with having

    if a = b():
       foo(a)

I know it doesn't work, but why couldn't it?

Too easy to typo when you mean

    if a == b():
        foo(a)

This is addressed in the PEP: https://www.python.org/dev/peps/pep-0572/#why-not-just-turn-.... Their argument seems to be that 'if a = b()' is too similar to 'if a == b():' and thus doing that would cause more mistakes, whereas at the moment (?) they are caught by the parser.

Yes. This mistake is very common in C. So much so that it's considered good practice to put the constant first when doing equality comparisons:

    if (10 == x)

This way accidentally missing a "=" triggers a compiler error instead of quietly doing the wrong thing.

Called Yoda conditions.

https://en.wikipedia.org/wiki/Yoda_conditions

Am I wrong to assume there was nothing wrong with the previous code and these changes are gratuitous (e.g. just because we can)?

Why not take a look at the examples in the PEP?

I have but this doesn't answer my question:

Am I wrong to say that a bunch of code that works is changed for no other reason than to exercise the new syntax?

No, you are not wrong to say that. Mainly because the pull itself explicitly says that it is using the standard library as an example of how the syntax could be used, with no intention to be merged.

Decorators, context managers, list comprehensions, multiple assignments, keyword arguments are all gratuitous. Should we remove them?

Which ones are you referring to?

These are all merely syntactic sugar that replicate functionality available in Python that would otherwise be done in a more awkward way. This PEP is doing the same thing.

My question is about the value of this particular patch-set not about usefulness of the PEP.

"The point of this PR is just to open discuss on coding style: discuss when assignment expressions are appropriate or not."

There is no new code and the author's intention seems to be more than a "let's see how it would look like" patch. (Discussing where to use new syntax on existing code is a waste of time unless the intent is to apply the changes)

`do while` loops are (effectively) coming to Python. Yay!

I was going to say - its application in this while loop is really nice I think:

https://github.com/python/cpython/pull/8122/files#diff-0ad86...

Unfortunately it's also directly at odds with Python's own EAFP [1] philosophy, which is to raise exceptions rather than return special values (StopIteration etc.). It's no longer clear to me which one the proper Python coding style is. Are we supposed to switch to returning special values instead of raising exceptions now?

[1] https://docs.quantifiedcode.com/python-anti-patterns/readabi...

Where do you see it returning special values? While loops have always stopped on a falsy value.

The issue is the readBlah() methods. What should they do when they can't read anything? They can return either 0 or an empty result (non-special falsy value), return -1 (special truthy value), return None (special falsy value), or raise an exception. In this case it seems they chose the first option, but this is at odds with something like next(iterator), which similar but raises an exception. It's not clear to me which is the proper pattern anymore, now that they're encouraging assignment.

These seem to be relatively thin wrappers around existing APIs.

A more typical way to read from a file-like object is argumentless .read() to get all content, or iterating over it to get it line by line, but if you're passing an argument to .read you're doing something fairly low-level anyway.

EAFP is already violated in a few cases where it makes more sense. This doesn't add new violations, it adds a cleaner way to work with them.

I can't wait for EAFP to die.

Awful syntax. If then =?, not =: which already is the pascal assignment and perl binding operator.

I think it was meant to mirror the Go syntax? Or maybe they both came to the same decision. Personally I think it looks "wrong" for Python (which has for better or worse always opted for word-based syntax), but I disagree that it's awful.

Welcome to Python, where proponents will proudly claim that the language is easy to read and intuitive.

I was reading the discussion above, where they argue whether a statement like

  if (foo := check_foo()):
        do_foo()

is readable. The entire time I’m thinking Willis. Tango. Foxtrot! This is the kind of stuff us C / AWK programmers do all of the time precisely because it’s concise, elegant and intuitive (not to mention well understood), and these guys are struggling since it turned into a huge discussion. Made me wonder if we’re in a programming kindergarten here or what?

This is also the kind of stuff that you C programmers write bugs with all the time, by mistaking = with ==; to the point where most C compilers these days would just flag something like if(a=1) as a warning with default settings.

Really, all this mess is the legacy of poor choice of = and == as operators back in C (or rather, B). Pascal really had it right with the assignment being very distinct from comparison, such that it's pretty much impossible to confuse the two. The sole reason for that design decision in C, so far as I know, was that assignments occur slightly more often than comparisons, and so they wanted it to be a single character. But I seriously doubt that it was worth all the bugs.

“This is also the kind of stuff that you C programmers write bugs with all the time, by mistaking = with ==; to the point where most C compilers these days would just flag something like if(a=1) as a warning with default settings.”

No it isn’t; we’re not beginners.

I learned Pascal before I learned C. Hated the retarded := as superflous.

If that were true, exist Yoda conditions would not.

This is an assignment operator, so why shouldn't it use an established syntax for an assignment operator?

Didn't read this pep, but have they considered C-like assignment, like:

    if (foo = func()) is not None:
        ...

I find this more readable and uniform than using := construct.

This can be easily confused with an equality comparison. That's why some in languages where assignment returns values favor "Yoda conditions"

Because it's easy to accidentally type foo = func() when you meant foo == func(). That mistake happens so often in C that modern compilers warn you unless you put an extra set of parentheses around it.

It would also cause ambiguity with keyword arguments. You'd have to write foo((x = 3)) because foo(x = 3) already has a meaning. I think foo(x := 3) is clearer.

Maybe it would have been best to use := for all assignment. But it's too late for that.

Reply on news.ycombinator.com