PEP 505: Bringing None-Aware Operators to Python | Hacker News Reader

PEP 505: Bringing None-Aware Operators to Python

python.org

43 pointsnp_tedious75 comments

Here's an example the PEP provides:

    if libpaths is None:
        libpaths = []
    else:
        libpaths =  libpaths.split(":")

After updating:

    libpaths = libpaths?.split(":") ?? []

Python is great for beginners for many reasons, two of which are: code is obvious when read, and we all write the same way so you can go read expert code and learn from it. There is so much more to programming and Python than just optimizing keystrokes and lines of code.

In my opinion, the PEP 505 version is entirely illegible to a reasonable python beginner. And this hurts the language more than spreading the instructions across a few extra lines.

I am concerned that PEPs are always written by experts who will, intentionally or not, bias towards an expert-friendly language.

I agree. This idea sacrifices readability in favor of terseness which is hostile to the beginner and to the professional. It results in less legible code for everyone. I hope this PEP is rejected.

Another voice in support of rejection.

are python devs the most curmudgeonly/conservative devs in the world? syntax is never a barrier to legibility of code because code is not meant to be parsed one character at a time (by programmers). the legibility always comes down to the semantics and abstraction. in my opinion a conciser ternary has very clear semantics and therefore is intelligible. I'm also of the opinion that it's very useful to save on things like `get(get(a, None), None)`

Yea good luck trying to pick that line apart as a beginner.

I could imagine myself as a beginner to python looking at that and thinking things like: Why does 'libpaths?' end in a '?'? What is the ??, is it an operator or something else? Neither would be particularly easily searchable things (e.g. <your fav. search engine>: "python ??" )

> libpaths = libpaths?.split(":") ?? []

> !google c# two question marks

> !google c# question mark dot

It doesn’t really seem to be a problem.

I think that a lot of the discussion of python problems is steered by the infinitely many python book writers and course instructors that have profited significantly by the current trend of "everyone must learn to code!".

If the recent changes to python syntax were to python educators would be the most affected ones since they would have to update their material or see it obsoleted. Of course there's the moral issue that they would have been misleading students if they were to say that python is as simple as it can be.

Every developer whom I spoke have similar feelings about the changes, they are minimal and if they use it, very frequent boiler-plate code could be dropped. These are of course developers that have been working with python for years and have significant code bases to maintain, novices are likely to feel otherwise.

> Every developer whom I spoke have similar feelings about the changes, they are minimal and if they use it, very frequent boiler-plate code could be dropped. These are of course developers that have been working with python for years and have significant code bases to maintain, novices are likely to feel otherwise.

The solution is for them to fire up the time-machine, go back to when the code-base was just leaving the prototype stage, and assault their earlier selves with clue-by-fours while chanting something to the effect of "use a real language for real problems!".

Your conspiracy theory is mistaken; if anything instructors profit handsomely by language churn, as people are forced to come back after this or that update.

No, the problem is simple featuritis spurred by adoption. Python has reached a point where the language is basically feature-complete, but adoption keeps growing. That means that more and more programmers arrive to the ecosystem from other fields, and advocate for constructs they are familiar with or that map more closely to their problem domains. This influence is a good thing in some cases, and a bad one in others.

Operators in particular are a minefield. Python is traditionally inclined not to use special operators, which helps readability quite dramatically. There are very few exceptions (basically only @ for decorators, which is outside code flow anyway). This is why people hate new operators so much: we work with Python to stay away from unreadable, write-only code full of special characters. I understand the frustration of rote in some areas, and any professional is free to sharpen his own tools in the way he prefers -- just don't force the ecosystem at large to lower its code quality just so you can check out 10 minutes earlier from your 9-to-5 large-codebase CRUD job.

I think you're being pretty uncharitable with that interpretation of educator's incentives.

That said, I think valuing relative beginners and non-programmers into the fold is and will be a strong point of the python ecosystem. It has fewer of R's excentricities and some of Java's straight forwardness while remaining relatively terse and to the point.

I think this addition would absolutely cut down on very frustrating boilerplate. The question is the cost to the ecosystem which is going to come down to speculation in any direction.

My opinion: no one change like this hurts a language much at all but the fear comes from the death of a thousand cuts.

This feature does two things: adds slightly crypic syntax which is very obvious to career programmers but somewhat opaque and unmemorable to anyone else.

Worse, it makes bad patterns convenient. Deep None filled object hierarchies (though occasionally unavoidable) will always be problematic in some way. This only serves to make those idiomatic.

I have a different view on this. I’m not a book writer or anything of the sort.

> Syntax is never a barrier to legibility of code

I’m not sure many would admit it even if they agree, but I don’t know if “legibility” wholly describes the concern. I think there’s something more like feng shui, or aesthetics at play. A lot of Python idioms are hard to defend on legibility alone, but the result has ultimately been a language that is comfortable to read more often than not.

There are very few code bases I would want to read in bed as I fall asleep. When I realized I was generally comfortable doing that with most Python code bases, I decided this language might add a subtle quality to my life if I worked with it every day.

Programming in the real world rarely cooperates with this ideal, but I think the ideal still has some meaning. To me, what you call curmudgeon/conservative efforts of Python devs is more a humurous pseudo-enlightenment mentality, so more a humanist project.

For the record, I have no plans to rigidly defend this view and can see how it might sound ridiculous. At the end of the day, I guess I just appreciate the passion, for whatever that is worth.

> syntax is never a barrier to legibility of code

Whatever theory you have about why it shouldn't be, in practice it can be. To also seem to confuse legibility, intelligibility, and mechanical lack of ambiguity; these are all related but not identical topics.

Reading Python involves a lot less mental translation than reading Haskell. I wonder why...

Not just for beginners, the proposed syntax looks horrible in general. It's moreover already possible to put this into one line:

  libpaths = [] if libpaths is None else libpaths.split(":")

Yes, this PEP looks like it is trying to turn Python into Ruby where the ? suffix is common for a function that returns a bool. Also weird syntactic magic that you can't really figure out, you have to be told about. I wonder if the original author of the PEP is a "rubyist".

Of course in Python [] is False and in Ruby it's true...

Looks exactly like adding the ?? [1] and ?. [2] operators from C#.

[1]: https://docs.microsoft.com/en-us/dotnet/csharp/language-refe...

[2]: https://docs.microsoft.com/en-us/dotnet/csharp/language-refe...

operators from C#

Why not go the whole hog then, and do a Option[1] ;-)

[1] https://en.wikipedia.org/wiki/Option_type#F#

Option is bubkes if the language itself still has the billion-dollar mistake; that's why F# and Scala are far less practical than OCaml. Python also has it with "None" (though since it is dynamically typed, it's arguable that from the perspective of any given type, every value of every other type is a fresh instance of the billion-dollar mistake).

And sadly, removing None from Python and replacing it by Option - let alone removing Exception and letting Result (aka Either) take its rightful place - would cause huge amount of Python code to need rewriting. And it wouldn't even be worth it because of dynamic typing.

What does you gain from optionals if you can't determine types at compile time?

> What does you gain from optionals if you can't determine types at compile time?

As well as type checking (which is relevant because Python has AOT checkers even if they are optional), Options, compared to bills, give you nestability, which is a runtime advantage not dependent on static type checking.

> Yes, this PEP looks like it is trying to turn Python into Ruby where the ? suffix is common for a function that returns a bool.

No, it's nothing like the Ruby convention. It's much more like th C# operators.

do people use

    x = x or []

or is it considered non idiomatic in python ?

It’s okay-ish. You do need to be careful about what falsey values `x` might be tho, as it could introduce edge-case bugs.

Using `x = [] if x is None else x` is more precise.

I would really just write `if x is None: x = []`

It can already be written nicely as

   libpaths = paths.split(":") if paths else []

Please don't adopt unnecessary sigil junk from other languages.

I think a better example to show why you might want a ?. operator would involve nested lookups. If you have something like node.child.attribute.value, handling potential Nones at each lookup with ifs would get very verbose very quickly (and that verbosity would be bad, as it would make it much harder to make out the main point of the code).

You could surround the nested lookups in try/except and catch the AttributeError, but that could mask other errors - for instance, if you misspelled the name of one of the nested attributes (arguably, it would be better if trying to look up an attribute on None raised a different exception, rather than AttributeError).

I don't think the existing pythonic way is not too verbose:

   if node and node.child and node.child.attribute: 
      …

for nested loop you can do

   for child in node.children or []:
      for attrib in child.attributes or []:
         …

That's generally considered unpythonic though. A more pythonic way to write that is

    for child in node.children if node.children else []:
        for attrib in child.attributes if child.attributes else []:
            ...

Which, granted, is a lot more verbose and probably not any more readable but at least it's not abusing the or operator.

This is subtly incorrect. Correctly, it should be:

    libpaths = paths.split(":") if paths is not None else []

or else falsey values of paths (0, [], False) can still cause an error.

If your path variable is set to 0, [], False then it's best to fail early anyways.

If you really need it to be short you could write:

  (libpaths or '').split(":")

I would be in favor of just using an if-else though, if the None case is special, to emphasize that. If it isn't, then you shouldn't be writing this in the first place and just make sure that libpaths is always a string; otherwise you'd be expressing the same state in two ways.

    (libpaths or '').split(":")

This isn't quite the same: if libpaths is None, the above will evaluate to [''], while the original evaluated to [].

Of course, the original code was rather awkward itself, redefining the same variable to have a completely different meaning, as well as potentially involving four different representations of nothingness. I'm not sure the syntax is the biggest problem here...

`if-else` for this will get messy quickly the more layers down you want to go.

  if a is not None and a.b is not None:
      c = a.b.c
  else:
      c = None

  c = a?.b?.c

The safe navigation operator is useful enough that beginners ought to be introduced to them anyway. I will admit that adding `??` may be too much though, we could easily just use an `or` (although the PEP does have some justifications for adding it).

We have a lot of Django-based code similar to

  if getattr(obj, 'foreign_obj') is not None and getattr(obj.foreign_obj, 'field_name') is not None:

Would this turn into:

  if obj?.foreign_obj?.field_name

Does is look shorter? - YES!. Is it more readable? Arguably yes. Would I vote to see this feature in Python? HELL NO! In most of cases, we can have an in-house "maybe()" function, like

  maybe(obj, 'foreign_obj.field_name')

There is no need to update the language syntax for that.

From PEP description:

  From bisect.py:

  def insort_right(a, x, lo=0, hi=None):
        # ...
        if hi is None:
            hi = len(a)
        # ...

  After updating to use the ??= augmented assignment statement:

    def insort_right(a, x, lo=0, hi=None):
        # ...
        hi ??= len(a)
        # ...

Seriously? To me the "if hi is None" looks times more readable and easily comprehensible than "hi ??= len(a)".

Finally, The Zen of Python

  Special cases aren't special enough to break the rules.
  Although practicality beats purity.

This case does not look neither special enough nor so much practical to me. Could anyone please give a hint of where can one vote against this PEP?

Since Guido stepped down as BDFL (last month), I’m not sure there is yet much of a system in place for making these decisions. I would hope it isn’t left to absolute democracy. If this PEP is implemented, optimism will be difficult.

...why does everyone forget that `getattr()` has an optional `default` argument? It's helpful for stuff like this.

Same reason people forget min and max have key arguments: You mostly don't need them until you really do.

> Does is look shorter? - YES!. Is it more readable? Arguably yes. Would I vote to see this feature in Python? HELL NO!

No one has quite summed up my thoughts better than you.

Hell, I would love it and love to use it but I thing it's relatively out of place in Python. Python just has many more ways to solve what should be a rare discouraged situation.

I think it has a better place in languages like C#/Java that tend to have deeper, more rigid object hierarchies and more reasons for using nulls.

I’ve found that operators boil down to saving time writing (once) at the expense of later readability. They are also almost impossible to trace if operators are overloaded. And you have to hope that the overloaded operator has a meaning consistent with other uses of the operator.

Stop trying to shorten code by 2 lines. It’s not that bad to write it out. If it helps, think ahead to the 1st or 5th revision of the code you’re about to write and imagine where you’d even be able to add new code, given magical operators. In a case like this, it’s easy to imagine a single “else” case with “?” needing to change into multiple else-if cases, requiring the entire fancy operator expression to be rewritten. No real saving.

"Stop trying to shorten code by 2 lines."

Agreed. Python code tends to be concise as is. There's no need to add little tricks to cut out a line or two at the expense of readability.

Along those lines, in a recent code review a developer balked at some proposed code and boasted how he could do the same thing in fewer lines. His "improved" code had a keyword parameter where the default was set to a lambda function containing nested list comprehensions plus a zip function... I voted against his "improvement" since it was the most unpythonic thing I'd ever seen. I'll take added readability and simplicity at the expense of a few extra lines.

> Agreed. Python code tends to be concise as is. There's no need to add little tricks to cut out a line or two at the expense of readability.

I must say I agree fully, and even so, in cases where you really want to cut down verbose sections or boilerplate, there is so much you can do through defining your own classes and functions and overloading operators. Even then there is little need for new fundamental syntax.

> Stop trying to shorten code by 2 lines. It’s not that bad to write it out.

That brings me back to Blub.

"""

After a certain age, programmers rarely switch languages voluntarily. Whatever language people happen to be used to, they tend to consider just good enough.

[...] I don't want to hurt anyone's feelings, so to explain this point I'm going to use a hypothetical language called Blub. Blub falls right in the middle of the abstractness continuum. It is not the most powerful language, but it is more powerful than Cobol or machine language.

And in fact, our hypothetical Blub programmer wouldn't use either of them. Of course he wouldn't program in machine language. That's what compilers are for. And as for Cobol, he doesn't know how anyone can get anything done with it. It doesn't even have x (Blub feature of your choice).

As long as our hypothetical Blub programmer is looking down the power continuum, he knows he's looking down. Languages less powerful than Blub are obviously less powerful, because they're missing some feature he's used to. But when our hypothetical Blub programmer looks in the other direction, up the power continuum, he doesn't realize he's looking up. What he sees are merely weird languages. He probably considers them about equivalent in power to Blub, but with all this other hairy stuff thrown in as well. Blub is good enough for him, because he thinks in Blub.

- http://www.paulgraham.com/avg.html

"""

Being able to express an idea concisely (IE, "save two lines") is a big part of why we bother to innovate on programming languages. It's what it means for one language to be more powerful or expressive than another.

None-aware operators can help express an idea more concisely (IE, make python more powerful a language). I've used them to that effect in Ruby and CoffeeScript.

I agree there is potential for terseness. Every project needs to have a sense of where the team stands on terseness versus time - probably don't use `??` much if you're creating tutorials or teaching, but also avoid other pitfalls - picking unclear variable names, writing deeply nested logic (cyclomatic complexity), or clunky nested list comprehensions.

Avoiding needless terseness is the job of the programmer or the code reviewer, not the language. The job of the language is to provide power, or the ability to concisely describe intent. None-aware operators have helped me with that in other languages, and would be great to have in python.

It will be interesting to see how this PEP is handled by the community with Guido off on a permanent vacation, and the mood among Python core devs seemingly "no major changes to syntax for the next little while" .

Personally, some of the examples using the new operators are in fact easier to read, but others border on illegibility, and I don't think the tradeoff is wise.

Then again, I felt the same way about Python's decorator syntax (ie. @) when it was proposed, and I still find it jarring.

> It will be interesting to see how this PEP is handled by the community

It was created three years ago, and it's still just a draft. Hopefully it won't go anywhere, with or without Guido around.

EDIT: Raymond Hettinger is on the ball as always: "This PEP also shares some traits with PEP 572 in that it solves a somewhat minor problem with new syntax and grammar changes that affect the look and feel of the language in a way that at least some of us (me for example) find to be repulsive. This PEP is one step further away from Python reading like executable pseudo-code. That trait is currently a major draw to the language and I don't think it should get tossed away just to mitigate a minor irritant"

And yet, PEP 572 was recently accepted...

Another +1 for rejection. Python is popular because it is easy to write idiomatic code that is easy to understand by humans, which I think is one of the features that separates bad from good code. This proposal would only add non-intuitive syntax (e.g. extensive operator chaining) that will make code more complicated to read, even if you are not a beginner, only to save 1-2 lines of code.

Null-awareness is something that, once you start relying on it, you really start to miss not having it. I think having idiomatic and easy ways to shuffle nulls out of the system is practically required for any dynamically-typed language.

You need terse ways to do it because once your codebase reaches that point where the behavior is sketched out and now you need more correctness, you're going to be using it everywhere, at least until you've discovered all the main sources of ambiguity so you can gateway those separately.

I'm a Rubyist, being dragged kicking and screaming into NodeJs world, so I don't have a dog in this fight, but this kind of thing and the reaction I'm seeing to it in this article makes me happy with my language choice.

Python seems to be seeking out a middle ground between a structured programming language and the web world. Which is fine and all, but it seems to be taking the worst part of both worlds. The verbosity and finickiness of syntax of structured languages and the performance of dynamic ones.

Here's a write up of the chilly reception this recently received on the python-ideas mailing list: https://lwn.net/Articles/760993/

The whole endeavor seems a bit quixotic to me. This kind of change would no doubt cause such a stir that there’d be a fork before long.

No, it's not that drastic, especially considering prior art in other languages

Not supporting the change, but let's not be dramatic

I'm fairly well attuned to the sensibilities of Pythonistas and between this and the recent stepping down of the BDFL, it would definitely invite a fork, fuelled in no small part by the vacuum that exists in spearheading the Python project.

Please don’t do this, Python!

I agree that the common patterns used for None values can be verbose; but, they’re readable and clear. PEP 505 syntax looks like Martian.

IMHO, core developers should spend the next two years focusing on performance.

If they can just make cPython 50% faster (avg) and use 25% less memory (avg), Python can capture so much more marketshare; this increases the value of the ecosystem and economies of scale therein.

The language itself needs no changes.

I am quite concerned about the kinds of PEPs that come up lately (edit: I now noticed it's three years old). Python is drifting farther and farther away from its nature and strength. The amount of complexity is constantly going up and I don't think it's a good thing. Please let this language be what is is.

A small amount of boilerplate in cases like this is fine and motivates you to restructure the code rather than hiding the mess in this way.

Why write

  data = data if data is not None else []

  data = data ?? []

if you could simply use

  data = data or []

which already exists and for all intents and purposes should achieve the same thing?

Need to make sure you can always append to something so you can drop some conditions?

  lst = get_log_list()
  lst?.append('A log message')

would have been

  lst = get_log_list() or []
  lst.append('A log message')

right now. In terms of reconsidering the structure of your code, I wonder if a log list not existing or being empty is a meaningful distinction. Perhaps it could return an empty list to begin with.

Even

  libpaths = libpaths?.split(":") ?? []

is fairly easy to deal with already if you absolutely want to it be short:

  libpaths = (libpaths or '').split(":")

Then again, if it is going to be treated as a string, why would it ever be None rather than an empty string? Is that distinction important? Being able to shove it under the rug by sprinkling some question marks over it, allows you to never stop and wonder about that, when in fact, you should.

> data or [] Unfortunately this kind of conditional assignment does not work if `data` is a numpy array. This arguably more numpy's fault than python's, though it is a mildly infuriating when it happens.

null in languages (or None in Python) is one of the few criticisms I hear from pure-functional minded evangelists that I have come around to. Projects started by novices tend to start with no null checks at all with lines like:

    some.object.other.method();

Then at some point after months/years of null pointer exceptions you get a code base littered with:

    if (some && some.object && some.object.other && some.object.other.method) {
        some.object.other.method();
    }

Or variants like assertions / try-catch blocks / etc.

This is even more cumbersome/ugly when needing to assign the result to a constant:

   const foo = some && some.method && some.method();

I would very much like to do away with null. There is some half-formed idea in my mind that this problem is related to how we allocate memory and the concept of the heap. I don't think syntax can solve this issue.

My main issue with syntax like:

   const foo = some?.object?.other?.method();

Is that it encourages sloppy error handling by making the conditional branches in the code implicit. I don't condone the (a && a.b && a.b.c) pattern so I don't really want a shorthand for it.

I'm disappointed to see that everyone hates this PEP. I think null-conditional operators are pretty self explanatory even to beginners and they have the potential to remove a lot of boilerplate which actually could improve code readability in many cases. I wonder if this massively negative response isn't just a reaction to everyone having been burned by the PEP 572 debacle.

You have been deeply disconnected from beginners if you believe this. I absolutely assure you, someone new to programming will have no idea what in the name of god this is, and will struggle with it.

How can you say that with such certainty? Do you have an example of a similar construct where you've experienced beginners being confused by it, or are you just dismissing my opinion based on your opinion?

sigh no, what Python should actually have is a way to compose fallible computations such that if any one should fail, the entire composition short-circuits and returns the error; while not needing each one to check its input for such a case.

It could even be generalized so that it would work properly even for things other than the ad hoc union-type formed by the billion-dollar mistake...

Why must so many languages incur upon themselves so much trouble by lacking such a simple thing as the monad abstraction?

Monad abstraction is a mere crutch to fulfill compiler greed. It is not needed in dynamically typed langs.

> compiler greed

What good compilers require, they repay hundredsfold in exhaustively eliminating entire categories of mistakes from every program that compiles without error.

> they repay hundredsfold

Most python apps is simple data proxies. Glue dicts from multiple sources into more complex dicts and pass further. If you will try to do proper modeling here you never accomplish a task.

On the contrary. A simple data proxy involves not much computation, but mostly reshaping; what kind of thing you have becomes more important than what thing you have, and eliminating confusion in the former realm is precisely what static-typed code with proper modelling, excels at. And any competent static-typed language will have enough macro-like capabilities to generate the model code (along with as much of the boilerplate as possible) from a JSON Schema or similar.

(Confusion in the latter realm, on the other hand, is one of the most common sources of remaining bugs once typing errors and other footguns are eliminated-with-prejudice.)

If you will do proper modeling in each intermidiate processing unit you never accomplish a task. It is software design 101. We check invariants on the endpoints not in the middle.

...isn't that called a `try` block?

Yes, except for things that helpfully return "None" instead of raising an exception, on account of "not finding something is not exceptional behavior" or something like that.

And since "raise" is a statement, you have to write your own "raise_if_none" function and sprinkle it everywhere...

Like many UI features, it is hard to get a feel for this without trying it.

Swift uses this syntax and I got quite used to reading and writing it.

I’m all in favor of it.

I've always thought Python was a little too readable and obvious. I'm glad people are hard at work trying to fix that.

This sort of functionality is at odds with the Zen of Python. It's not intuitive at all.

Very strange PEP. Most `??` cases can be covered with simple `or`.

Not necessarily...

  x = 0
  y = (x or 42)

isn't the same as

  x = None
  y = (x or 42)

Yes, it isn't the same in a generic sense. But can be applied in many practical cases.

On small trick that Pythonistas hate:

    x = x if None else x

Saves a ton of CPU time and is beautiful.

Huh, what's that do? It seems to assign x = x which is reminiscent of my pet "ugly" optimization: adding x = x to the top of a nested scope. It binds the x from a global scope to x in a local one. Local variable lookup can be substantialy faster given a hot enough loop.

Reply on news.ycombinator.com