PEP 505 – None-aware operators (2015) | Hacker News Reader

PEP 505 – None-aware operators (2015)

peps.python.org

75 pointsedward58 comments

I think this would actually be a good addition.

I'd like to use 'or' more often, but it always makes me stop to think "Could this get passed a falsey value?" I've seen a lot of Python bugs because people used "x or 5" when they needed the much uglier "5 if x is None else x".

Making anything other than false and none falsy was a mistake; it’s much easier in Lua, for example, there you just say “x or 5” because only nil and false are falsy.

> Making anything other than false and none falsy was a mistake

Yes. That's how you end up with messes like this one: https://lwn.net/Articles/590299/

Whattttttttt?!? TIL!

It's no longer the case, was changed in Python 3.6

Wow! That's UTC as well, so where I am it'd be like having an exception for exactly 4pm, it wouldn't even occur to me that it was a zero!

Fun fact:

In most people's Haskell code, you can only use actual Booleans when you need true or false. (As you might expect from a well-disciplined, strongly typed language.)

But it's actually really easy to add your own version of eg `if` etc that accept nearly anything, and automatically convert it to True or False. You can do this as a normal library for use in anyone's code.

... Until you encounter a case where false is a valid value, in which case it’s ~= nil all over again. (Tip: on LuaJIT, x == nil is also true for FFI pointers that happen to be null, which can sometimes force an FFI temporary to be allocated; using type(x) == 'nil' instead may help.)

If your language supports a genuine nil and not just a distinguished singleton token (AFAIK only Lua does that consistently, though perhaps “undefined” in non-strict-mode JavaScript also counts), I’m actually leaning towards having operators for dealing with it specifically. Icon does something like that, arguably, although its behaviour (for a “failing” expression that yields no values) is more like a propagating NaN / relational NULL / poison value than a nil.

Nix (though it has a faux null like everybody else, as well as a poisoning bottom typical of lazy languages) has a slick partial solution: the field access operator is optionally ternary, expr.name [or expr], so you can write things like

  systems.flakeExposed or systems.supported.hydra

when you want to fall back from one to the other, even though in the fallback case the value of systems.flakeExposed by itself is not nil, it’s an error. (Using or outside of field access is a syntax error, the Boolean operator is spelled ||.)

Good point, I still have to check nils on some stuff in Lua, but way way less than I do in my Python day job!

I liked Kotlin's null-handling when I worked in it because there were some tools there to handle nulls in a reasonable manner. I think the `something?.prop` would do null checking on `something` before trying to access `prop` and then you could use `?:` (Elvis operator, haha) to check a null and do something if it was null, like `something ?: executed_if_null` and combining those would effectively deal with nulls in a simple and readable way.

Lua? You’re kidding right? Lua is so full of gotchas that using it is an exercise in frustration. Tack on LuaJIT, and you’ve got yourself a twelve gauge footgun.

Lua is JavaScript-level stupid, minus the useful ecosystem.

I like Lua but it’s not my religion or anything. Your use of a throwaway for this signals I probably shouldn’t even engage with you, but why do you hate it so much? I’ve found it to be concise, yet powerful.

Probably forced to use it for some job plus learning aversion?

>Lua? You’re kidding right? Lua is so full of gotchas that using it is an exercise in frustration.

Which is irrelevant, as the author mentioned a specific feature of Lua that's designed better than Python's same feature, didn't say everything is better in Lua.

(Not to mention it's not really true anyway, Lua is fine).

0 and 1 are iconic for true and false.

Because true and false used to just be 1 and 0, not for any good or thoughtful reasons (much of tech is the way it is because of constraints we no longer have). Today, new languages should avoid pitfalls like truthiness and null values.

They are both good and thoughtful, fundamental to how computers work.

I guess it wouldn’t matter much without null values, or if they made null values neither true nor false then it might fix it. If not for null then truthiness wouldn’t seem as dangerous to me, I think.

As far as zero and one; if you compare equality on a machine you’re generally doing an xor type operation between the two values and if they’re equal you get zero, so that’s flipped. If you’re comparing two numbers it might even result in a ternary value of LT, EQ, and GT.

I think true and false are high level programming language or mathematical concepts that don’t map directly to the low level testing results. So, I don’t really consider the traditional “true is 1, false is 0” to be anything more than convenient abstractions in programming based on things that some folks did for convenience at the beginning of software development.

0 perhaps.

1 doesn't mean much regarding true, if every other non-zero integer is also true.

Binary numbers and electronics would like a word with you.

I don't know, a boolean is usually a byte (or a words?), so should true really be 0b00000001? Why not 0b11111111? I think it is in some Forth implementations.

Talking about integer values treated as false/true in a language without first class booleans. Not about the use of 0 and 1 in binary.

And my point is that since 0 alone is treated as false among integers in those cases, it can be considered as iconic or "standard". But since any other integer is true, 1 is not much of the "canonical" true, but just one of billions of others.

In binary it's 0 and 1 alone, so the comment doesn't apply there.

Also, if you compare the equality two binary strings using an xor, you get zero for equal. Zero and one have no specific meaning in binary logic other than being the symbols used for binary strings; they’re just abstractions that were used by the first programmers to indicate true and false before languages has dedicated atoms for those concepts.

Python has first class bools.

If you don’t want to test an integer, if it doesn’t meet your needs, don’t use it. There are an unlimited number of other choices for comparisons.

The problem is that it also has second-class bools (other stuff service as booleans).

First class bools alone being logic-checked, is a better design.

In C and other such languages, the comparison operators return 0 or 1. So that gives 1 special status.

I'm happy to assert that making anything other than false be false is a mistake.

The problem is sum types. Specifically, languages which "don't have" sum types often secretly do have sum types, absolutely everywhere, but no language facilities to deal with them because they "don't have" sum types. That's what the billion dollar mistake is, it's sum types (thing | NULL) with no language facilities. In Python that's (thing | None) but the same penalty accrues.

Now, PEP 505 introduces a bunch of actual language facilities. They might be a good idea for Python, other languages successfully did this, but I think as a lesson going forward what we should take away is that we do want Sum types and we should stop pretending we (sometimes?) don't, which means new languages need to be designed for them.

It seems like most more modern languages actually have facilities for expressing proper sum types (even if it is in some languages more verbose than in others).

Certainly Kotlin, Swift, Rust, etc. do have them. Even Java, I believe, is planning to add them, though I suspect that many people won't necessarily understand why they're useful even when they land.

It's probably helped by functional programming becoming more mainstream and "obvious" sum types needing to be implemented (Optional and Result, mostly).

I agree. It's absolutely silly that there's nothing to make statements like

    something['nested']['blah']['bloop']
    some.namespace.with['stuff']

safe with None. Your options are using jmespath, your own similar function, or wrap it in a try block. Otherwise the whole syntax is essentially unusable. And because of the quirks of how __getitem__ is evaluated you can't even hack an infinite defaultdict to do the right thing because you can't detect the "last" access to return None so it's only good for when you want to assign to it.

but python is simple /s

    something.get('nested', {}).get('blah', {}).get('bloop')

works just fine but is mighty ugly.

...which is a hint to use something else in this case. dataclass, typed dictionary, deserialize properly, whatever.

Ultimately there’s some things that are best done with dictionaries, so it would be nice to have a simple tool to do this. Something like ‘x.?y.?z[“something”.?]’ would be nice, where the ? might pass any None through to the end without causing attribute and key errors.

That's like Lisp versus Scheme:

Lisp: (or x 5)

Scheme: (if (null? x) 5 x)

For comparison, here are the Javascript RFCs for the equivalent operators:

https://github.com/TC39/proposal-optional-chaining

https://github.com/tc39/proposal-nullish-coalescing

IMO optional chaining is a lifesaver when dealing with deeply nested JSON, and it's become indispensable in our Typescript code. But we also deal with JSON from various third-party endpoints in Python, and far too often we resort to an inefficient deep_get utility to make our code sensical.

> if json?.get("foo")?.get("bar")?[0]?.get("baz")

may seem messy but would be infinitely better than the fragile (note the nested list-of-a-single-dict, and how bugs can appear if it's not there):

> if json.get("foo", {}).get("bar", [{}])[0].get("baz")

or the inefficient

> if deep_get(json, "foo.bar.0.baz")

It's not nearly as good as the Typescript json.foo?.bar?.[0]?.baz - but it's good enough!

I love the walrus operator - and use it quite a bit, but, replacing

       if hi is None:
          hi = len(a)

With:

       hi ??= len(a)

Just sounds gratuitous. I'm happy this was deferred.

It's actually simpler then that

    hi = len(a) if hi is None else hi

Your `if-else` is more complex than the original that just uses `if`.

The line itself is longer, but the idiom in the original uses mutation, which requires reading more adjacent lines to understand and make sure you haven't missed anything.

(In the concrete example, both versions use mutation.)

I'm not sure I understand. Could you give a concrete example where only one version uses mutation?

Any example that does what the proposed ??= operator does, which is inherently conditional mutation, will use mutation.

You can get examples equivalent to the proposed None-coalescing ?? operator does that don't use mutation, though.

There's a major difference though: `value if predicate else alternative` is itself a value, something you can assign to a variable or pass to a function as a parameter.

I like this but I cheated on python with typescript and rust, and I don't think I'm coming back, so I don't think my vote is appropriate.

Recently did the same. TypeScript's type system is amazing. Shame I hate JavaScript so much.

Hack's typesystems is similar-ish.

They are both basically what you get when you retrofit something a bit like Haskell's typesystem onto dynamic languages.

Trade offs. Not > / <..

Rust is great. But you just listed 3 programming languages that each have their own purpose and place!

It’s hard to imagine what you couldn’t build with those three. (Certain vendor locked systems notwithstanding).

The examples in that PEP range from pointless (`if hi is None: hi = len(a)` => `hi ??= len(a)`) to cryptic (`libpaths = libpaths?.split(":") ?? []`).

Remember the Zen of Python: “Readability counts”.

That latter "cryptic" example is exactly what Typescript looks like in some cases. But there, it works because the type system is guaranteeing you the correct return type, and can warn you when it's unnecessary. To add it to Python would be a huge mistake, because it would encourage throwing in ?'s everywhere "just in case", making it even harder to understand what's being passed around.

The `?` operators also fit better within TypeScript's overall syntax, whereas Python traditionally prefers English words over punctuation characters.

In a language which considers `and` and `or` more readable than `&&` and `||`, surely `is not None` is more readable than `??`.

True, but it is tricky because `is not None` doesn't work here. For example, `my_var ?? 0` would become `my_var if not None 0` which doesn't really make sense and also collides with the pre-existing ternary syntax (`my_var if not None else 0`).

I'm glad this was deferred because those replaced examples from the stdlib are difficult to read. I've used `or` coalescing plenty of times and definitely prefer it to a new operator.

That said, I really loved using null coalescing/propagation in C#, so I have to assume my bias is coming from wanting python to be more readable.

Or-coalescing can work fine for things that can either be None or truthy, but Python (unlike, say, Ruby) has fairly expansive falsey values, which create lots of places where None coalescing would work but or-coalescing doesn't.

Didn't realize this was a PEP but I'm glad it was deferred. I have a bit of PTSD from my foray into Haskell and trying to understand the internals of libraries like attoparsec. The proposed `??` operator makes me think of things like the `<*>` operator for applicatives. The average person's intuitions don't serve them in guessing at the meaning of either. And extra syntaxey characters really disrupt the flow of things. But I guess that's just my opinion.

In any case, this feels like something that belongs in Ruby more than Python. The designers of Python made the deliberate choice of sacrificing conciseness for readability in certain cases. If we give that up, I think Python will end up feeling like something that doesn't know what it wants to be.

Keep in mind that in Haskell all operators can be defined in user-level code.

They can also be shadowed by your own code.

If you don't like `<>`, you can also easily make your own function and use that instead:

    my_function a b = a <*> b

I agree that (infix) operator syntax can make some code hard to read, but it can also make some code easier to read. I do feel strongly that allowing people to make up new operators is preferable to undisciplined overloading of existing operators:

Eg in Haskell when you see `<>` at least you know that something unusual might be going on. In C++, when you see `>>` (or even `+`) you have no clue whether it's just plain old bit-shifting, or whether someone overloaded it to do IO or perhaps launch the missiles.

?? looks like decent shorthand for:

    bar if foo is None else foo

or perhaps more clearly:

    foo if foo is not None else bar

Seems like it would save some typing.

Please also let me use it as a type hint!

`str?` would be much nicer than `Optional[str]` or (new in 3.10) `str | None`

As of 3.10 you can use "str | None" instead of "Optional[str]".

They rejected the exception aware operator, it would’ve been the most useful application.

Reply on news.ycombinator.com