What's New in Python 3.12
docs.python.org
docs.python.org
I write 'may' because the difference between None and an empty dict may be very subtle and rarely does the API specify with enough precision what are supposed to be the semantics of each case.
It does though, you can chain as many of these as you want:
some_dict.get("level_1_key", {}).get("level_2_key", {}).get("level_3_key", {})...
>> but you still need to check for None with this code
> Not sure what you mean here. You only have to check for None if you use `.get("key")` and don't provide a fallback value.
GP was talking about `{"foo": None}` and trying to drill deeper, which I misunderstood. Still, a simple try/except allows you to short-circuit the deeply nested access.
>>> foo = {"foo": None}
>>> print(foo.get('foo', {}).get('bar'))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'NoneType' object has no attribute 'get'
>>> print((foo.get('foo') or {}).get('bar'))
None> Blowing up on `None` is an easy short-circuit.
The thing being discussed is attempting to access deeply nested values, so short-circuiting here is a win-win. I.e., you wouldn't want to unnecessarily traverse tons of empty dictionaries using the `.get() or {}` trick.
This has the side effect of replacing falsy values like False or 0 with an empty dictionary instead of giving you the actual value. For the exact intended behaviour, you do need to explicitly check for None instead of using a short circuit trick with or.
I would say the only reasonable choice is Optional[dict], in which case it should be sufficient, but Python being Python, it could be anything. And in these cases you need to handle all cases way more carefully.
https://glom.readthedocs.io/en/latest/api.html#defaults-with...
e.g.
target = [{"a": {"b": "c"}}, {"a": {"c": "e"}}, {"a": {}}]
results = glom(target, [Coalesce("a.b", "a.c", default="")]) # -> ["c", "e", ""] def safe_navigation(obj, path: List[str], default=None):
try:
for key in path:
obj = obj[key]
return obj
except KeyError:
return default
But really, how do you know if your path is missing or the object at the end of the path is None, without using exceptions for communicating this?
def safe_navigation(obj: Any, *path: str, default: Any = None, strict: bool = False) -> Any:
sentinel = object() if strict else None
for key in path:
obj = obj.get(key, sentinel)
if obj is sentinel:
return default
return obj
But I really don't like having this much `Any`; it's generally a sign of poor design.
This isn't like C++ with its intrusive `=`.
$ python3
Python 3.11.6 (main, Oct 2 2023, 20:46:14) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from typing import List
>>> def safe_navigation(obj, path: List[str], default=None):
... try:
... for key in path:
... obj = obj[key]
... return obj
... except KeyError:
... return default
...
>>> obj = {1: {1: {1: 2}}}
>>> safe_navigation(obj, [1,1,1])
2
>>> obj
{1: {1: {1: 2}}}https://github.com/jmespath/jmespath.py
print(jmespath.search(
expression="some.deep.nested.value",
data={"some": {"deep": {"nested": {"value": 2}}}},
))
prints 2
print(jmespath.search(
expression="some.deep.nested.value2",
data={"some": {"deep": {"nested": {"value": 2}}}},
))
prints NoneWhat's actually a bit odd about Python missing that operator is that its `or` operator acts as a rough equivalent of C#'s null-coalescing (`??`) operator. To me the safe navigation operator goes hand-in-hand with that.
Python `or` is C# `||` plus type coercion. It is not even roughly `??` because important non-None Python values are falsey (0, False, empty collections).
> python3 -c 'print(None or "hello")'
hello
In C# even if `||` automatically coerced the types, the output will always be a bool.
I saw some suggestions for overridable behaviour in the data model for None-like objects, but didn't scrutinise them closely. Either way, it seems controversial. On the one hand, the ability to override operators is really key to Python's design, but on the other it could lead to rather surprising behaviour from overeager overriding. (But to be fair, this is true of other, more common operators too.)
So, for now we're stuck with `if _ is not None else`, I suppose!
That'll be really helpful for nested method calls which proxy a lot of their arguments along. I can also see that helping out when creating stub libraries.
(I don’t know about others, but it was obvious to me from the start that the way the *args and **kwargs type annotations refer to the sequence values rather than the args/kwargs bindings was always going to cause confusion, and was very likely to cause problems down the line when they wanted more precise typing; and that day has arrived for kwargs, no idea if they have any plans for similar on args, which is sometimes useful. They should just have required `*args: List[T], **kwargs: Dict[str, T]` from the start.)
* Some types are better than no types. I love Python types, and I consider them required. Even if they're not type-checked they're better than no types. If they're type-checked it's even better. If things are typed properly (no any etc) and type-checked that's even better. And so on...
* Having said this, Python's type system as checked by mypy feels like a toy type system. It's very easy to fool it, and you need to be careful so that type-checking actually fails badly formed programs.
* The biggest issue I face are exceptions. Community discussed this many times [1] [2] and the overall consensus is to not check exceptions. I personally disagree as if you have a Python program that's meticulously typed and type-checked exceptions still cause bad states and since Python code uses exceptions liberally, it's pretty easy to accidentally go to a bad state. E.g. in the linked github issue JukkaL (developer) claims checking things like "KeyError" will create too many false positives, I strongly disagree. If a function can realistically raise a "KeyError" the program should be properly written to accept this at some level otherwise something that returns type T but 0.01% of the time raises "KeyError" should actually be typed "Raises[T, KeyError]".
* PEP 695 will help because typing things particularly is very helpful. Often you want to pass bunch of Ts around but since this is impractical some devs resort to passing "dict[str, Any]"s around and thus things type-check but you still get "KeyError" left and right. It's better to have "SomeStructure[T]" types with "T" as your custom data type (whether dataclass, or pydantic, or traditional class) so that type system has more opportunities to reject bad programs.
* Overall, I'm personally very optimistic about the future of types in Python!
[1] https://github.com/python/mypy/issues/1773 [2] https://discuss.python.org/t/extend-type-hints-to-cover-exce...
By definition, it is, because there is defined behavior for unhandled exceptions.
If you want to—and this is a valid preference—wrap all exceptions thrown at a lower level and present an API where they are part of the return type that must be addressed by code to pass typechecking, then you do that and return actual values of an appropriate type (you can still use exception types, if you wish) instead of raising exceptions.
Exceptions which you force client code to handle to typecheck are not exceptions, they are return values and should be converted to explicitly that, rather than adding checked exceptions.
If you’re saying that I must handle an exception in my code rather than crash because my “fail early, fail often” design choices offend your religious preferences… then keep it to yourself and out of PEPs.
def wrapper():
try:
return handler()
except Exception:
logger.exception("unknown exception")
webserver.status = 500
def handler() -> T: ...
def wrapper() -> NeverRaises[T]: ...
webserver_add('GET', '/handler', wrapper)
def webserver_add(method: str, path: str, handle_func: Callable[[], NeverRaises[T]]) -> None: ...
Of course, here we need to specially handle `KeyboardInterrupt`, signals and exceptions returned by `logger` etc. I would personally recommend ignoring `KeyboardInterrupt`, and signals and type annotating `logger.exception` as `NeverRaises`. 95% is still better than 0%.
I disagree, in part because “handles all exceptions” is a lie; exceptions can occur at any point, including in exception-hnadling code.
It is not possible to have a Python function which provides the guarantee you want, so it makes no sense to have a Python type system which expressed it: it will either be never used or a lie.
If you think you need this guarantee, you need to step back and ask what the functional requirement actually is and find a different way of meeting it.
I started experimenting with Kotlin just when Python was starting to get type hints and after my experience with Kotlin I am totally sold on the benefits of typing. So to me it's great to see these recent developments. It just feels weird to me to have a language that is fundamentally dynamically typed (and committed to that paradigm) invest so much effort in developing this elaborate but optional typing system.
Is date a string or an object? It gets tiresome.
F# is sort of the opposite of Python. It's strongly, statically and soundly typed, based on Hindley-Milner. It has very good type inference, meaning that explicit type annotations are usually not required. But it's still type checked, and compilation will fail if there are type errors. So F# gives you type soundness without requiring type annotations (in most cases); Python now supports type annotations but doesn't (yet) give type soundness guarantees.
Of course, F# doesn't have the Python ecosystem, and that's a major issue. Whilst it sits on the .Net platform, it's very much Sunday League to C#'s premier division. Whilst most all .Net libs can be used in F# thanks the .Net CLR, very little is idiomatic: F# is a functional language, C# is object-oriented at heart.
I think my fantasy platform would be some child of F#-the-language and Python-the-ecosystem. Maybe Python's incremental support for typing will get me there someday.
You can't do this in Python your bucket will always have hole you can't plug. Arbitrary exceptions can appear anywhere in your code thanks to signal handlers.
KeyboardInterrupt is the one you probably know without knowing.
Exceptions can even come from higher in the stack down to you with the .throw method on generators.
Was this such a big problem?
In my experience, the GIL, faster start-up times are so much higher on the totem pole, why this now?
Hell, personal testimony, I don't give a rat's ass about the GIL or start-up times, but f-string limitations are a daily pain in the ass. So I'm absolutely grateful for whoever worked on this and looking forward to it.
People are so out of touch with how OSS, and the volunteer maintainers that operate it, work. It's a shame. The sense of entitlement in a subset of the userbase ("Why aren't they fixing MY issues RIGHT NOW!") is mindblowing.
Politics, other forces, compellation will emerge as a stronghold barrier to your meritocratical hopes and dreams.
Being able to just use any quotes in f-strings is just godsend QOL change and probably would be the biggest reason I upgrade to 3.12 (for my pet project).
For me that formalization is nice because they've brought f-strings into the PEG parser which can point at the precise location of a syntax error. I'm wary of the implications of arbitrarily nested f-strings for legibility... but oh well.
This change isn't so much done for the sake of being able to next quotes in f-strings, it's done because there was a dedicated extra parser for the python syntax inside f-strings, but with the new parser this can be parsed without this special case extra parser.
It's all to manage technical debt.
I suspect, a lot of people that spend time in jupyter notebooks are in the same boat. You could argue I should be setting a variable but these are experimental scripts and it's annoying. I welcome it!
Are you aware that more than one change can be worked on at a time?
>f"""{f'''{f'{f"{1+1}"}'}'''}"""
It has become an unpleasant chore to keep up with PEP squabbles and accepted changes.
On the utility of features, the arbitraty nesting of f-strings stands out. Why in tarnation would anyone need such nesting? Isn't it simpler and readable to keep non-trivial expressions out of the f-string itself?
But then you have to define what non-trivial expressions are, and more importantly, people using the language have to learn that definition too. By having f-strings accept all valid expressions, the language actually becomes simpler.
The asyncio improvements in Python 3.12 especially (plus perf generally) have been instrumental in enabling real world use of this. With Python 3.12 asyncio, uvloop, and FastAPI it works remarkably well[1]. As the demo video shows not only does it not delay responsiveness, it has granularity down to inches.
[0] - https://heywillow.io/
```
...
type IntOrStrSequence[T: (int, str)] = Sequence[T] # TypeVar with constraints
```
```
type IntOrStrSequence[T: int | str] = Sequence[T]
```This has been the new strategy to avoid another python2-python3 debacle. As long as you're aware it's not semver and minor versions can have breaking changes it's no biggie.
How is this not a major version change if a module is removed from the standard library? Other breaking changes too. This is confusing to me.
Something even crazier would be to have an equivalent of `console.log` in python. It would be an amazing feature but I think I'm the only one wanting it. I know I can use `print` or different logger. But it's a lot more complicated to use and the output is a lot less navigable than in javascript. PHP also has `var_dump`. But we don't have any equivalent in python.
It's an absolutely terrible idea and I'm thankful that there's so little chance it'll ever happen. I don't want random objects to become mappings, nor do I want mapping entries and their attributes to conflict. Javascript is a bad language and this is one of its worst features.
> Something even crazier would be to have an equivalent of `console.log` in python. It would be an amazing feature but I think I'm the only one wanting it. I know I can use `print` or different logger. But it's a lot more complicated to use and the output is a lot less navigable than in javascript.
You... can just call `logging.whatever()`, after having called `logging.basicConfig()` to set up your basic config?
I fail to see how that would change anything to navigability. `console.log` is not inherently navigable, it's the browser's console UI which provides some navigability.
"There should be one-- and preferably only one --obvious way to do it." (zen of python)
I do agree that python logging is a weak point. It is too easy to do it wrong -- particularly when you are a few modules deep.
> "There should be one-- and preferably only one --obvious way to do it." (zen of python)
Look at all the other possibilities to do `f"hello {name}`. That are more than one obvious way to do it.
* For new languages: It's not generic enough since there is no equivalent of `x[t]` if t is of a non-string type. E.g. there is no way to express `x[(1,2,3)]` or `x[3]` or `x[frozenset({1, 2, "foo"})]` this way.
* For existing languages like Python: this would be a breaking change since things that can do `x.y` and `x[t]` are structurally different in Python so they're typed differently. One are called "mappings" and the other are "objects", they're completely different things. Hence, you'll get cases where `x["foo"] == 5` but `x.foo == 4` so this will for sure break some programs. Too much pain for no gain.
Don't you think it should not be possible to have such a thing ? To me it's so prone to error and there is absolutely too much gain to fix this.
By the way, there is a vulnerability (Prototype Pollution) that is only possible due to this behaviour in JS: https://portswigger.net/web-security/prototype-pollution
No, having used lots of OO languages before JS and its “objects are dictionaries are arrays and member access and string indexing and integer indexing are all equivalent and can be used interchangeably, except you can't use member access where the key isn't a valid identifier” approach, which I find more confusing and error prone (though I’ve since had to use JS enough to be proficient in that, too.)
Indexing an object as an indexable collection and accessing a member (field, property, or method) of the object are fundamentally different things, and having a collection item with a particular string index isn’t the same thing as an object member with a similar identifier name.
This use of objects as also quasi-associative-arrays is so broken that JS’s actual associative-array type (Map) can’t use indexing notation because of it, and has to use .get() and .set() instead, unlike the associative array types of most other dynamic languages (and several statically-typed OO languages).
The JS way is less bad as a type-specific behavior (e.g., Ruby ActiveSupport’s HashWithIndifferentAccess), though.
No. They're different notations; one means `x.__getitem__('foo')` and the other means `x.__getattribute__('foo')`. Why should they be the same? It isn't confusing that `5-4 == 1` but `5+4 == 9`, after all.
If we assume that the dict class was enhanced with your proposed equivalence, would you want `d['items']` to be the function `d.items`? Would that not make 'items' a forbidden key?
x = {"users": [12,21,54], "items": [1,2,3]} # group owned items
>>> x["items"]
[1,2,3]
>>> x.items
<built-in method items of dict object at 0x00>Most languages that have both objects and associative arrays (so not Lua or JavaScript) make this distinction.
It's not particularly error prone. You're simply refusing ("don't care why") to learn how to use the tool you're given.
user instance from a django model => user.email
It is error prone. You're simply refusing to see an aberration.
That's how the language works, but it doesn't mean it's intuitive and easy to understand especially for a language known for being easy to use and understand.
When all you have is a hash table, or when all you have is an object, you get to refer to keys and properties uniformly - because they are the same thing. When you have both, you refer to them differently - because they're different. That's it. There are some languages where objects and hash tables use the same syntax for access even though they're different things, but... you probably never used any of them, and certainly none of them is in the Top 20 on TIOBE.
I'll kick the venerable HN guidelines aside for a second and mention this: you're being heavily downvoted, all your comments in this thread are in various shades of gray, and many people offer many different arguments as to why you're wrong. Yet, you're undaunted and continue posting - I don't want to break the guidelines that much, but honestly, it reads like trolling. You don't engage with the arguments, you're just repeating the same thesis over and over again, without citing evidence. Why?
There's not one perfect format to rule them all but the "=" "self-documenting" debug format such as `f"{some_obj=}"` probably gives you a good starting point for what you are looking for most of the time. Sometimes you still want the `str()` or `repr()` representation specifically and would want something like `f"{some_obj=!s}"` or `f"{some_obj=!r}"` respectively. In some cases objects you want pretty-printed to a console might have custom `__format__()` representations as well and it might be something like `f"{some_obj=:some-custom-format}"`.
It's obviously all still differently useful than JS having a full object browser embedded in most common consoles today, but there is interesting power to explore in f-string formats if you need quick and dirty logs of object states.
https://docs.python.org/3/whatsnew/3.8.html#bpo-36817-whatsn...
Maybe pprint is for you:
from pprint import pprint
# usage, but can do more as well and has more config stuff if needed
pprint({'a':'b'})
https://docs.python.org/3/library/pprint.htmlAlthough it'd be nice if it worked with slotted classes instead of just blowing up.
Dataclasses have asdict which work on both, but it's annoying there's nothing for regular classes.
1. Write a class with custom __getitem__ and __getattr__ methods
2. Use a template system like Django or Jinja that implements this in certain situations
3. Use a library like https://pypi.org/project/python-box/