Python Pattern Matching
github.com
github.com
I also think those who complain about the decline of Python's beginner-friendliness need to take a step back and understand that this is one of the most popular languages in use today. Reducing code maintenance issues (by eliminating boilerplate) will save many programmers time, and when you multiply average savings per person by the total number of programmers using Python day-to-day, I think adding certain features is a net positive. The time taken to learn this feature is small, and it's a one-time investment for each programmer. I don't think this is even a difficult feature to learn at a coarse level, since parameter unpacking/binding is already part of Python (think `for k, (x, y) in enumerate(list(zip(xs, ys)))...`.
All that said, I'm a bit concerned about how this is implemented. Many have raised the issues of unintuitive binding of non-dotted names, lack of local scope, and overloading of the `|` operator. I can't understand why these problems were not addressed during the PEP process. Does anyone have any insight?
IOW, design by committee doing what design by committee does best.
I agree about the unintuitive binding of names, though presumably the rationale is simply that any other approach would have introduced a significant amount of boilerplate. The only syntax I can think of would require a "bare as", e.g.:
FOUR_HUNDRED = 400
NOT_FOUND = 404
match code_and_val:
case (FOUR_HUNDRED, as x):
print(x) # FOUR_HUNDRED is not bound at all, but simply matched
case (NOT_FOUND as x, as y):
print(x, y) # prints 404, code_and_val[1]
not sure why the `|` overloading is a big deal here? there's no other significantly useful meaning for it in this context. Can you be more specific?
Followed by three decades of not wanting to do that particular breaking change, because doing so would be far more painful than anything that happened in Python 3.0.
Is this really true? I don't think lack of block scope is ever used for anything but causing bugs.
There’s a short discussion of this here: https://github.com/gvanrossum/patma/issues/146
So how do I do a bitwise-or operation within a pattern?
I think given issues like this and the commonly-cited NOT_FOUND example [0], it might have been better to only allow pattern matches on compound data and disallow its use with simple values. That’s where pattern matching is most useful, as in the article‘s second example (`match point`), plus it would make it more obvious that the behaviour of `match` is different from C’s `switch`.
[0] The following code unintuitively assigns the value of status_code to the variable NOT_FOUND:
NOT_FOUND = 404
match status_code:
case NOT_FOUND:
print("HTTP Not Found")
Therefore if you need to match against a calculated value, you can’t do the calculation inline. Instead you need to calculate the value outside the match and give it a name - but you can’t match against a named local variable either!
I frequently miss immutable data when working in Python; in general I’m conflicted about languages that retrofit functional programming features, because immutability is such a huge benefit.
I do wish we had better first-class immutable data in Python, but that wouldn't help with the
NOT_FOUND = 404
match val:
case NOT_FOUND:
...
Your "rightly" obviously varies, but this scenario is the poster child for enums, and you can match on enum members. The featured article has an example.
LAST_MATCHED_STATUS_CODE = 404
match status_code:
case LAST_MATCHED_STATUS_CODE:
print("HTTP Not Found")I gather that the proposed solution is to use something like enums? Or not do this at all?
import foo
from foo import BAR, BAZ, QUXNeat syntax overall but it's not clear what it makes more succinct than the equivalent `elif` statements.
> it's not clear what it makes more succinct than the equivalent `elif` statements
But pattern matching is shorter than if-else in many cases and can actually differentiate between structures which makes it very flexible. It's like switch-case should have been from the start imho.
As an example, in Rust I often do comparisons between certain values by simply matching a tuple of them:
match (foo, bar) {
(x, x) => true, // foo == bar
(Foo::VariantA(42), _) => unreachable!(), // has to be a bug
(Foo::VariantA(a), _) => a < 0,
_ => false,
}
Although I'm not sure what to think of Python's preference to put the condition and code in different lines; maybe that's optional, which could make it more succinct when the situation permits it.
« A match statement may have at most one irrefutable case block, and it must be last. »
So if I understand correctly the following will be an import-time error.
NOT_FOUND = 404
I_AM_A_TEAPOT = 418
match status_code:
case NOT_FOUND:
print("HTTP Not Found")
case I_AM_A_TEAPOT:
print("HTTP I'm a teapot")I fear a poor design like this is going to not only discourage people from using the feature at all, it is also going to poison the well and discourage people from using pattern matching in languages with a much more robust implementation. Which is a shame, because I actually like pattern matching.
(Tinfoil-hat take: Maybe that’s the whole point, after all Guido doesn’t like functional programming.)
This seems sensible and intuitive, `NOT_FOUND` is not different than `not_found` or `bob`. It follows the normal rules of Python bindings.
I'm way more annoyed by the special case for dotted names.
[0] https://twitter.com/brandon_rhodes/status/136022610839909990...
And hey, if it finally gets explicit variable declarations in Python I'm 100% all for it. implicit locals, `global`, and `nonlocal` are awful, viva `let` and `const` or whatever.
As to the content of the tweet
> The Substitution Principle, learned in algebra?
Python is not algebra, and already breaks algebraic reasoning to and through. Just consider that `a = a + b` and `a += b` are completely different operations with possibly (and actually, for some types) completely different effects.
Getting off-topic, but IMO that’s a major design flaw in the language. Some others agree, e.g. https://twitter.com/joepolitz/status/1304992696293978112
My point was that if you were teaching people that pull thon follows strict algebraic principles you were already setting them up for a rude series of surprises.
It is not that confusing once you have grooked it.
In ML and Rust, mistyping a a data constructor's name produces a variable, and thus matches everything, whereas OCaml and Haskell require that constructors start with an uppercase name, and disallows variables to do so, so this cannot easily occur.
But, since in Rust data constructors are typically qualified with a namespace except for some frequently used ones, this is not a problem in practice.
Meanwhile, the top post on /r/python for the last few days has been "Ladies and gentlemen - switch cases are coming!" This feature and its footgun syntax are going to be a disaster for the casual Python audience.
Can you do either of those in other popular languages with pattern matching? Presumably the reason for wanting pattern matching wasn't to invent something completely new, but to implement something similar to pattern matching in other languages. (I know that C-style switches allow you to write constant arithmetic expressions in case labels.)
Ruby supports matching against a local variable by pinning the variable (using a ^ prefix).
match status_code
case NOT_FOUND:=_ :
print(“clearly we’re binding a variable here”)
case _:=NOT_FOUND:
print(“and here we’re matching”)In the programming language I made a while back, my match statement was similar to Python's, and every case statement was also a valid l-value, so you could write `Point(x, y) | (x, y) = something` or `123 = x`, which would fail unless `x == 123`.
To be honest, I kind of enjoyed that the last decade when Python was frozen at 2.7 for users most of the time.
I understand arguments that simplicity means easier to learn, but also think that in most subjects in life we start with the basics and then break those rules to add nuance and continued expansion to the topics. See math, language, science.
I mean, you can say that about C++. You use it like C (it's a superset of C, after all), and gradually learn the features (STL, template meta-programming, polymorphism, etc.) as you go along. But I doubt that many would say c++ is beginner friendly.
As soon as you get into a team setting, though, all these advanced features become a way for more advanced users to unwittingly create a hostile environment for less skilled programmers.
Which maybe isn't a big deal in the software engineering department, where you can perhaps assume that everyone is a crack programmer. It can be a huge deal, though, for a lot of non-software-engineer departments that still spend a lot of time working with code. Those teams tend to be heavily dependent on languages like Python.
Of course, by extension, some languages have more complicated syntax (perhaps "less intuitive" based on languages you may be used to).
All said, I agree with you that some developers will indeed use higher level concepts which may not be understood immediately by lower level developers. I think this would occur even without those concepts embedded in the language itself, as the higher level developer would need create a solution to solve the problem anyway.
I hear this a lot about features in Python. Type hints, dataclasses, walruses, and now pattern matching. Whenever someone complains about language bloat, the reply is “it’s optional”.
But that’s only true as long as you’re writing code by yourself. Most developers spend much more time reading code than writing it, and with no control over which features others use, it’s necessary to be comfortable with the entire language.
What's more, you can even choose not to understand it. An easy way to do that is to choose not to work with other people's Python code.
Don't apply for Python jobs, and in a non-Python job, dodge tasks that involve touching Python code that contains constructs you don't wish to understand.
My argument still stands. I am so sick of people complaining about progress, especially armchair quarterbacks on HN.
We have to acknowledge that some newcomers are eager to learn everything, and that a bigger everything is harder than a smaller everything.
That's just a fact; it doesn't meant that a smaller everything is better.
Now that Python has pattern matching, newcomers should learn it.
In my opinion, newcomers should learn pattern matching before they learn lower-level, more verbose ways of writing code for classifying objects for type, shape and content.
The more verbose code should be the appendix, like "in older Python before pattern maching, we would have had to write it like this ...".
Or, to put it alternatively, newcomers no longer should learn how to use basic primitives like if comparisons and whatnot, to write code which combines multiple tests for object type, shape and content in multiple cases. They should be steered toward pattern matching, with a caution about how that is new and doesn't work with Python versions lower than x.y.z.
Pattern matching is particularly good for beginner programmers, because beginners programmers are not good at planning the design of a program ahead of time: they don't have an intuition for the right data structures and such. Pattern matching can patch over some representational difficulties/tensions in the program so that the code looks good and is easy to reason about and debug.
If all you have is a switch/case type statement, you might be told that, at some point, the pain you're feeling is from abusing that and should switch to something else, like proper OOP. In some of those same situations, you could instead switch to pattern matching. Pattern matching is less abstract and more literal-minded, since it uses "pictures" of data, which can overcome disparities in its structuring in a way that looks obvious in the code. Pattern matching has processing "under the hood", but what that does is not only largely intuitive, but also localized to the spot in the program where it is happening. I believe all of that favors the skill level of newbies.
The pitfalls in this miserable implementation are another story, but I think that will only help Python. Pitfalls bring Python people together to form communities and help each other. People band together and cooperate when they have common enemies, like sabre-toothed tigers, inclement weather or shitty programming languages and systems. The six-week-old Python newbie is eager to explain the quirks of pattern matching to the day-old newbies, and so it goes.
Other than the Python-specific quirks, newbies will likely struggle with bugs involving the order of pattern matching cases, when they place a more general case before a more specific one, such that it unexpectedly takes some (or even all) matches from that one. Adding new cases to a pattern matching block ahead of existing cases is tricky, if they match anything in common.
With pattern matching, you can write a small amount of code that handles a lot of ad hoc complexity---and if that doesn't have test coverage, that can spell trouble.
At least with unknown syntax there's usually some kind of keyword you can look up with Google. It's generally the non-syntactic sorts of unknowns that are much harder to figure out for a given codebase.
The quality of this proposal is much lower than other new features of the last decade.
> async model
> GIL
> scoping rules
> decorators
but these are not relevant for a beginner in programming, which is I think what advocates refer to.
Python makes it easy to read files, do something with the content and write it back, to print any variable. It runs even if you have syntactic errors in an unused function. A beginner's Python code can be dirty, but I think one reason why it's taught so often is because it's less frustrating to fix runtime bugs than cryptic compilation errors you see in safer languages, although as you said, it's detrimental to the code quality in the long run.
For instance the switch / case construct is on https://learnxinyminutes.com/docs/java/ (interestingly not on https://learnxinyminutes.com/docs/c++/ !), implying that it's quite basic there.
Plenty of languages make it very easy to read data from a file, process it, and then write it back out. F#, Elixir, Racket, LabVIEW, etc. all come to mind.
In those regards most new python 3-features are more on the good side. I know how often I googled for old python features like string substitution (the %-thing) or list comprehensions (the loop inside []) and I just could not figure out and remember the names, because % and [] are simply not self-explaining.
Not really, no. It hasn't been beginner-friendly for a decade or so now.
Regardless? Is there some protocol to allow user defined classes to be matched? One of the situations with algebraic datatypes is that it's very clearly defined what can and cannot be matched so that nothing is special and built in, but it also requires that concepts that should honestly be private be exposed. — it would perhaps be better to have some kind of protocol whereby some sort of “match object”is returnen for this functionality.
for x in input:
backup = x
point = f(x)
match point:
case Point(x = 6, y = 9):
print('found a nice point')
case _:
print('not nice point')
assert x == backup # this may fail
https://www.python.org/dev/peps/pep-0634/#match-semantics
> During failed pattern matches, some subpatterns may succeed. For example, while matching the pattern (0, x, 1) with the value [0, 1, 2], the subpattern x may succeed if the list elements are matched from left to right. The implementation may choose to either make persistent bindings for those partial matches or not. User code including a match statement should not rely on the bindings being made for a failed match, but also shouldn't assume that variables are unchanged by a failed match. This part of the behavior is left intentionally unspecified so different implementations can add optimizations, and to prevent introducing semantic restrictions that could limit the extensibility of this feature.
It also includes a "launch binder" button which fires up a Jupyter Notebook running a fork of Python with the new syntax ready for people to try out interactively.
match val case True:
print("Ok")
case False:
print("Not ok")
print("Done")Still, it looks like a decent addition.
I wonder if they could have used * for the wildcard.
1. Why not make patterns objects like pretty much everything else in Python (in particular: types, context managers, etc.)? I'm disappointed that it will be impossible to re-use patterns.
2. Why is the syntax for placeholders the same as for variables in usual Python expressions? Why not go with with `?placeholder` or `$placeholder`? I'm aware of the "symmetry" between construction and deconstruction that PEP-635 mentions[4] but in this case I find it rather disadvantageous: It will now be extremely cumbersome to insert constants into a pattern, as it will have to be done either through guards[1] or through attribute access[2]. More importantly, though, Python's syntax has now become extremely confusing over night – some identifiers in the pattern refer to placeholders, some others to existing variables. Sure, this is no different in function declarations that include type hints and default values. But the syntax here is a lot more complex and there is no clear line of separation as with function calls where the ":" and "=" tokens make it clear which identifiers are placeholders and which are existing objects. I am aware that PEP 640[3] was rejected but was it because of the question mark syntax or because of the fact that it tried to change lots of language features at once with no real benefit? (I myself would have rejected it for that reason, too.)
3. What was the idea behind overloading the class/constructor syntax as in `match [str(my_string_placeholder), foo]`? I can think of a lot alternatives and I have yet to find one that I find as confusing as this one.
4. Coming back to the idea from 1) of making patterns ordinary objects, such an object would basically be… a type. Or, put differently, pattern matching amounts to structural type checking plus the matching part. But Python already has structural typing through type hints, generic types and Protocols. The only drawback – apart from a couple pitfalls with Protocols – is that it is still impossible to use isinstance() to do check against generic types. My personal impression, though, is that a lot of people want that feature and that it will ultimately arrive. Or at least this is my hope. Then, however, we will have two ways to do structural type checks, which is clearly at odds with the Zen of Python. Did anyone consider this interplay of patterns vs. generic types? What will the future of generic types be? My complaint here is basically that the pattern matching PEP doesn't go far enough and that pattern matching on its own is a nice but rather niche feature. I think I would have liked a general way (or syntax) to describe the structure of objects in Python – but instead of improving the one existing way (type hints with generics and Protocols) we now have two suboptimal ways.
[0]: https://lwn.net/Articles/845480/
[1]: https://www.python.org/dev/peps/pep-0635/#guards
[2]: https://www.python.org/dev/peps/pep-0635/#id24