What's Coming in Python 3.8
lwn.net
lwn.net
People are wtf-ing a bit about the positional-only parameters, but I view that as just a consistency change. It's a way to write in pure Python something that was previously only possible to say using the C api.
Digression on the old way's shortcomings: Probably the most annoying thing about the old "format" syntax was for writing error messages with parameters dynamically formatted in. I've written ugly string literals for verbose, helpful error messages with the old syntax, and it was truly awful. The long length of calls to "format" is what screws up your indentation, which then screws up the literals (or forces you to spread them over 3x as many lines as you would otherwise). It was so bad that the format operator was more readable. If `str.dedent` was a thing it would be less annoying thanks to multi-line strings, but even that is just a kludge. A big part of the issue is whitespace/string concatenation, which, I know, can be fixed with an autoformatter [0]. Autoformatters are great for munging literals (and diff reduction/style enforcement), sure, but if you have to mung literals tens of times in a reasonably-written module, there's something very wrong with the feature that's forcing that behavior. So, again: f-strings have saved me a ton of tedium.
Have you looked at textwrap.dedent?
raise ValueError("File exists, not uploading: "
f"{filename} -> {bucket}, {key}")
raise ValueError(("File exists, not uploading: {filename} -> "
"{bucket}, {key}").format(filename=filename,
bucket=bucket,
key=key))
raise ValueError(
(
"File exists, not uploading: {filename} -> "
"{bucket}, {key}"
).format(filename, bucket, key)
)
raise ValueError("""File exists, not uploading: {filename} ->
{bucket}, {key}
""".dedent().format(filename=filename,
bucket=bucket, key=key))
str_fmt = "File exists, not uploading: {filename} -> {bucket}, {key}"
fmt_vals = dict(filename=filename, bucket=bucket, key=key)
raise ValueError(str_fmt.dedent().format(**fmt_vals)) raise ValueError(“File exists, not uploading: {filename} -> {bucket}, {key}”.format(**locals())) raise ValueError(("File exists, not uploading: {filename} -> "
"{bucket}, {key}").format(filename=filename, bucket=bucket, key=key)) print('I do not get executed :)')
f'{!}'
File "stefco.py", line 2
f'{!}'
^
SyntaxError: f-string: empty expression not allowed
I've used assignment expressions in other languages too! Python's version doesn't suffer from the JavaScript problem whereby equality and assignment are just a typo apart in, eg., the condition of your while loop. Nonetheless, I find that it ranges from marginally beneficial to marginally confusing in practice.
Ergonomically, I see little benefit for the added complexity.
Special cases aren't special enough to break the rules, Although practicality beats purity.
I don't want a type error for the most clear use case, in the same way I don't want one for print, because if I wanted a behaviour other than the basic string representation then I would still need to call something differently anyway.
Given that explicit control of the __str__ method is also baked into the language it's also very clear what to expect.
I like type errors when handling something like '1' + 1 because the JavaScript alternative is surprising and hides bugs. No surprises that a string formatter would coerce to string for me automatically (although I get that's maybe just personal feeling).
I love the f strings, they have made my codebase cleaner and clearer. Definitely a pythonic win.
"The straw that broke the camel’s back was a very contentious Python enhancement proposal, where after I had accepted it, people went to social media like Twitter and said things that really hurt me personally. And some of the people who said hurtful things were actually core Python developers, so I felt that I didn’t quite have the trust of the Python core developer team anymore."
Source: https://www.infoworld.com/article/3292936/guido-van-rossum-r...
Note that Guido also was in support of the walrus operator, it's not like he stepped down because he disagreed with it.
We've spent decades debating on the merits of spaces vs tabs, and Vim vs Emacs, and a ton of other completely pointless stuff.
What you're hoping is just a pipe dream, people will get invested in the most petty and asinine stuff out there, and take it as a personal insult if you disagree.
[1] https://www.python.org/dev/peps/pep-0572/#alternative-spelli...
That post makes a few things very clear:
* The argument over the feature did not establish an explicit measure of efficacy for the feature. The discussion struggled to even find relevant non-Toy code examples.
* The communication over the feature was almost entirely over email, even when it got extremely contentious. There was later some face-to-face talk at the summit.
* Guido stepped down.
But just because the feature shipped and design-by-committee is upon us doesn't mean we need to accept the outcome. Why couldn't there have been a more evolutionary path for this feature? For example, there is surely a way to write a prototype library to accomplish the same effect with slightly different syntax. (How about a single function `walrus(a, b)` that does what `:=` does?). Then let real user adoption drive the change. Maybe somebody will discover case statements from scala and want that instead.
I hope the committee models some amount of their work after WG21. C++ hasn't evolved so effectively because some people were magic visionaries. For the past decade, C++ has mostly ridden on the proven success of boost. And skipped a lot of the parts of boost that suck.
But there's a long standing trend of adding more and more of these small features to what was quite a clean and small language. It's becoming more complicated, backwards compatibility suffers, the likelyhood your coworker uses some construct that you never use increases, there is more to know about Python.
Like f-strings, they are neat I guess. But we already had both % and .format(). Python is becoming messy.
I doubt this is worth that.
I need to do 2 changes every time I change a format string - I need to remove the symbol representing it's placement and then I need to remove the argument passed to .format.
Also old formatting does not easily support arbitrary expressions in the placements, thus in order to get those you need to change the arguments passed to the .format.
f-strings get rid of those issues altogether. What you're showing is whatever you have in the brackets - 0 indirection and thus less margin for (unnecessary) errors.
That said, I also struggle to understand how you’d claim parent clearly not using Python enough, when your description of str.format shows a lack of understanding to it yourself. One of the advantages of str.format over %-formatting is exactly that you do not need to modify the arguments passed to str.format when removing components from the source string:
>>> '{0} {1} {2}'.format('a', 'b', 'c')
'a b c'
>>> '{0} {2}'.format('a', 'b', 'c')
'a c'
>>> '{x} {y} {z}'.format(x='a', y='b', z='c')
'a b c'
>>> '{x} {z}'.format(x='a', y='b', z='c')
'a c'
It's not the string that most of the developers care about, it's the presence of the arguments to that string. The issue they are solving is "I would like to see A, B and C", rather than the issue of "I have provided A, B and C - would you please hide B from the view".
>> But again, this doesn’t matter to parent’s argument. Nobody is arguing with you that f-string is better than alternatives for what it can do; we are trying to tell you that there are things it can’t do, and you did not get it.
Please elaborate on what the f-string can't do? You have not provided the answer in your post. In my opinion, the only issue f-strings haven't solved is capturing the arguments in lambdas (before interplation) instead of their direct values. You, on the other hand - do not provide a clear explanation.
Disagree. In cases where it's useful it can make the code much clearer. Just yesterday I wrote code of the form:
foos = []
foo = func(a,b,c,d)
while foo:
foos.append(foo)
foo = func(a,b,c,d)
foos = []
while foo := func(a,b,c,d):
foos.append(foo)
Also, it's times like these I'm really glad docker exists. Trying that out before docker would have been a way bigger drama
It's all about the culture. And Python culture has been protecting us from abuses for 20 years, while allowing to have cool toys.
Besides, in that release (and even the previous one), appart from the walrus operator that I predict will be used with moderation, I don't see any alien looking stuff. This kind of evolution speed is quite conservative IMO.
Whatever you do, there there always will be people complaining I guess. After all, I also hear all the time that Python doesn't change fast enough, or lack some black magic from functional languages.
I think this metric is grossly overestimated. Or your scope for "out there" is considering some smaller subset of python code than what I'm imagining.
I think the evolution of the language is a great thing and I like the idea of the type hints too. But I don't think most folks capitalize on this yet.
Of course, in a world of Python 2.7 still being a large code base and Python being used a lot for scripting, this will far from the truth for the entire ecosystem.
There is a bit of learning curve (because, well, it's not shell which is what most people are used to), and you do have to please the compiler before being able to run your script, but OTOH, you'll basically never have that "oops, I just deleted my working directory because I used a bad $VARIABLE" experience.
1. Anything that is in the world when you’re born is normal and ordinary and is just a natural part of the way the world works.
2. Anything that's invented between when you’re fifteen and thirty-five is new and exciting and revolutionary and you can probably get a career in it.
3. Anything invented after you're thirty-five is against the natural order of things.”
― Douglas Adams, The Salmon of Doubt
I stopped writing C++ at some point in 2008/2009 but I still keep track of it to some extent and I'm continually surprised by the nonsense that is introduced into the language. The whole RAII movement, for example, is just one massive band-aid on top of the previous mistake of allowing exceptions, etc..
It'd be mostly fine in the long run, but you have all these people using like 15% of C++ and complain about it all day long, making their libraries not usable from stuff that understands C (most of which have drastically improved on the whole paradigm). There's a solution here and it's not using whichever arbitrary percentage you've decided on of C++, it's realizing that there are way better languages with real interoperability in mind to talk about lower-level things.
FWIW, I tend to think of comparisons to Perl as being a lot like Nazi comparisons, only for programming languages. And I do think there's some wisdom to the Godwin's Law idea that the first person to make a Nazi comparison is understood to have automatically lost the argument.
It's just that, at this point, Perl is both so near-universally reviled, and so well-understood to be way too optimized for code golf, that any comparison involving it is kind of a conversation-killer. As soon as it shows up, your best options are to either quietly ignore the statement in which the comparison was made, or join in escalating things into a flamewar.
I've never regretted a Perl program that I wrote, used and discarded. And I've never been content with a Perl program I found I was still using a week after I wrote it.
IMO, the more complex a system is, the more fragile it tends to become. The same is true for programming languages. Features will clash with each other. You'll end up having 10 different ways to achieve the same thing, and it won't be obvious which direction to go.
Furthermore, I did my grad studies in compilers. I've thought about writing an optimizing JIT for Python. I really feel like CPython is needlessly slow, and it's kind of embarassing, in an age where single-core performance is reaching a plateau, to waste so many CPU cycles interpreting a language. We have the technology to do much better. However, the fact that Python is a fast moving target makes it very difficult to catch up. If Python were a smaller, more stable language, this wouldn't be so difficult.
Many have tried and failed, Google and Dropbox to name a couple, and countless other attempts.
Lastly, AFAIK, CPython's FFI API is actually more of a problem than the dynamic semantics of the language. You can expose Python objects directly to C code. That makes it very hard for a JIT to represent said Python objects in an efficient way internally.
I disagree with this, which is precisely why I prefer feature rich languages like Java or better yet Kotlin. It doesn't get much more readable than something like:
users.asSequence()
.filter { it.lastName.startsWith("S") }
.sortedBy { it.lastName }
.take(3)
Probably before then too.
(He'd also written about that more at length in the 1990s in On Lisp, but many more people became acquainted with his language-design ideas in the 2001–2003 period, thanks to Lightweight Languages and his increasingly popular series of essays.)
Anyway, I'm talking specifically about the use of the identifier "it" in Kotlin, not implicitly or contextually defined identifiers in general, which are indeed a much more widespread concept, embracing Perl's $_ and @_, awk's $0 (and for that matter $1 and $fieldnumber and so on), Dyalog APL's α and ω, Smalltalk's "self", C++'s "this", dynamically-scoped variables in general, and for that matter de Bruijn numbering.
Compared to the existence of Perl, yes. Anyone who does any amount of Perl learns that $_ is the implicit argument ("like 'it'") to most functions. It's pretty much one of Perl's main deals. The talk has about 100K views on YouTube, which is pretty good, but Perl is in another league.
I think if you like this idea, you will really like the book. Better still, you can download the pdf for free.
[1] https://en.wikipedia.org/wiki/Anaphoric_macro [2] http://www.paulgraham.com/onlisp.html
users.asSequence()
.filter { user ->
user.lastName.startsWith("S")
}
.sortedBy { user ->
user.lastName
}
.take(3)Though I will conceed that I also find the fluent interface variant nicer.
sorted((u for u in users
if u.last_name.startswith("S")),
key=lambda u: u.last_name
)[:3]
sorted((u for u in users
if last_name(u).startswith("S")),
key=last_name
)[:3]
On top of the lessened readability, the Kotlin version makes it very easy to add, subtract, or comment out lines/actions which really helps when debugging. The Kotlin version is almost identical in structure to how you’d do it in Rust, Elixir, etc.
It should also be noted that Python would allow a more functional style, possibly leaving out the list comprehension.
The map filter chaining is obviously simpler, but python code is not that difficult and it's a no brainer task anyway.
It does contain a generator expression though, which is the same as a list comprehension in general structure, but slightly more confusing because it doesn't have the relationship to lists that square brackets in a list comprehension would have given it.
My point, though, was that not being able to tell the difference was a key "tell" that the comment author was not very familiar with Python — in some contexts, that would tend to undermine credibility in their comment (and then it would be rude to point it out), but in this context, it probably makes their opinion more objective.
Python is for the most part overrated when it comes to these things, IMO. It's a nice enough language but it's aged badly and has an undeserved reputation for concision, readability and being "simple".
The method chaining syntax and the query syntax are alternatives. I think most devs lean towards the former, considered to be cleaner... whereas the latter is probably easier to learn in the beginning, to those unfamiliar with piping/functional style - owing to its SQL feel-alikeness.
ReSharper would offer converting the latter to the former, and that's how I learned method-chaining LINQ back in the day.
users.apply {
asSequence()
filter { it.lastName.startsWith("S")
sortedBy { it.lastName }
take(3)
}
That's not universally true. C# has more features than Java but is generally easier to read and the intent of the code is easier to follow. The lack of features, like properties or unsigned integers, leads to Java coders creating much more convoluted solutions.
If languages with less features were universally better we would all be using C and BASIC for everything.
I can plainly see how these changes will actually make my code cleaner and more obvious while saving me keystrokes.
I also don't think these changes are very drastic. They're opt-in, doesn't break anything and looks to lead to cleaner code. I love the walrus operator (not so sure about the name, but hey. C++ is getting the spaceship operator... As has been said, naming things is hard). To me, the change of print from a statement to a function has been the hardest Python chamge over the years. Just too much mental momentum. Even though ive been on Python 3 for years, I still make the mistake of trying to use it as a statement. That said, I think it was the right (if painful) move.
I don't speak for everyone over 35, just myself.
Anecdote : i'm fairly young, but i've been involved with python long enough and traveled to enough pycons to be a bit jaded with regards to change within the language.
I'm fairly certain it's only due to the additional cognitive load that's thrust upon me when I must learn a new nuance to a skill that I already considered myself proficient at.
in other words : i'm resistant to change because i'm lazy, and because it (the language, and the way I did things previously) works for me. Both reasons are selfish and invalid, to a degree.
Yep, I entered the Python world with v2. I eventually reconciled myself to 2.7, and have only recently and begrudgingly embraced 3. Being over 35, I must be incredibly open minded on these things.
My understanding of Python will probably never be quite as good as my understanding of C, but I can live with that.
- Music That Matters: https://omny.fm/shows/kexp-presents-music-that-matters/playl...
- KEXP Song of the Day: https://omny.fm/shows/kexp-song-of-the-day
- All Songs Considered: https://www.npr.org/rss/podcast.php?id=510019
- KCRW Today's Top Tune: https://www.kcrw.com/music/shows/todays-top-tune/rss.xml
I’d argue it’s easier now to stay in sync than even when MTV was popular. MTV you needed a cable subscription and be sitting at a TV, now SiriusXM or Apple/Google/Spotify can stream it right to your phone laptop or tablet, and regular FM radio will play it on the local Top 40 station.
Anyone making this argument should be prepared to to accept every single criticism they make in their life moving forward can be framed as 'their resistance to change'.
This kind of personalization of specific criticism is disingenuous and political and has usually been used as a PR strategy to push through unpopular decisions. Better to respond to specific criticisms than reach for a generic emotional argument that seeks to delegitimize scrutiny and criticism.
ninjaedit: indeed https://www.python.org/dev/peps/pep-0572/#why-not-just-turn-...
The new operator (like many Python operators) can only be used in expressions (statements can contain expressions, but expressions are not a subset of statements.)
> The fact that there is now an operator that can only be used in certain statements just makes things more confusing
Because the “=” operator is the thing that defines an assignment statement. Even if this could be resolved unambiguously for language parsers, so that “statement defining” and “within expression” uses didn't clash, it would be create potential readability difficulties for human reading. Keeping them separate makes the meaning of complicated assignment statements and assignment-including expressions more immediately visually clear.
I think the major argument (at least, the one I see most frequently) is that the walrus operator does create readability difficulties for humans, which is exactly why many people view it as non-pythonic. This is one of the few times I've seen someone argue that ":=" makes things more readable.
This is in no way contrary to the argument that the walrus operator improves readability of expression assignments compared to using the same operator that defines assignment statements.
>>> locals()['a'] = 1
>>> a
1
x = [1, 2, 3, 4]
def foo():
x[0] += 3 # Okay
def bar():
x += [3] # UnboundLocalError
def qux():
x = [5, 6, 7, 8] # Binds a new `x`. def bar():
x += [3] # UnboundLocalError
0 LOAD_FAST 0 (x)
2 LOAD_CONST 1 (3)
4 INPLACE_ADD
6 STORE_FAST 0 (x)
8 LOAD_CONST 0 (None)
10 RETURN_VALUE
Maybe it would have been better to only have a single := assignment operator to begin with. But it's a few decades too late for that.
For what it's worth, := is for expressions only. Using it as a statement is a syntax error. So there won't be a lot of cases where both are equally elegant options.
a = 17
print("a=", a)
print("a=" + str(a))
print("a=%s" % a)
print("a={}".format(a))
print(f"a={a}")
# python 3.8 =>
print(f"{a=}")
But, if it sounds like I agree with you, I actually don't. I feel that the Zen of Python has taken on an almost religious level of veneration in people's minds, and leads to all sorts of unproductive debates. One person can latch onto "there should be one obvious way to do it" and another onto "practicality beats purity" and another onto "readability counts." Who's right? All can be. Or none. All could be applied to this particular case.
The Zen of Python is just a set of rough heuristics, and no heuristic or principle in the field of software development applies 100% of the time, IMHO. <= except for this one ;)
print string.Template("a=$a").substitute(a=a)In cases like this, different ways to do it (all equally good) are needed to get a good coverage of different tastes in obviousness and different nuances in the task.
The point is not uniformity, but avoiding the unpleasant and convoluted workarounds caused by non-obviousness (thus making the language easy to use).
String formatting is not trivial: there is the split (sometimes architectural, sometimes of taste, sometimes of emphasis) between concatenating string pieces, applying a format to objects, setting variable parts in templates, and other points of view; and there is a variety of different needs (cheap and convenient printing of data, dealing with special cases, complex templates...)
You do know that this:
x := 1
When is an expression allowed to have := in it, is
(x := 1)
(x := 1)
But while there are now two ways of doing assignment, I wonder how often people will actually encounter situations where it's difficult to figure out which choice is better.
[0] https://www.python.org/dev/peps/pep-0572/#exceptional-cases
Every possible line of code has an alternate ugly way to write it. This isn't a valid criticism. Anyone who decides to start writing simple assignment statements like that deserves to be ridiculed for writing ugly code.
I just dislike that the simple syntax rule "any expression can be used as a statement" now has an exception.
I haven't been able to think of scenarios where that might have consequences (code generation or refactoring tools?) but that doesn't say much as I'm not that smart.
Edit: having looked at the cases that are disallowed, they remind me of generator expressions. Those are usually written with parens, that can optionally be omitted in some cases. := is the same except they can be omitted in so many cases that it's easier to list the cases where they can't.
I think a generator expression used as a statement already requires the parens, even though they can be omitted e.g. as a single parameter of a function call. So that's probably ok then.
> There should be one-- and preferably only one --obvious way to do it.
There are plenty of ways to do assignments. Walrus assignments are only the obvious way in certain cases, and in general there aren't other obvious ways. For testing and assigning the result of re.match, for instance, walrus assignments are clearly better than a temporary.
I can think of lots of nonobvious ways to do assignments, like setattr(__module__...)
Python is one of the least "only one way to do things" languages I've used. This even extends to its packaging system, where you can choose between virtualenv, pipenv, pyenv, etc. Same goes for the installation method too, do you want to install Python with Anaconda or use the default method?
As for my personal take on this feature: I think it's really useful. When web-scraping in Python, I oftentimes had to do this:
while True:
html_element = scrape_element()
if html_element:
break
sleep(10)
while not html_element := scrape_element():
sleep(10) for html_element in iter(scrape_element, None):
And pyenv is just a version manager, like rbenv or nvm. I wouldn’t consider its existence confusing, not would I say being able to install something in more than 1 way has any relevance to the zen of Python!
Should Python create some cross-platform Uber-installer so that there is only one download link?
Regardless of what pyenv, the rest of my comment about the complexity of Python's tooling still stands. There's too many choices. I also seen people use pyenv as an alternative to virtualenvs, which is something I have never seen with nvm.
I don't understand why the Python community hasn't coalesced around a single solution to package management that has minimal complexity. It seems like pipenv is the solution, but there is controversy around it and it should have come several years ago. The fact that Python packages are installed globally by default is also pretty terrible, I much prefer it when applications bundle their dependencies. When I do `npm install --global`, the resulting program will always work, regardless of what other packages I have installed on my system.
> Any critiques on Python-the-language?
The point of my original comment was not to necessarily critique the Python programming language, rather it was to point out that adhering to the "zen of Python" is a lost cause because the language/development environment is not designed as a curated experience.
And my original comment did make points about Python-the-language. I talked about how there's many ways to do a single task in Python. One of the responses to it even proved my point:
"Prior to that you could use the special two-argument version of the `iter` function which makes it act completely different than the single argument version: <code sample>".
That unfortunately demonstrates my point.
This is one of the reasons I love Python. It's a great exercise to rewrite the same code imperative, recursive, with generators, with iterables, etc. Python is very good at supporting a wide range of programming styles.
Also, I can't bring myself to call it the walrus operator. Sorry, guys. I had a Pascal teacher ages ago who pronounced it "gets" and that has always stuck.
happy python programmer since 1.5, currently maintaining a code base in 3.7, happy about 3.8.
It was to allow only certain HTTP verbs on a controller function. A pattern adopted by most Python web frameworks today.
The development has been going quite well:
There are lots of issues that are being fixed. Strange nitpicking on alpha software.
If I wanted a language with "only one way to do it", i'd use Brainfuck. Which, btw, is very easy to learn, well documented, and the same source code runs on many, many platforms.
The walrus operator makes while loops easier to read, write and reason about.
Type annotations were a necessary and IMO delightful addition to the language as people started writing bigger production code bases in Python.
Data classes solve a lot of problems, although with the existence of the attrs library I'm not sure we needed them in the standard library as well.
Async maybe was poorly designed, but I certainly wouldn't complain about its existence in the language.
F strings are %-based interpolation done right, and the sooner the latter are relegated to "backward compatibility only" status the better. They are also more visually consistent with format strings.
Positional-only arguments have always been in the language; now users can actually use this feature without writing C code.
All of the stuff feels very Pythonic to me. Maybe I would have preferred "do/while" instead of the walrus but I'm not going to obsess over one operator.
So what else is there to complain about? Dictionary comprehension? I don't see added complexity here, I see a few specific tools that make the language more expressive, and that you are free to ignore in your own projects if they aren't to your taste.
x = 0
do:
x += 1
while:
x < 10But I don't think having it all on one line would be that bad.
(Edit: Actually, I think I know what you were saying now, and those aren't quite the same thing as they need a line after them.)
I do think the condition on the next line isn't the way to do solve this problem though (and I don't think it needs solving, while True: ... if ...: break does the job).
do:
body()
body()
while x < 10
Decorators already precede a function (or class) definition[2], and one alternative for the ill-fated switch statement[1] was to have switch precede the case blocks to avoid excessive indentation.
So there's plenty of precedent in the other direction.
[1]: https://www.python.org/dev/peps/pep-3103/#alternative-3
My main gripe is the indentation. Your code reads as if the while condition is tested after the loop finishes. What if the while statement was part of the loop and could be placed arbitrarily?
do:
body1()
body2()
while x < 10
body3()
Addendum: I would also entertain forcing the `while` to be at the end of the loop -- as I'm not sure what this would do
do:
if foo():
while x < 10> What if the while statement was part of the loop and could be placed arbitrarily?
If you're open to that, I had thought this was a bridge too far, but:
do:
body1()
break if some_condition
body2()
continue if some_other_condition
while True:
do_body
break
do:
thing()
thing()
continue if condition
do:
thing()
thing()
if condition:
continueNo, f-strings handle a subset of %-based interpolation. They're nice and convenient but e.g. completely unusable for translatable resources (so is str.format incidentally).
F strings are obviously non-lazy, but _(tmpl).format(_(part)) seems fine?
% only lets you format the values you're given.
> and isn't something like Django's _(str) better anyway
They're orthogonal. You apply string formatting after you get the translated pattern string from gettext. In fact, Django's own documentation demonstrates this:
def my_view(request, m, d):
output = _('Today is %(month)s %(day)s.') % {'month': m, 'day': d}
return HttpResponse(output)Doesn't mean nothing good comes out of them, and if it's simplicity that motivates people then eh, I'll take it, but gosh darn the cycle is a bit grating by now.
Projects like qmail discovered the reason in a somewhat _harder_ manner. And yes, I'd argue Python is yet another case, as it grew _at least_ as complex as Perl.
Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp. - Philip Greenspun
The "pow" example looks more like a case where the C side should be fixed.
There's plenty of situations where a named argument does not help, and encoding it can only hurt. It makes little to no sense to name the first argument to `dict.update` for instance. Or the argument to `ord`.
That, incidentally, is why Swift added support for positional-only parameters (though it has no concept of keyword-or-positional).
Yes, it limits your ability to rename a local variable, but that seems minor.
Swift's syntax is a lot more intuitive and consistent:
function(parameterWithImplicitlyRequiredLabel: Int,
differentArgumentLabel internalParameterName: Int,
_ parameterWithoutLabel: Int,
variadicParameterWithLabel: Int...)
function(parameterWithImplicitlyRequiredLabel: 1, differentArgumentLabel: 2, 3, variadicParameterWithLabel: 4, 5, 6, 7)
In reality, the 2.x releases had a lot of significant changes. Of the top of my head, context managers, a new OOP/multiple inheritance model, and division operator changes, and lots of new modules.
It sucks that one's language is on the upgrade treadmill like everything else, but language design is hard, and we keep coming up with new cool things to put in it.
I don't know about Python 3.8, but Python 3.7 is absolutely amazing. It is the result of 2 decades of slogging along, improving bit by bit, and I hope that continues.
I do agree that Python is moving further and further away from the only-one-way-to-do-it ethos, but on the other hand, Python has always emphasized practicality over principles.
Some kinds of data can be passed back and forth between processes with near zero overhead (no pickling, sockets, or unpickling).
This significantly improves Python's story for taking advantage of multiple cores.
Would this work with e.g. large NumPy arrays?
(and this is Raymond Hettinger himself, wow)
"multiprocessing.shared_memory — Provides shared memory for direct access across processes"
https://docs.python.org/3.9/library/multiprocessing.shared_m...
And it has the example which "demonstrates a practical use of the SharedMemory class with NumPy arrays, accessing the same numpy.ndarray from two distinct Python shells."
Also, SharedMemory
"Creates a new shared memory block or attaches to an existing shared memory block. Each shared memory block is assigned a unique name. In this way, one process can create a shared memory block with a particular name and a different process can attach to that same shared memory block using that same name.
As a resource for sharing data across processes, shared memory blocks may outlive the original process that created them. When one process no longer needs access to a shared memory block that might still be needed by other processes, the close() method should be called. When a shared memory block is no longer needed by any process, the unlink() method should be called to ensure proper cleanup."
Really nice.
But still copying?
If not, then how does it interoperate with garbage collection?
So it's not for containing normal Python dicts, strings etc that are individually tracked by GC.
https://docs.python.org/3.8/library/multiprocessing.shared_m...
I thought that when you use multiprocessing in Python, a new process gets forked, and while each new process has separate virtual memory, that virtual memory points to the same physical location until the process tries to write to it (i.e. copy-on-write)?
Empty space in internal pages gets used allocating new objects, refence counts updated or GC flags get flipped etc, and it just takes one write in each 4kb page to trigger a whole page copy.
It doesn't take long before a busy web worker etc will cause a huge chunk of the memory to be copied into the child.
There are definitely ways to make it much more effective like this work by Instagram that went into Python 3.7: https://instagram-engineering.com/copy-on-write-friendly-pyt...
E.G: live settings, cached values, white/black lists, etc
Sharing post-fork data is where it gets interesting.
With mmap you have to specify a file name (actually a file number), but so long as you set the length to zero before you close it there's no reason any data would get written to disk. On Unix you can even unlink the file before you start writing it if you wish, or create it with the tempfile module and never give it a file name at all (although this makes it harder to open in other processes as they can't then just mmap by file name). The mmap object satisfies the buffer protocol so you can create numpy arrays that directly reference the bytes in it. The memory-mapped data can be shared between processes regardless of whether they use the multiprocessing module or even whether they're all written in Python.
You may continue working on the standard library, optimizing, etc. Just no new language features.
In my opinion, someone should be able to learn all of a language in a few days, including every corner case and oddity, and then understand any code.
If new language features get added over time, eventually you get to the case where there are obscure features everyone has to look up every time they use them.
One of the takeaways is, that most languages and their features converge to a point, where each language contains all the features of the other languages. C++, Java and C# are primary examples. At the same time complexity increases.
Go is different, because of the simplicity first rule. It easens the burden on the programmer and on the maintainer. I think python would definitely profit from such a mindset.
Why should this be true for every language? Certainly we should have languages like this. But not every language needs to be like this.
When will this talking point die? It's not "ongoing". There's an overwhelming majority who have adopted Python 3 and a small population of laggards.
That small population includes every BigCo with a large python codebase.
Trivializing that by suggesting it was some offhand, unneeded solution to a problem that some dreamy “language designer” thought up is at best completely and utterly ignorant.
Also maintenance, in all forms, is work. That does involve updating your systems from time to time.
Sure, if you don’t program and just write ad-hoc (unmaintainable?) scripts then the transition is annoying. But it’s also not required. Don’t re-write your scripts, you can always ensure that Python 2 is present.
But if you’re maintaining a project that uses the wider ecosystem, then you are at the mercy of that ecosystem. And, at the time of the decision to make Python 3, that ecosystem was saying “Python 2 has a lot of horrible legacy decisions that make it harder than it should be to write good code”.
I have not seen a clear win in real benchmarks. 3 was slower for the longest time, and nowadays it seems head to head depending on the project.
https://speed.python.org/comparison/?exe=12%2BL%2B3.6%2C12%2...
https://speed.python.org/comparison/?exe=12%2BL%2B3.6%2C12%2...
Though I'm surprised nobody really wrote a transitional fork (six gets you a lot of the way but "Python 2.8 which has _just_ the str/bytes change" would have been useful).
Ultimately Python 2 isn't a better language, it's just the language everyone's code was in...
If you don't want to change/add something to the language, then why fork it?. You can just continue using it as it is!
Also Common lisp specs never changed since the 90s and is still usefull as a "quick and dirty" language, with few basic knowledge required. But the "basic feature set" can make everything, so the "understand any code" is not really respected. Maybe Clojure is easier to understand (and also has a more limited base feature set, with no CLOS).
From the v1.9 release just a few weeks ago: https://elixir-lang.org/blog/2019/06/24/elixir-v1-9-0-releas...
> As mentioned earlier, releases was the last planned feature for Elixir. We don’t have any major user-facing feature in the works nor planned. I know for certain some will consider this fact the most excing part of this announcement!
> Of course, it does not mean that v1.9 is the last Elixir version. We will continue shipping new releases every 6 months with enhancements, bug fixes and improvements.
I imagine churn will still happen, except it will be in the library/framework ecosystem around the language (think JavaScript fatigue).
Lisps avoid this by building abstractions from the same material as the language itself. Basically no other language family has this property, though JavaScript and Kotlin, via different mechanisms, achieve something similar.
How long has the code which was transitioned to python lasted?
A long time. 2to3 was good for ~90% of my code, at least
I write a lot of python for astrophysics. It has plenty of shortcomings, and much of what's written will not be useful 10 years from now due to changing APIs, architectures, etc., but that's partly by design: most of the problems I work on really are not suited to a hyper-optimized domain-specific languages like FORTRAN. We're actively figuring out what works best in the space, and shortcomings of python be damned, it's reasonably expressive while being adequately stable.
C/FORTRAN stability sounds fine and good until you want to solve a non-mathematical problem with your code or extend the old code in some non-trivial way. Humans haven't changed mathematical notations in centuries (since they've mostly proven efficient for their problem space), but even those don't always work well in adjacent math topics. The bra-ket notation of quantum mechanics, <a|b>, was a nice shorthand for representing quantum states and their linear products; Feynman diagrams are laughably simple pictograms of horrid integrals. I would say that those changes in notation reflected exciting developments that turned out to persist; so it is with programming languages, where notations/syntaxes that capture the problem space well become persistent features of future languages. Now, that doesn't mean you need to code in an "experimental" language, but if a new-ish problem hasn't been addressed well in more stable languages, you're probably better off going where the language/library devs are trying to address it. If you want your code to run in 40 years, use C/FORTRAN and write incremental improvements to fundamental algorithm implementations. If you want to solve problems right now that those langs are ill-suited to, though, then who cares how long the language specs (or your own code) last as long as they're stable enough to minimize breaking changes/maintenance? This applies to every non-ossified language: the hyper-long-term survival of the code is not the metric you should use (in most cases) when deciding how to write your code.
My point is just that short code lifetimes can be perfectly fine; they can even be markers of extreme innovation. This applies to fast-changing stuff like Julia and Rust (which I don't use for work because they're changing too quickly, and maintenance burdens are hence too high). But some of their innovative features will stand the test of time, and I'll either end up using them in future versions of older languages, or I'll end up using the exciting new languages when they've matured a bit.
Python, judged against JS, is almost sedate in its evolution.
It would be nice if a combination of language, libraries, and coding orthodoxy remained stable for more than a few years, but that's just not the technology landscape in which we work. Thanks, Internet.
Python was explicitly designed and had a dedicated BDFL for the vast majority of its nearly 30 year history functioning as a standards body.
JS, on the other hand, was hacked together in a week in the mid-90s and then the baseline implementation that could be relied on was emergent behavior at best, anarchy at worst for 15 years.
"Understanding" what each individual line means is very different from understanding the code. There are always higher level concepts you need to recognize, and it's often better for languages to support those concepts directly rather than requiring developers to constantly reimplement them. Consider a Java class where you have to check dozens of lines of accessors and equals and hashCode to verify that it's an immutable value object, compared to "data class" in Kotlin or @dataclass in Python.
I was really happy, in some ways, when Python 2 was announced as getting no new releases and Python 3 wasn't ready, because it allowed a kind of unification of everyone on Python 2.7.
Now we're back on the treadmill of chasing the latest and greatest. I was kind of annoyed when I found I couldn't run Black to format my code because it required a slightly newer Python than I had. But... f strings and walrus are kind of worth it.
So has John von Neumann's 29 state cellular automata!
https://en.wikipedia.org/wiki/Von_Neumann_cellular_automaton
https://en.wikipedia.org/wiki/Von_Neumann_universal_construc...
(Actually there was a non-standard extension developed in 1995 to make signal crossing and other things easier, but other than that, it's a pretty stable programming language.)
>Renato Nobili and Umberto Pesavento published the first fully implemented self-reproducing cellular automaton in 1995, nearly fifty years after von Neumann's work. They used a 32-state cellular automaton instead of von Neumann's original 29-state specification, extending it to allow for easier signal-crossing, explicit memory function and a more compact design. They also published an implementation of a general constructor within the original 29-state CA but not one capable of complete replication - the configuration cannot duplicate its tape, nor can it trigger its offspring; the configuration can only construct.
Edit: I actually forgot about the split between LuaJIT (which hasn’t changed since Lua 5.1), and the PUC Lua implementation, which has continued to evolve. I was thinking of the LuaJIT version.
All of which are ones that I once thought were quite enjoyable to work in, and still think are well worth taking some time to learn. But I submit that the fact that none of them have really stood the test of time is, at the very least, highly suggestive. Perhaps we don't yet know all there is to know about what kinds of programming language constructs provide the best tooling for writing clean, readable, maintainable code, and languages that want to try and remain relevant will have to change with the times. Even Fortran gets an update every 5-10 years.
I also submit that, when you've got a multi-statement idiom that happens just all the time, there is value in pushing it into the language. That can actually be a bulwark against TMTOWTDI, because you've taken an idiom that everyone wants to put their own special spin on, or that they can occasionally goof up on, and turned it into something that the compiler can help you with. Java's try-with-resources is a great example of this, as are C#'s auto-properties. Both took a big swath of common bugs and virtually eliminated them from the codebases of people who were willing to adopt a new feature.
That said, it is nice that I can take a Prolog text from the 1980s or 1990s and find that almost all of the code still works, with minor or no modifications...
As soon as people start using a language, they see ways of improving it.
It isn't unlike spoken languages. Go learn Esperanto if you want to learn something that doesn't change.
Though to me that's like saying, "I want this river to stop flowing" or "I'd prefer if the seasons didn't change."
Some of this stuff seems to me like it's opening the doors for some antipatterns that I'm consistently frustrated about when working with Perl code (that I didn't write myself). I had always been quite happy about the fact that Python didn't have language features to blur the lines between what's code vs what's string literals and what's a statement vs what's an expression.
mutation is tricky, a whole field of programming language research is built on avoiding mutation
A reasonably large number of the bugs I encounter relate to the order or number of formatting arguments not matching the slots in the format string. It's pretty hard to make that kind of mistake with an fstring.
I don't want it to scream on a missing docstring for every single thing. I do want to be able to use a, b and x as variables. No, this root var is not a constant, don't ask me for uppercase. Etc.
$ pylint pylint_sucks_at_partials.py
************* Module pylint_sucks_at_partials
pylint_sucks_at_partials.py:7:0: C0103: Constant name
"add5" doesn't conform to UPPER_CASE naming style
(invalid-name)
#!/usr/bin/env python
""" proving pylint still sucks at partials """
from functools import partial
from operator import add
add5 = partial(add, 5)
if __name__ == '__main__':
import sys
print(add5(int(sys.argv[1]))) if logger.isEnabledFor(logging.DEBUG):
logger.debug(f'{expensive_func()}')Using percent formatting is superior in many ways:
- errors that occur in formatting a message will be logged instead of raising an exception into your application
- error reporting services like Sentry can properly aggregate events
- interpolation won't occur if you don't have the logging level enabled
- it's recommend by the Python docs: https://docs.python.org/3/howto/logging-cookbook.html#use-of...
It kind of goes to the question: When is a language "finished"?
finished is dead, to put it less mildly.
If `print("{x}")` printed "{x}" in Python 3.5, it shouldn't print something else in a newer version. But `print(f"{x}")` was a syntax error before f-strings, so no code is broken by giving it a meaning.
JavaScript can't interpolate ordinary strings either, for the same reason. You need to use backticks (``).
Thank you for the explanation of the f-strings. I'm pretty sure that migrating old strings to "\{x\}" could be automated but we can't force everybody to migrate their code. There is probably no other way than the one they followed.
The correct way to escape braces in f-strings is with double braces, so f"{{x}}". This is consistent with the way str.format works. f-string syntax is very similar to str.format syntax in general.
I don't think I've come across any f-string abuse in the wild so far, and my tentative impression is that there's a few patterns that are improved by assignment expressions and little temptation to use them for evil.
It helps that the iteration protocol is deeply ingrained in the language. A lot of code that could use assignment expressions in principle already has a for loop as the equally compact established idiom.
I'm not familiar much with Python, beyond a little I wrote in my linear algebra class. How much does the statement/literal distinction matter to readability? What does that do for the language?
I obviously don't want that. I don't think anybody wants that. But I also don't think that's going to happen as a result of the recent changes in the language. If anything, I feel like the average code quality in the wild has gone up.
The first part of the statement (at least one obvious way to do it) goes to gaining a lot of expressive power from having learned only a subset of the language specification corresponding to the most important concepts. So you invest only a small amount of time in wrapping your head around only the most important/basic language concepts and immediately gain the power that you can take any thought and express it in the language and end up not just with some way of doing it, but with the right/preferred way of doing it.
The second part of the statement (at most one obvious way to do it) makes it easy to induce the principles behind the language from reading the code. If you take a problem like "iterate through a list of strings, and print each one", and it always always always takes shape in code by writing "for line in lst: print( line )" it means that, if it's an important pattern, then a langauge learner will get exposed to this pattern early and often when they start working with the language, so has a chance to quickly induce what the concept is and easily/quickly memorize it due to all the repetition. -- Perl shows how not to do it, where there are about a dozen ways of doing this that all end up capable of being expressed in a line or two. -- Therefore, trying to learn Perl by working with a codebase that a dozen people have had their hands on, each one preferring a different variation, makes it difficult to learn the language, because you will now need to know all 12 variations to be able to read Perl reliably, and you will only see each one 1/12th as often making it harder to memorize.
print(*lst, sep='\n')
depends on what you want! for example this Haskell package¹ defines three versions of while:
-- collect the result of each iteration into a list
whileM :: Monad m => m Bool -> m a -> m [a]
-- combine the result of each iteration together into a final result
whileM' :: (Monad m, MonadPlus f) => m Bool -> m a -> m (f a)
-- drop the results and return `()`
whileM_ :: Monad m => m Bool -> m a -> m ()
¹ http://hackage.haskell.org/package/monad-loops-0.4.3/docs/Co...
never understood the need for this. why do you even need statements?
if there's one thing that annoys me in python it's that it has statements. worst programming language feature ever.
That said, I think some things have unquestionably gotten more "Pythonic" with time, and the := operator is one of those. In contrast, this early Python feature (mentioned in an article [1] linked in the main one) strikes me as almost comically unfriendly to new programmers:
> Python vowed to solve [the problem of accidentally assigning instead of comparing variables] in a different way. The original Python had a single "=" for both assignment and equality testing, as Tim Peters recently reminded him, but it used a different syntactic distinction to ensure that the C problem could not occur.
If you're just learning to program and know nothing about the distinction between an expression and a statement, this is about as confusing as shell expansion (another context-dependent syntax). It's way too clever to be Pythonic. The new syntax, though it adds an extra symbol to learn, is at least 100% explicit.
I'll add that := fixes something I truly hate: the lack of `do until` in Python, which strikes me as deeply un-Pythonic. Am I supposed to break out of `while True`? Am I supposed to set the variable before and at the tail of the loop (a great way to add subtle typos that will cause errors)? I think it also introduces a slippery slope to be encouraged to repeat yourself: if assigning the loop variable happens twice, you might decide to do something funny the 2:Nth time to avoid writing another loop, and that subtlety in loop variable assignment can be very easy to miss when reading code. There is no general solution I've seen to this prior to :=. Now, you can write something like `while line := f.readline()` and avoid repetition. I'm very happy to see this.
[0] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
[1] https://lwn.net/Articles/757713/
[edit] fixed typos
This is relevant to what I've been doing in OpenCV with reading frames from videos! In tutorial examples on the web, you'll see exactly the sort of pattern that's outlined in the PEP 572 article.
>line = f.readline()
>while line:
> ... # process line
> line = f.readline()
Just, replace readline() with readframe() and the like. So many off-by-one errors figuring out when exactly to break.
> for line in iter(f.readline, ''):
> ... # process line
The sentinel argument is not unused.
> Where else does anything like this exist in the language?
In the narrow sense of “in the language” (builtins only), it's unique. There may be other examples in stdlib. That aside, functions that do a descriptively similar thing (i.e.,“build an iterator”) but where the details of how they do it differ significantly by aritt aren't weird among programming languages generally; don't see why it would be particularly problematic.
for x in iter(f.readline, ""):
for x in iter(lambda:f.readline() or None, None):
while predicate(x:=whatever()):
I will say, though, that I was not comfortable using iterators when I first learned python; walrus strikes me as easier to grok for a novice (one of the ostensible Python target demographics) than iter. I'll bet this is why this simple form is not idiomatic (though you're right, it should be).
How isn't it entirely obvious? := is the assignment operator in tons of languages, and there's no reason not to have assignment be an expression (as is also the case in many languages).
It is? Which ones? Other than Go, I can not think of a single language that has ":=" as an operator. Java does not, JavaScript does not, C/C++ do not, Ruby does not, I don't think PHP does, Erlang/Elixir do not, Rust does not... (I could be wrong on these, but I've personally never seen it in any of these languages and I can't find any mention of it in these languages' docs).
I tried looking around the internet at various popular programming languages and the only ones I could find that use ":=" are: Pascal, Haskell (but it's used for something else than what Python uses it for), Perl (also used for something else), and Scala (but in Scala it isn't officially documented and doesn't have an 'official' use case).
I don't have a strong opinion about ":=" in Python but I do agree that it's unintuitive and thus not very "Pythonic".
At least 18 prominent languages use that syntax.
The fact that they use some "new languages" (e.g. whatever derivative stuff happens to be in fashion atm) and are not even aware of the debt of those languages to the list above, doesn't qualify them...
If programming is intellectual onanism for you, then sure you're free to entertain that idea.
There aren't even 18 prominent languages...
What qualifies as prominent to you? How old are you? On Tiobe Index only Pascal and Go are in the first 50, while half of them aren't even listed in the first 100. Sure they're important and had an impact on new languages, but most of them were made ~50 years ago.
So many new languages were developed since then, which are far more useful and prominent than these legacy ones. If almost none of the modern ones have implemented it so far, is it really that useful/needed?
But perhaps not as interested in trying old and significant languages?
>What qualifies as prominent to you? On Tiobe Index only Pascal and Go are in the first 50, while half of them aren't even listed in the first 100. Sure they're important and had an impact on new languages, but most of them were made ~50 years ago.
Well, Lisp was made 60+ years ago, and C 50 years ago, so?
Besides Go, Smalltalk, Ada, and Pascal would be significant languages in any book, and I'd add Simula, Oberon, Eiffel, and Dylan to the list.
Seriously, if one haven't at least heard of Simula (the father language of OO) I'm not sure how qualified they are to pass PL judgement.
Well, they're still seeing widespread use, that's why they're on Tiobe, while others faded into obscurity. Those languages are historically significant, but nowadays they're basically useless apart from scientific use and maintaning old software.
Maybe you should understand that the majority of programmers are younger than Python and don't study the same material they did 30 years ago, because a whole lot of history happened in that time. Also I'm not sure how not knowing about Simula makes me unqualified for anything.
I've noticed, not just in this reply, but in all of your comments; your condescending tone and indirect addressing make you seem like an unpleasant person.
Using these qualities makes one seem like some stuck-up pseudointellectual boomer.
Actually, there are a lot of other variants assignments get written in various computer languages. https://en.wikipedia.org/wiki/Assignment_(computer_science)#...
The operator was inherited by Pascal, Ada, Delphi. That line of language syntax died off in the late 90's though, so I can see why younger (and in particular self-taught) programmers wouldn't be familiar with it.
So from the perspective of Python as executable pseudocode it makes some sense.
I've had to use a workaround for that every time I've tested a regular expression match that I wanted to process for example. Also problematic in comprehensions...
I should know, I've worked with Python for 22 years...
This feature is kind of a symbol, the first real decision of the transition between the bdfl and the next era.
I'm not worried about it, but yes, it was really against python core principles.
The zen of Python is not a binding constitution. It does not mean nothing can be added if there is some way to do it already, especially if that way improves things. It’s no more “going against the principles” than f-strings where, and they turned out just great.
So one might argue that a principle of python is that the language is dictated by how people read and write rather than the other way around... it's pretty hard to say what principles are "core" when they all conflict and you have to weigh various tradeoffs.
[1]: https://www.python.org/dev/peps/pep-0572/#the-importance-of-...
Regarding that it took 25 years, the normal Python syntax has been enormously successful. People don't tend to look for problems in things that work.
Guido also rejected several ideas that would totally fit with Python in those decades. There are lots of concerns (including implementation ones), not just what fits with some hypothetical "Zen", which was never meant as a contract anyway.
Besides Guido finally agreed to it, and even quit because of it.
If Python was to be kept "simple" at all costs, it would have added 20 other things, from operator overloading to yield from over those decades, some far more complex, and non-local than the operator change.
You can browbeat people all you like but we are not forced to work with any particular language and if it diverges away from what we liked we will just switch away.
In the link above, the "as" keyword would work with (very close to) all of those lines and arguably more readable.
x = function_that_might_return_none()
if x: do_stuff
All in all the debate has been heated and long, but it has been decided that the python community will use it intelligently and rarely, but that when it matters, it can help a lot.
I'm against this feature, while I was pro f-string. However, I'm not too worried about missuse and cultural shift because I've seen 15 years of this show going on and I'm confident on it's going to be indeed tagged as "risky, use it knowing the cost" by everybody by the time 3.8 gets mainstream.
They are most of the time a fantastic indicator of the cognitive load a feature will add in prod. Because of course a feature doesn't exist in a vacuum, it's always in a more complex context. So what's easy to understand for a student will be alright for a pro in the complexity of real life engineering. And I found the opposite to hold quite often as well.
I haven't tried the walrus on them yet, but I'm pretty sure of the result.
There are already so many complicated semantics about mutability, iteration, bytes/strings, scoping, et al. And yet somehow a tiny piece of syntax that finally lets us stop writing "while true/break" is the big pain point?
That is fine if you want to optimize language for "fresh students". That is not representative how brain process stuff after getting even some experience.
m = re.match(...)
if m:
... do something ...
if m := re.match(...):
... do something ...
if m := re.match(...); m {
... do something ...
}
with open('file') as f:
... do something ...
if re.match(...) as m:
... do something ...The walrus simultaneously names the value being tested so you can refer to it within the condition; it's sort of the inverse of Perl code using $_. So instead of
if (do_something()) {
act_on($_);
}
if placeholder := do_something():
act_on(placeholder)
If you really want to read the Python as it's written, it corresponds to the second of these sentences:
- If the substring from 3 to 5 is "fg", crash.
- If variable_name, the substring from 3 to 5, is "fg", crash.
Once way to test if this works is to take the code, read it aloud, and then use the read-aloud version to rewrite the code. If you don't have a high degree of certainty that you end up with the same code, something has failed along the way.
In this case, if I take "if x:= y()" and read it aloud as "if y", I think the vast majority of people would translate that to code as "if y():", which isn't the same thing.
1. You read more than one line of code.
2. Executing the code in the conditional more than once doesn't matter.
If you meet those two assumptions, then the reading I suggested will transform this
if x := y():
act_on(x)
if y():
act_on(y())
And if you're referring to this statement from your original comment:
> If variable_name, the substring from 3 to 5, is "fg", crash.
I don't find this to be a clear statement at all. If I read this aloud to any of my programming students, I doubt any of them would be able to decipher it into any code, let alone the code string which you've suggested.
A couple of reasons:
- The walrus eliminates a line of code in the very common scenario where you would prefer not to recalculate (or retype) y().
- The walrus makes it easy to avoid recalculating y() in the less common scenario where you need to avoid doing that.
> I don't find this to be a clear statement at all.
Nonetheless, it is the normal English syntax used to name something and then immediately define the meaning of that name. If you want to map the Python structure to an equivalent English structure, that is the equivalent English structure. ("Thomas Jefferson, the third president, was a man about whom much can be said.") If you want to map the Python code to an English statement of what the code does, use the reading I first suggested, "if y(), do something with it". If you want to dictate Python code to an English speaker, use the reading "if x colon-equals y()".
So let me ask you: is the problem you'd like to solve "I want to understand what this code does", is it "I like thinking about English grammar", or is it "I'm too busy to type my own code; that's what my secretary is for"?
The problem that has already been solved by every version <3.8 of Python, and that I would hate to see become un-solved by Python in the future, is that Python's greatest attribute, by far, is that it has always stuck to a code of being "pythonic", increasing accessibility, readability, and teachability to wide audiences. A hugely significant reason for Python's incredible rise in popularity and its use today is specifically because of it's readability. As much as it gets meme'd about, the fact that Python pseudocode is so close to real Python code is an enormous boon for the language.
I teach programming at a university level, and Python is the go-to, default language to teach programming. Dictating code so that it can be discussed in a classroom setting is very important, and as I mentioned before, your suggestion for reading it aloud just wouldn't cut it. Python is also the go-to, default language for programming-adjacent fields like data science, a lot of statistics, and every other non-programming-but-still-IT field. And again, this is because the people in these fields love the fact that, even with zero previous programming experience, they can look at or hear a piece of code and almost immediately understand what it is doing.
Python's strict adherence to being "pythonic" is hugely responsible for an entire generation of programmers, and hopefully will continue to be pythonic enough to continue lowering the barriers of entry to future programmers. I get that many seasoned developers are adopting an "well I got mine" attitude and don't care if future developers have a harder time learning the trade, but I personally find that to be very selfish, and I would hate to see future generations of programmers suffer just because the current generation apparently can't be arsed to do something like write one single, short extra line of code every now and then.
Every now and then? If you look at the example in the post, you'll see it saving one line of code per branch of the if-elif ladder.
Python code is characteristically tall, and this saves a lot of lines, and it saves them all in exactly the same way.
Yes, every now and then. The example in this post is an edge case. And even if it wasn't, the walrus operator saved a grand total of three (3!) keystrokes/characters per conditional in the example. "saves a lot of lines" is hyperbole. If the goal of this was saving programmer time or reducing the amount of code, the walrus operator should have been one of the absolute last things in Python to change.
Even just within this HN thread, even the staunchest proponents of this change are admitting that it is only going to be used sparingly, and at best saves a handful of lines of code per project.
If you're seriously telling me that saving a measly three keystrokes is more important to you than maintaining the pythonic philosophy that has made Python successful for decades, I can say nothing else other than that I strongly encourage you to reevaluate your priorities.
edit: I actually did the math wrong. It's only two (2!) keystrokes saved per conditional.
Even if y isn't mutating itself, it may be calling from a database updated from another thread.
You end up storing the value. The walrus simplifies the code.
For what it's worth, "x = y()" is one of the harder things for new programmers to translate to English in the first place -- it reads most naturally as "x equals y," but leads to better intuitions as "x gets the value of y". I think that's what makes this clunky to verbalize, rather than the "if truthy" bit.
> For what it's worth, "x = y()" is one of the harder things for new programmers to translate to English in the first place
Right... so why have we added more complexity to something known to be very complicated?
to quote the PEP:
reductor = dispatch_table.get(cls)
if reductor:
rv = reductor(x)
else:
reductor = getattr(x, "__reduce_ex__", None)
if reductor:
rv = reductor(4)
else:
reductor = getattr(x, "__reduce__", None)
if reductor:
rv = reductor()
else:
raise Error(
"un(deep)copyable object of type %s" % cls)
if reductor := dispatch_table.get(cls):
rv = reductor(x)
elif reductor := getattr(x, "__reduce_ex__", None):
rv = reductor(4)
elif reductor := getattr(x, "__reduce__", None):
rv = reductor()
else:
raise Error("un(deep)copyable object of type %s" % cls)No, “if <expression>” expands to “if <expression> has a truthy value”.
“<var> := <expression>” is itself an expression, which can be best (IMO) read in English as “<var> (which is <expression>)”
“... x := foo ...” is read “... x, which is foo, ...”
Looking at the module changes, I think my top pick is the changes to the `math` module:
> Added new function math.dist() for computing Euclidean distance between two points.
> Added new function, math.prod(), as analogous function to sum() that returns the product of a ‘start’ value (default: 1) times an iterable of numbers.
> Added new function math.isqrt() for computing integer square roots.
All 3 are super useful "batteries" to have included.
I think that the most important thing Python can do in each release is to improve performance, incrementally.
IMHO, the usefulness of this new operator outweighs the slight learning curve required to get past the awkwardness you will experience when you are first acquainted to it.
Here is that talk:
https://www.python.org/dev/peps/pep-0589/
I know it's almost always better to use objects for this, but tons of code still uses dictionaries as pseudo-objects. This should make bug hunting a lot easier.
I could see with c++ that between 2003 and 2014 a fair few underlying machine things were changing and that needed addressing in the language.
But Python is not quite as close to the machine, and I don't see how something like the walrus is helping much. If anything it seems like you'd scratch your head when you came across it. And for me at least one of the main attractions of python is you're hardly ever surprised by anything, things that are there do what you guessed, even if you hadn't heard of them. Function decorators for instance, you might never have seen one but when you did you knew what it was for.
Same with the debug strings. That seems to be a special case of printing a string, why not leave it at that? I'm guessing a lot of people don't ever read a comprehensive python guide, what are they going to do when they see that?
My guess would be "run it and see what it does".
It would be great if there was more momentum on this again, as it would be helpful in all sorts of places.
Like https://github.com/Tygs/ayo
It's not as good as having it in the stdlib, because people can still call ensure_future and not await it, but it's a huge improvement and completly compatible with any asyncio code.
When I was into Python, I liked it because it was a tighter, more to the basics language. Not having 4 ways to format strings and so forth. I don't think Python can defeat Java by becoming Java. It'll lose there due to multiple disadvantages. The way Python "wins" (as much as it could at least), is focusing on "less is more". They abandoned that a while ago.
My vision of a language like Python would be only 1-way to do things, and in the event someone wants to add a 2nd way, a vote is taken. The syntax is changed, and the old bytecode interpreter handles old scripts, and scripts written with the latest interpreter's bytecode only allows the new syntax. For me that's the joy of Python.
I think a lot of people wanted Python's original vision, "one way to do things". If I want feature soup, I'll use what I program in daily. Which I do want feature soup by the way, I just have no need to replace it with another "feature soup" language like Python turned into because it's inferior on technical and for me, stylistic levels.
By that standard, the walrus operator is not only acceptable but essential. Right now there are at least 3 ways to process data from a non-iterator:
# 1: loop condition obscures what you're actually testing
while True:
data = read_data()
if not data:
break
process(data)
# 2: 7 lines and a stray variable
done = False
while not done:
data = read_data()
if data:
process(data)
else:
done = True
# 3: duplicated read_data call
data = read_data()
while data:
process(data)
data = read_data()
while (data := read_data()):
process(data)
Also, this motto should be interpreted in the appropriate historical context – as taking a position in relation to that of Perl, which was dominant when Python was gaining popularity and had the motto "there's more than one way to do it".
Just recently 'Declined Proposal: A built-in Go error check function, “try”' https://news.ycombinator.com/item?id=20454966 made the front page, explaining how a controversial potential Go feature was being declined early.
Python on the other hand, went ahead with what seems to be a proposal at least as controversial as 'try' in Go.
Potential vulnerabilities aside, I got bitten by some migration issue back in the 2.2 to 2.4 transition where some built-in types changed how they did their __setstate__ and __getstate__ (iirc) and that caused objects picked under 2.4 to not unpickle correctly under 2.2 or something like that. After that I never wanted to use pickle in production again.
There's a bunch of changes in the official "what's new" doc that I think are more interesting:
https://docs.python.org/3.8/whatsnew/3.8.html
* Run-time audit hooks, to see if your modules are making network requests, etc.
https://www.python.org/dev/peps/pep-0578/
https://tirkarthi.github.io/programming/2019/05/23/pep-578-o...
* multiprocessing SharedMemory for fast data sharing between processes
https://docs.python.org/3.8/library/multiprocessing.shared_m...
* Duck-typing for the static annotation checkers
https://www.python.org/dev/peps/pep-0544/
* Literal checking for the static annotation checkers. ie: It's not enough to check that you're passing a string for the mode in open(), you want to check that it's 'r' or 'w', etc.
https://www.python.org/dev/peps/pep-0586/
* The compiler now produces a SyntaxWarning when identity checks (is and is not) are used with certain types of literals (e.g. strings, ints). These can often work by accident in CPython, but are not guaranteed by the language spec. The warning advises users to use equality tests (== and !=) instead.
* A bunch of speed and memory optimizations:
- "Sped-up field lookups in collections.namedtuple(). They are now more than two times faster, making them the fastest form of instance variable lookup in Python."
- "The list constructor does not overallocate the internal item buffer if the input iterable has a known length (the input implements __len__). This makes the created list 12% smaller on average."
- "Doubled the speed of class variable writes."
- "Reduced an overhead of converting arguments passed to many builtin functions and methods. This sped up calling some simple builtin functions and methods up to 20–50%."
reductor = dispatch_table.get(cls)
if reductor:
rv = reductor(x)
else:
reductor = getattr(x, "__reduce_ex__", None)
if reductor:
rv = reductor(4)
else:
reductor = getattr(x, "__reduce__", None)
if reductor:
rv = reductor()
else:
raise Error(
"un(deep)copyable object of type %s" % cls)
if reductor := dispatch_table.get(cls):
rv = reductor(x)
elif reductor := getattr(x, "__reduce_ex__", None):
rv = reductor(4)
elif reductor := getattr(x, "__reduce__", None):
rv = reductor()
else:
raise Error("un(deep)copyable object of type %s" % cls) m = re.match(p1, line)
if m:
return m.group(1)
m = re.match(p2, line)
if m:
return m.group(2)
m = re.match(p3, line)
...
if m := re.match(p1, line):
return m.group(1)
elif m := re.match(p2, line):
return m.group(2)
elif m := re.match(p3, line):
Unchecked type annotations remain the worst addition since 3.0. Actual typing might be useful; it allows optimizations and checking. But something that's mostly a comment isn't that helpful.
Ugh, how did this get approved? It's such a bizarre use case, and debugging by print should be discouraged anyway. Why not something like debug_print(foo, bar) instead (because foo and bar are real variables, not strings)?
Also, it's part of the format string and not a special print function so that it can be used for logs and other output as well, not just the console.
I use it myself all the time, but it just shows the weakness of the tooling that people have to resort to such measures. Fortunately, some people are working on it [1].
>Also, it's part of the format string and not a special print function so that it can be used for logs and other output as well, not just the console.
Since print (and hypothetical debug_print) are no longer statements like in 2.x, there's nothing preventing them from returning the string that's supposed to be printed. For example print's keyword file is sys.stdout by default. Why not borrow from Common Lisp's format and make it return the string if file=None is passed? Then you could do logging.warning(debug_print('Unusual situation', foo, bar, file=None)) and it would print "WARNING: Unusual situation: foo=foo_value, bar=bar_value" to the logs. It's so much clearer.
It's not resorting to anything. It's a valid means of debugging. People use it even in languages like c and Java and in-browser JavaScript with very capable debuggers. It's quick, simple, and doesn't require intervention to record or examine anything.
> Why not borrow from Common Lisp's format and make it return the string if file=None is passed?
Because thatsa terrible idea because it's non-intuitive, verbose, and potentially confusing. Debug format strings are common in other languages, such as rust, so this isn't some half-thoughtout, python-only idea.
I.e. since print isn't a statement, you can give other functions the same argument conventions that it has, instead of making the programmer use a string-generating shim.
Speaking of Common Lisp, it has functions other than format which take format arguments.
$ clisp -q
[1]> (error "value of ~s should be less than ~s" 'bloop 42)
*** - value of BLOOP should be less than 42
The following restarts are available:def fun(a, b, /, c, d, *, e, f):
or
print(f'{now=} {now=!s}')
and guess what it does before actually reading the article.
Worst, the rationales of the PEPs are weak, presenting improvement for "performances" or enforcement of API because of low level stuff as C.
Back when I was 18 years old, Python was high level, rules were simple with only one way of doing things and performances weren't a concern, because you would just use the right language for the right task. There was no enforcement and you could always hack a library to your liking. Python now is getting closer to what Perl looked to me 10 years ago, trying to optimize stuff it shouldn't
It's still marked as a 3.8 target
Specifically "There should be one -- and preferably only one --obvious way to do it."
If this was any other language, the addition would be welcome, but I feel that the walrus operator fundamentally disagrees with what python is about.
It's not about terseness and cleverness, it's about being clear, and having one way to do things (Unless you are Dutch).
The abbreviated f-string syntax looks weird and kinda wrong to me. But then I'm not even sure I've got comfortable yet with the object field initialization shortcuts in Javascript and Rust (where you also get to omit stuff to avoid repeating yourself).
F strings are pretty awesome. I’m coming from JavaScript and partially java background. JavaScript’s string concatenation can become too complex and I have difficulty with large strings.
>Python 3.8 programmers will be able to do: print(f'{foo=} {bar=}')
Pretty cool way to help with debugging. There are so many times, including today, I need to print or log some debug string.
“Debug var1 ” + var1 + “ debug var2” + var2...and so on. Forgot a space again.
console.log({var1,var2,var3});
And the logged object will get created with the variables content and the variable nem as key, so it will get logged neatly like
{var1: "this is var1", var2: 2, var3: "3"}
More compact code at the cost of higher learning curve.
It violates the philosophies of Python and UNIX where one function, or one line, should preferably only do one thing, and do it well.
I get the idea behind the :=, but I do think it's an unnecessary addition to Python.
A lot of folks see Go as a Python successor which surprises me because I don't think the languages favor the same things at all. Maybe my perspective is weird.
Python never had that philosophy... You might confused it with "there should be one, and preferably only one, obvious way to do anything".
no, it evaluates to the left side's value after assignment
>>> '%i' % 's' Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: %i format: a number is required, not str >>> '{}'.format('s') 's' >>> '{}'.format(10) '10'
This is slated for 3.9: https://www.python.org/dev/peps/pep-0554/
(Yeah I know, ; is a statement separator, not a statement terminator in Pascal.)
As long as you're being just like Pascal, did you know Python supported Pascal-like "BEGIN" and "END" statements? You just have to prefix them with the "#" character (and indent the code inside them correctly, of course). ;)
if x < 10: # BEGIN
print "foo"
# ENDFWIW, I agree with the sentiment; I use := for assignment in my language precisely because that's the correct symbol. But even there, my grammar accepts = as assignment as well because I type it from habit.
Generally, library authors won't be able to use it if they want to support many versions; same as with f-strings.
if val = expr():
Most likely, the assigned value is stored for use in one of the conditionals, so it really doesn't change any of that.
Let's also understand that we are dealing with a decorated assignment here, so a = (b = c) should be no different from evaluating (b = c). It's not complicated, the way I at least look at it.
= (existing) is statement assignment
== (existing) is expression equality
:= (new) is expression assignment
a=42
if b = a:
print(b)
else:
print("no")
int a,b;
a=42;
if(b=a){
printf("%d\n",b);
} else {
printf("no\n");
}It works less well if they can be used everywhere an expression can occur, including the right side of assignments—especially since Python has both multiple (x = y, z) and chained (x = y = z) assignment, which can be used together.
What does this mean if = is used for both assignment statements and assignment expressions:
x = y, z = 10, 20
x = y, z = 10, 20 # x: (10, 20), y: 10, z: 20
x = y, z := 10, 20 # x: (<existing value of y>, (10, 20)), y: <unchanged>, z: (10, 20)So much so that many have adopted the rule that the variable goes on the right, "if 42 = b", to make sure the compiler barfs when you intended to write "if b == 42".
With := it's less likely that mistake is made. I also find it visually more distinct, so easier to parse, but that might be very subjective.
if x = y:
if x == y:
if x := y:
[0]: https://www.python.org/dev/peps/pep-0572/#why-not-just-turn-...
The difference between = and == in an if causes many hugs in other languages. Using := for assignment in an expression instead of == means that you can't simply have a typo and have a bug.
Also fix the GIL.
pr<TAB> --> print(" ")
# ^ cursorReally, I dont like anything that trys to force a future developer into using your code the way you expect them to.
def my_format(fmt, *args, **kwargs):
...
fmt.format(*args, **kwargs)
You could always force developers into using your code the way you expect by parsing args/kwargs yourself, so it's not like this really changes anything about the "restrictiveness" of the language.
If you run `help(pow)` as early as Python 3.5 it lists the signature as `pow(x, y, z=None, /)`. The first time I saw that `/` I was pretty confused, and it didn't help that trying to define a function that way gave a syntax error. It was this weird thing that only C functions could have. It's still not obvious what it does, but at least the signature parses, which is a small win.
Another thing it's good for is certain nasty patterns with keyword arguments.
Take `dict.update`. You can give it a mapping as its first argument, or you can give it keyword arguments to update string keys, or you can do both.
If you wanted to reimplement it, you might naively write:
def update(self, mapping=None, **kwargs):
...
If you want to write it in pure Python < 3.8, you have to do something like this:
def update(*args, **kwargs):
if len(args) > 2:
raise TypeError
self = args[0]
mapping = None
if len(args) == 2:
mapping = args[1]
...
Arguably you shouldn't be using keyword arguments like this in the first place. But they're already used like this in the core language, so it's too late for that. Might as well let people write this:
def update(self, mapping=None, **kwargs, /):
...Although that being said I always really liked Perl
To be clear, that's not a new feature in 3.8.