PEP 701 – Syntactic formalization of f-strings

"Highlight" in the sense of making it more pronounced in some way (e.g. background color, a link to the release notes etc). Also it takes time from the acceptance to the general availability of given PEP, and I think it shouldn't be highlighted for that period.

> because performance matters - concat(a,b) is just "{a}{b}" if and only if a compiler is sufficiently smart to reduce it to the minimum of operations needed

If I remember correctly these kinds of format strings have to be resolved by the compiler since they refer to variables in the source by name, which is information most languages do not keep around at runtime and a security nightmare if done dynamically. Also performance sensitive language like C aren't exactly known for a fast strcat, it is usually left as an exercise to the user to avoid catastrophic runtime behavior caused by the eternal search for nul.

It is really not a problem for even a very simple compiler to rewrite "{a}{b}" into a simple function call.

I'm not sure what you mean by "runtime introspection" in this context, because literals with embedded expressions don't require any runtime parsing or lookup - such a literal is parsed completely at compile-time, and in languages with lexical scoping, all variables etc are resolved then. It really is just syntactic sugar for concatenating; it's not at all like runtime templates.

String interpolation is not the same as C-style format strings.

Format strings live in userland as part of the standard library and have the many drawbacks you note.

Proper string interpolation is part of the language grammar itself, and the bits between the braces are usually full-fledged expressions that resolve variable names the same way that every other expression does. Since it's a language construct, it's trivial for it to just compile down to a series of concat(a, b) operations.

It sounds like Python had a half-complete string interpolation mechanism until this PEP, so I can understand why there'd be confusion on this point, but that is a problem with Python's implementation, not something that is an inherent risk with string interpolation. In a new language it is easier to implement string interpolation properly than to introduce a half measure like Python's.

Semantically, these say the same thing, so all of those concerns exist for “concat(a, b)” too. Avoiding having to parse the concat every time the line is invoked, avoiding having to apply runtime introspection to find out mapping between the variables and their locations, etc.

Agreed. This is one of those cases where "there should be one -- and preferably only one -- obvious way to do it" ought to apply. Of course, there are practical backwards compatibility reasons to keep + working as it does.

But overloading + for lists in general is a footgun anyway, especially in a dynamically typed language where the actual behavior is determined at runtime. And Python went one step further and also defined += for lists, and did it in a way that is even worse, because this:

   xs += [1]

does not behave the same as this:

   xs = xs + [1]

(the latter creates a new list and assigns the reference to it to xs; the former mutates the list in-place without changing the value of the variable)

Then on top of that you have scoping issues, too, because += is considered an assignment operator, and thus variable on the left side of it is considered a local unless you have a "global" declaration. So if you do this:

   xs = []
   def foo(): xs += [1]

it doesn't work as expected, because xs inside foo is a new local that shadows the global xs - and it fails at runtime because local xs is not initialized. You have to either do "global xs", or else write xs.append(1) in this context. At which point you might as well just use .append() everywhere, since at least it works consistently.

All in all, it's a good illustration of what happens when people get too "clever" with syntactic sugar when designing a PL.

I prefer the js:

  ys = [1, ...xs, 5]

It is more visually indicative of a 'spread' of values.

i think this was probably doable with some form of tuple magic in one form or another, but i agree. now if only they would add the spread operators to the ‘operator’ package… (don’t ask why— the heart wants what it wants…)

I think you're confusing string interpolation with C-style format strings.

Format strings are what you describe: either too weak or a completely insane sub-language.

The kind of string interpolation that OP is talking about is different in that what is in between the braces is just an expression and can usually be an arbitrary expression. It doesn't need to be a complex mini language because it just uses the same expression language that you're using everywhere else, with the only additional rule being that your expression must return a string (or in some languages something that can be coerced to a string).

From reading this PEP, it sounds like Python's original implementation was a bizarre half measure—it was almost like proper string interpolation but had a bunch of restrictions that showed that they stopped short of changing the parser to just go into expression mode in between the braces.

But string interpolation like that is, effectively, a language operator. Which even in the simplest case (no mini language to format values) is exactly as expressive as concat operator.

Python just lets you multiply strings, so you can do ("foo" * 80) etc. Which of course has all the same problems as overloading + for string ops. Perl had a better idea there, but that in turn is kinda forced by the design that implicitly converts strings to numbers and vice versa - you really, really don't want to be in a situation where 1+2, "1"+2, 1+"2", and "1"+"2" are all valid but inconsistent (looking at you, JS...).

TIL. str.format and % parse the format string at runtime, so it's expected that they're slower. But I thought the f-string desugars to something similar to the last one. Turns out it's not, it compiles to specialized opcodes and avoids function calls:

https://bugs.python.org/issue25483

https://bugs.python.org/issue27078

I used to be skeptical, but the great David Beazley convinced me that f-strings rocked because they were faster and better, in his talk "The Fun of Reinvention (Screencast)".

The point he made was to embrace the new features of Python, and put old Python out of its misery, instead of wasting your time with backwards compatibility.

Just call whatever you're doing in the latest version of Python a "prototype", then ship it.

At 10:20 he talks about things you can do to put old Python out of its misery (dead parrot), including f-strings and ordered dictionaries.

The Fun of Reinvention (Screencast):

https://www.youtube.com/watch?v=js_0wjzuMfc

Invited Keynote Talk from PyCon Israel, June 12, 2017. I cause trouble and build a framework using all sorts of new Python 3.6+ features.

Surprising that the double concatenation is the second fastest!

Logging modules mostly, especially if you have objects that can take a while to serialize to a string, can benefit from the "old" format string approaches. (Percent formatting is for some reason favored, although there's no real good reason to not just use the .format() approach outside of the standard logging library being a dinosaur.)

For conditional formatting (where you don't know if the formatting is needed or the contents of what you're formatting into the string but you know what you want to format), f-strings also lose their use pretty much immediately.

They're great for a quick inline format but less useful for things like string templates you want to use over and over.

Be sude that would make it hell to write strings with literal {}.

I like having different types of strings with different sets of magic characters. Even having identical `'` and `"` strings just to avoid backslashes in a few cases is great.

From the start? I guess because the spirit of Python's syntax is anti-Perl.

Now? Maybe because another breaking change in strings is the worst nightmare of the current generation of Python developers.

Python f-strings are modelled after `string.format`, which is inspired by C# and introduced relatively later. Such interpolated strings would have been much more inconvenient for the older printf-like syntax.

Yea, Swift really shows how it should be done. One string literal type that can do everything f-strings, r-strings, and """-strings can do and much more.

> Never really understood why not all strings were f-strings in python.

Sometimes you don't want interpolation, and it seems more reasonable to default to a version without such a feature than to default to a version with it and have an alternate syntax for non-interpolated strings. Or, at least, that makes sense to me, but I have no knowledge of their motivations.

Because it wouldn't be backwards-compatible.

If Python were designed from scratch today, I suspect it'd do something similar to JS.