Python string literals are kinda funny | Hacker News Reader

Python string literals are kinda funny

sebsite.pw

63 pointsjandeboevrie61 comments

Whenever I read people's takes about Python on HN, I always feel like I'm using an entirely different language.

I feel ya. I don’t write a lot of it anymore, but have written probably hundreds of thousands of lines over the years. It has a few rough edges, but over all it’s solid and I’ve used it to make lots things I’m proud of.

That you know how to use the language without such oddities presenting any difficulty does not make them less strange. That r-string parsing solution is fine for the interpreter, but we are not interpreters, so it's not a logical outcome for Python authors,

The most puzzling thing is few languages use the absolute simplest solution to escaping quotes, which happens to be especially useful for non-expanded literals, and that's good ol' quote-doubling. Difficult for Python to introduce now since it'd be a bc-break for implied concatenation, but if space were required between them, we could have had

x = r'ex\x20cape!\'

y = 'diff''rent'

z = 'diff' 'erent'

respectively containing

ex\x20cape!\

diff'rent

different

TOML has a similar issue: it is impossible to store its delimiter for non-expanded multi-line strings inside a multi-line non-expanded string. There's no method to escape it. Whilst this is rarely an issue in practice, it's odd that there's unnecessary difficulty in writing about TOML inside a TOML document. Again, it could have used the simpler quote-doubling method for all strings (even easier because no concatenation), with similar rules for non-expanded and multi-line variants. Then there would be no limitation on what can be stored in any literal type. Instead it has a peculiarity that's illogical and offers no benefit to us authors.

I'm not a fan of f-strings.

I feel like 90% of new Python features in the last 10 years just increased language complexity without any benefit.

Indeed, and I get that one can ignore those features (I do), but finding them in existing code or having the AI coding agent use them diminishes the "easy for beginners" aspect.

Are f-strings not easy for beginners? What would you prefer instead?

I'd prefer one way of doing things. I do understand that the improvements to string literals are improvements, but it means there are multiple things to learn.

Your one way is f-strings, that's it. Disregard the others unless you have an extremely good reason not to.

You and Claude agree. ;-) And me too, I must admit.

How is, say, `age = 5; print(f"Age: {age} years old")` not easy for beginners? IMO it's as easy as it gets: you want the value of `age` printed {HERE}, so you just put it where you want it, surrounded by curly brackets.

I'm a HUGE fan of f-strings and I think they should be added to more or less every language in existence. (This is just to show how much I like f-strings, not to be taken literally) `printf`-style format strings seem outdated: why use format strings when I can put my variables IN THE STRING? I want the result of `x+y` to be put {HERE} in this string. Well, just write `"This is here: {x+y} blah"` — it makes perfect sense and immediately lets me see what the resulting strings generally look like.

Basic usage is a no-brainer: write your string, put variables or short expressions in curly braces, add `printf`-style format specifiers after the colon. This is also great because it's a natural extension of `printf`-style format strings.

Of course you can write complicated and confusing f-strings. But then you can write complicated and confusing... anything, really. Many programming languages have extremely weird quirks and cases where basic syntax can be transformed into an unreadable monstrosity, like C syntax for pointers to functions and arrays.

Sure, this increases the language's complexity, but you don't have to use all of it to reap the benefits.

Those aren't the only two options. Like GP I don't like f-strings, but there was something introduced before that: the format function.

  "The thing is {foo}, and also {foo} again".format(foo=x+y)

It also supports positional with empty {}. And like f-strings, you can put formatting information after a colon.

%-based printf-style did also have named variables like this but it seemed less known.

This is still significantly more clunky than f-strings, especially when you're writing them a lot for debugging purposes

> why use format strings

Because these are used for locale-specific configuration data. `"This is here: {x+y} blah"` on the other hand isn’t a string (mere data), it’s a program, because you can have arbitrary expressions inside the braces. You don’t want to repeat the `x+y` in each localization file.

I have nothing against ergonomic program constructs for composing strings, but please let’s not confuse such program constructs with mere string literals.

If your string needs to be presented in multiple locales, you probably want to go a step further and use something like the ICU syntax and a proper parser and formatter rather than rely on manually formatting things yourself. Otherwise, that dynamic data is going to give you headaches when you have to deal with plurals and genders and such.

I’m not sure what you mean by manual formatting. You do need a place in your program where you fill in the parameters into the respective localized string template. I agree that the printf format syntax is somewhat limited for localization [0]. But whatever localization string format you use, you don’t want arbitrary expressions embeddable within it.

[0] though GNU libc does let you extend it: https://sourceware.org/glibc/manual/latest/html_mono/libc.ht...

It is baffling to me that Python seems to insist on reinventing language features and coming up with the most incomprehensible of designs. The whole dataclass and serialisation ecosystem comes to mind, as well as the evolution of typing.

The language exposes so much of its internals that even if the design were consistent, the ecosystem of (buggy) libs and tools makes it inconsistent.

PKL files and the entire multiprocessing paradigm is one of the worst experiences I’ve ever had with a programming language feature. Surely there has to be a better way.

I like f-strings but I don't like that there are at least five ways to format stings.

1. %-formatting [1]

2. str.format [2]

3. string.Template [3]

4. f-string [4]

5. t-string [5]

What happened to "one-- and preferably only one --obvious way to do it"? [6]

Also, the way string formatting interacts with logging is a total mess. People just pass f-strings to logging, which seems to be an intuitive way to do it. Except it limits your options if you want to collect structured logs and it doesn't allow you to use late evaluation based on log level.

[1] https://docs.python.org/3/library/string.html#format-example...

[2] https://docs.python.org/3/library/stdtypes.html#str.format

[3] https://docs.python.org/3/library/string.html#string.Templat...

[4] https://docs.python.org/3/reference/lexical_analysis.html#f-...

[5] https://docs.python.org/3/reference/lexical_analysis.html#t-...

[6] https://peps.python.org/pep-0020/

> What happened to "one-- and preferably only one --obvious way to do it"?

There arguably still is—just use f-strings for everything, unless you need to support ancient Python, in which case use %-formatting.

t-strings are a special case, but in theory most functions should only accept regular strings or templates, so there should only be one choice there too.

> Also, the way string formatting interacts with logging is a total mess [...] it doesn't allow you to use late evaluation based on log level.

This all seems to be a side-effect of the fact that string formatting produces static strings, so I don't think that there's much that can be done here (but maybe t-strings can be creatively used here somehow).

> maybe t-strings can be creatively used here somehow

Maybe they could, but I would be the first to oppose introducing creative ways to do logging or string formatting.

Such basic things should be done in the most standard way possible to reduce cognitive effort required to read and understand the code.

But the standard way is kinda ugly. Most people would expect f-strings to be used for "normal" string formatting and %-style to be used for logging, because it is the default and most codebases do it this way. Therefore, you basically forced to have (at least) two different formatting syntaxes in your codebase.

I say "at least", because if the program serves html pages, you most likely also have some other template engine like jinja...

t-string main use case is SQL (for instance, in psycopg¹)

You can write things like that without SQL injection:

  t"INSERT INTO mytable (first_name, last_name) VALUES ({first_name}, {last_name})"

This is a very pleasant feature, very useful

[1] https://www.psycopg.org/psycopg3/docs/basic/tstrings.html

It doesn’t have to be beneficial, it just has to be “pythonic.”

Goofy edge cases aside, I think f-strings are great.

In his 2017 PyCon Israel keynote, "The Fun of Reinvention", Dave Beazley made a wonderfully mischievous case for taking advantage of new Python features instead of making every new project carry the accumulated baggage of old Python versions.

Talking about Python 3.6, he said it was "probably one of the most major Python releases that has ever been made." His demonstration intentionally used new features that made the code incompatible with older interpreters. This was a prototype, so why not use the interesting new tools?

He was especially enthusiastic about f-strings: "F-strings are just awesome."

I was originally skeptical about them, but he convinced me to reconsider. One implementation detail that later helped win me over was that CPython 3.6 added dedicated FORMAT_VALUE and BUILD_STRING opcodes for f-strings. That does not mean an f-string is faster than simply doing a + b when both values are already strings -- simple concatenation usually wins that particular race -- but f-strings are generally cleaner, and often faster than older formatting machinery such as str.format().

I agree with his argument. There are vanishingly few situations in which a new project genuinely must support ancient Python releases. If your employer refuses to let you use a reasonably current version without a concrete technical reason, that is a warning sign. Life is too short to program indefinitely for obsolete interpreters.

The keynote:

https://www.youtube.com/watch?v=js_0wjzuMfc

Beazley's other talks:

https://www.dabeaz.com/talks.html

I think this is pretty intuitive (I was able to answer the question correctly before opening the spoiler), but I really like raw string designs in Rust and C++11 that allow you to stop worrying about escaping completely.

here's a valid f-string:

>>> f'{'}'}' '}'

Huh? I remember learning the rule that you can't nest quotes inside fstrings if they are the same kind (unlike in bash) - e.g

  f"{mydict["foo"]}"

would be a syntax error, but

  f"{mydict['foo']}"

  f'{mydict["foo"]}'

would be valid.

The reason being the same like for the rstring weirdness: The lexer comes first and identifies the string literal, then for fstrings, the python parser is invoked again for each {...} expression to parse it.

This is unlike other nested expressions, which are already split up by the lexer and then parsed in one go.

Did that change at some point?

Update: They really changed it! (in python 3.12)

There was a PEP about it:

https://stackoverflow.com/questions/78388333/nested-quotes-i...

https://docs.python.org/3.12/whatsnew/3.12.html#pep-701-synt...

Wow, I didn't know this either. I find it a bit crazy. It must make no sense without syntax highlighting. But I suppose who writes code without highlighting any more?

It's been a long time since I've had to SSH into a box to fix an issue in production. But it is comforting to know that I _could_.

VI (not even VIM) on minimal Debian installs does not have syntax highlighting.

How often do you have SSH access to a box but don't have SFTP access?

Most IDEs support editing files on a remote machine through SFTP.

It would be scp, not sftp, but I often don't know what I'm about to edit until I get in there and dig around.

No, I was talking about SFTP, not scp. OpenSSH server has SFTP enabled by default. You can mount it using sshfs or browse it using any SFTP client (or any program with embedded SFTP client, which is most IDEs).

So, if "digging around" means looking at folders' structure, you can do it through SFTP without copying everything (unlike scp or rsync).

And if digging around means running commands, nothing prevents you from having a session for running commands open in parallel.

I block [s]ftp at the firewall and disable it. I also keep ssh open only after port knocking. After reading logs and such, it's easier to open files with copy and paste via screen or tmux. Or even `Atl-.` depending on what I'm doing, framework, etc.

How exactly do you block sftp at the firewall if it uses the same port as ssh? From the firewall's perspective it is just some encrypted traffic inside ssh tunnel. I think that you might be mistaking sftp with ftps (ftp through ssl/tls tunnel), these are different protocols. I suggest you try connecting to any box you have access to using Sftp protocol and ssh credentials and see the result.

I completely forgot that sftp is on 22. Like I said, I block the port until it is properly knocked. In any case, I usually don't need the convenience of an IDE. It's much easier for me to just do everything over ssh. My IDEs (Jrtbrains, VS Code) use VIM keybindings too. And I could just as easily apt get vim on the server.

In any case, it's literally been years since I've had to.

Interesting read. I had no ides that they switched the parser https://peps.python.org/pep-0617/

do not get the funny part of the pythons string.

Whenever I'm building a JSON document, I revert to the old %s syntax just because it's more tidy than an f-string.

Am I understanding that you’re building JSON with string interpolation? If so any special reason you wouldn’t build a dict and json.dumps() it?

When there is a multiline template, you can make it look natural within

json_doc = '''{}''' % some_dictionary

and then populate json_doc via %(some_dictionary_key)s values inside of the brackets.

I find it more readable.

Furthermore, json_doc can now live elsewhere, if sizeable, and get imported.

I guess that template strings may be the newer way to do this, but this is very backward compatible.

Not really about the content of the blog post but am I the only one bothered by the complete lack of capitalization? Granted it may be because of my screen reader but my TTS engine of choice doesn't pause on un-capitalized words/sentences/phrases/etc. which follow a full stop, so this post unless I read it line by line (minus the code) just blends into complete noise.

Been using python for almost 10 years. I never use any of the funky strings.

Instead of reading like 10 PEPs for f strings, I just use the + operator on strings and backslash escaping, big whoop.

Right, because x + " " + y is more clear than of simply f"{x} {y}".

Been using python for longer than 10 years and immediately started using f-strings when I could. It takes almost no time to understand the basics.

Personally I do find the first version clearer, because it is based on general language rules.

Especially if you come from other languages. a + " " + b will be clearer to devs that aren't senior python developers. the f string thing is just a marginal improvement at the cost of alienating other devs. But it's great for gatekeeping and job security.

> come from other languages

According to Wikipedia[1] the history of string formatting goes back to 1950s. Also, basically every major programming language these days implements it in some form.

[1] https://en.wikipedia.org/wiki/Printf

> senior python developers > great for gatekeeping and job security

Ah, okay, it's just a troll.

But the link you shared describes a syntax that is very different.

"%T", T t

  not "{ (expr).__str__()} "

> Ah, okay, it's just a troll.

If that helps you sleep at night

age = 32

print(f"Age: {age}")

Thus concludes the lecture on f-strings.

I find it funny and satisfying that Python has converged to it's own "printf", by which I mean print(f"").

It’s unclear what the big advantage is over `print("Age: "+age)`. It’s even more characters, in that specific example.

If age is a number, that won’t work.

Admittedly I’m not so familiar with Python; it does work in Java. In that case, introducing a less hazardous string concatenation operator would seem more universally useful.

In Python, that less hazardous string concatenation operator is an f-string.

I’m assuming you can only use it with string literals, hence it is less universally useful.

Nope, you can put expressions in them:

f'{a} = {functionThatReturnsAButHasSideEffects()}'

Toy examples aren’t the use case, rather multiple variables or expressions, perhaps with formatting (padding) as well.

f-string is shorter, less err prone, and faster for medium complexity or above.

  >>> age = 32
  >>> print("Age: " + age)
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  TypeError: can only concatenate str (not "int") to str

  >>> print("Age: " + str(age))
  Age: 32

As a side note, I always found it amusing that Python is dynamic but doesn't let you do this, while C# has static typing but lets you do string + number. (I looked into it a while back, I think it's because both are Object, so it does operator overloading on Object+Object and then checks the types...)

Instead of 10 PEPs, you could read one helpful cheatsheet: https://www.pythonmorsels.com/string-formatting/#cheat-sheet....

Doesn't cover:

- the trailing slash detail detailed in the OP

- raw strings at all

- more complex stuff like raw format strings

Also, my mind is a temple, I don't learn python from cheatsheets from random secondary sources.

Great new way to write blog posts!

Reply on news.ycombinator.com