Hacker News Reader

Comment by Waterluvian | Hacker News Reader

I often read concerns that complexity keeps being added to the language with yet another flavour of string or whatnot. Given that those who author and deliberate on PEPs are, kind of by definition, experts who spend a lot of time with the language, they might struggle to grok the Python experience from the perspective of a novice or beginner. How does the PEP process guard against this bias?

View on HN

There are many long-term users of Python who participate in PEP discussion who argue for beginners[0], often because they professionally are teaching Python.

There are also loads of people basically defaulting to "no" on new features, because they understand that there is a cost of supporting things. I will often disagree about the evaluation of that cost, but it's hard to say there is no cost.

Nobody wants a system that is unusable, slow, hard to implement for, or hard to understand. People sometimes just have different weights on each of these properties. And some people are in a very awkward position of overestimating costs due to overestimating implementation effort. So you end up in discussions like "this is hard to understand!" "No it isn't!"

Hard to move beyond, but the existence of these kinds of conversations serve, in a way, as proof that people aren't jumping on every new feature. Python is still a language that is conservative in what it adds.

This should actually inspire more confidence in people that features added to Python are _useful_, because there are many people who are defaulting to not adding new features. Recent additions to Python speeding up is more an indicator of the process improving and identifying the good stuff rather than a lowering of the bar.

[0]: I often think that these discussions often get fairly intense. Understandability is definitely a core Python value, but I Think sometimes discussions confuse "understandability" with "amount of things in the system". You don't have to fully understand pervasive hashing to understand Python's pervasive value equality semantics! A complex system is needed to support a simple one!

All discussion on PEP's happens in public forums where anyone can opine on things before they are accepted. I agree that the experts are more likely to participate in this exchange. And while this is wish-washy, I feel like the process is really intended to benefit the experts more than the novices anyways.

There have been processes put into place in recent years to try to curb the difficulty of things. One of those is that all new PEPs have to include a "how can you teach this to beginers" section, as seen here on this pep: https://peps.python.org/pep-0750/#how-to-teach-this

I think "how can you teach this to beginners?" is a fantastic, low-hanging fruit option for encouraging the wizards to think about that very important type of user.

Other than a more broad "how is the language as a whole faring?" test, which might be done through surveys or other product-style research, I think this is just plainly a hard problem to approach, just by the nature that it's largely about user experience.

"How does this fit with everything else beginners have to learn to understand basic code?" is sorely needed.

The average Python developer does not even know what a "PEP" is. Open discussion is good yes, but no one really knows what the average developer wants because they simply does not care if its Python or Java or whatever else.

"Some hammers are just shaped weird, oh well, just make do with it."

For example, some people that I interview does not "get" why you have to initialize the dict before doing dict[k] += 1. They know that they have to do some ritual of checking for k in dict and dict[k] = 0. But they don't get that += desugars into dict[k] = dict[k] + 1.

defaultdict is so regularly useful for just this reason!

You might find the Python discussion forums ([0] and [1]) interesting; conversation that guides the evolution of PEPs happens there.

As Nick mentioned, PEP 750 had a long and winding road to its final acceptance; as the process wore on, and the complexities of the earliest cuts of the PEPs were reconsidered, the two converged.

[0] The very first announcement: https://discuss.python.org/t/pep-750-tag-strings-for-writing...

[1] Much later in the PEP process: https://discuss.python.org/t/pep750-template-strings-new-upd...

Building off this question, it's not clear to me why Python should have both t-strings and f-strings. The difference between the two seems like a stumbling block to new programmers, and my "ideal python" would have only one of these mechanisms.

For one thing, `f"something"` is of type `str`; `t"something"` is of type `string.templatelib.Template`. With t-strings, your code can know which parts of the string were dynamically substituted and which were not.

The types aren't so important. __call__ or reference returns type string, an f and a t will be interchangeable from the consumer side.

Example, if you can go through (I'm not sure you can) and trivially replace all your fs with ts, and then have some minor fixups where the final product is used, I don't think a migration from one to the other would be terribly painful. Time-consuming, yes.

Not sure that's true. `Template`s don't provide a `__str__` function, so you need to pass them to a processing function to get a `str` back.

https://peps.python.org/pep-0750/#no-template-str-implementa...

f-strings immediately become a string, and are "invisible" to the runtime from a normal string. t-strings introduce an object so that libraries can do custom logic/formatting on the template strings, such as decided _how_ to format the string.

My main motivation as an author of 501 was to ensure user input is properly escaped when inserting into sql, which you cant enforce with f-strings.

> ensure user input is properly escaped when inserting into sql

I used to wish for that and got it in JS with template strings and libs around it. For what it’s worth (you got a whole PEP done, you have more credibility than I do) I ended up changing my mind, I think it’s a mistake.

It’s _nice_ from a syntax perspective. But it obscures the reality of sql query/parameter segregation, it builds an abstraction on top of sql that’s leaky and doesn’t even look like an abstraction.

And more importantly, it looks _way too close_ to the wrong thing. If the difference between the safe way to do sql and the unsafe way is one character and a non-trivial understanding of string formatting in python… bad things will happen. In a one-person project it’s manageable, in a bigger one where people have different experiences and seniority it will go wrong.

It’s certainly cute. I don’t thing it’s a good thing for sql queries.

I understand your concern, and I think the PEP addresses it. Quite bluntly, t"foo" is not a string, while f"foo" is. You'll get a typecheck error if you run a typechecker like any reasonable developer, and will get a runtime error if you ignore the type mismatch, because t"foo" even lacks a __str__() method.

One statement the PEP could put front and center in the abstract could be "t-strings are not strings".

> "t-strings are not strings"

t-string is an unfortunate name for something that is not a string.

I wish it were called "string templates" instead, with t"whatever" form being called a "template literal".

Simpson's Individual Stringettes!

https://www.youtube.com/watch?v=7qNj-QFZbew

Away with floods! Away with workaday tidal waves!

That all make senses to me. But it definitely won't make sense to my intro to programming students. They already have enough weird syntax to juggle.

Then dont teach them t-strings

Give it a few years to when f-string usage has worn off to the point that a decision can be made to remove it without breaking a significant number of projects in the wild.

That will never happen.

Well if it continues to be popular then that is all good. Just keep it. What matters is that usage isn't complex for anyone.

Well now we'll have four different ways to format strings, since removing old ones is something that doesn't actually happen:

    "foo %s" % "bar"
    "foo {}".format("bar")
    bar = "bar"; f"foo {bar}"
    bar = "bar"; t"foo {bar}" # has extra functionality!

This is where an opinionated linter comes in handy. Ensures people gradually move to the “better” version while not breaking backwards compatibility.

It does suck for beginners who end up having to know about all variations until their usage drops off.

The linter is a big deal, actually. I've worked with Python off and on during the past few decades; I just recently moved onto a project that uses Python with a bunch of linters and autoformatters enabled. I was used to writing my strings ('foo %s % bar), and the precommit linter told me to write f'foo %{bar}'. Easy enough!

And if it's being used, and isn't considered problematic, then it should remain. I've found use for all the current ones: (1) for text that naturally has curlies, (2) for templating (3) for immediate interpolation, and improved at-site readability

I see (4) being about the flexibility of (2) and readability of (3). Maybe it'll eventually grow to dominate one or both, but it's also fine if it doesn't. I don't see (1) going away at all since the curly collision still exists in (4).

printf-style formatting ("foo %s" % "bar") feels the most ready to be retired (except insofar as it probably never will, because it's a nice shortcut).

The other ones at least are based on the same format string syntax.

"foo {}".format("bar") would be an obvious "just use f-string" case, except when the formatting happens far off. But in that case you could "just" use t-strings? Except in cases where you're (for example) reading a format string from a file. Remember, t- and f- strings are syntactic elements, so dynamism prevents usage of it!

So you have the following use cases:

- printf-style formatting: some C-style string formatting is needed

- .format: You can't use an f- string because of non-locality in data to format, and you can't use a t- string due to dynamism in

- f-string: you have the template and the data in the same spot lexicographically, and you just want string concatenation (very common!)

- t-string: you have the template and the data in the same spot lexicogrpahically, but want to use special logic to actually build up your resulting value (which might not even be a string!)

The last two additions being syntax makes it hard to use them to cover all use cases of the first two.

But in a specific use case? It's very likely that there is an exact best answer amongst these 4.

.format is also nice because you can have more complex subexpressions broken over multiple lines instead of having complex expressions inside the {}.

> printf-style formatting ("foo %s" % "bar") feels the most ready to be retired (except insofar as it probably never will, because it's a nice shortcut).

It’s also the only one which is anything near safe for being user provided.

I don’t think I’ve ever used % formatting in Python - what makes it safer than `format`?

`str.format` allows the format string to navigate through indexes, entries, and attributes. If the result of the formatting is echoed back and any non-trivial object it passed in, it allows for all sorts of introspection.

printf-style... does not support any of that. It can only format the objects passed in.

Very good point. While I think we could do away with the syntactic shorthand, definitely would want to keep some function/method around with the capabilities.

Don't forget string.Template:

    import string
    t = string.Template("foo $bar")
    t.substitute(bar="bar")

Five, if you count the log module. I hope t-strings will come there soon.

log.error("foo happend %s", reason)

There is also the template module from the python 2 days.

Putting down my marker on the opposite. Once you're targeting a version of python that has t-strings, decent linters/libraries have an excuse to put almost all uses of f-strings in the ground.

No backward compatibility?!

If the usage of a feature is significantly close enough to 0 because there is a well used alternative, what need is there for backward compatibility? If anything, it can be pushed to a third party package on PyPI.

Reply on news.ycombinator.com