Hacker News Reader

Comment by scrollaway | Hacker News Reader

String interpolation is one of those features where I'm really confused about the direction the Python team wants to take. It's implicit and magic, and yet another non-obvious way to do do string formatting.

What's more, it's being hailed at a plus for localization which it isn't. Localizers should never, ever deal with string interpolation - anything past what .format() does is essentially untranslatable.

View on HN

Why is it implicit and magic? Looking at this post, and the PEP, it seems like interpolation is actually pretty close to .format in semantics, but syntactically simpler. There are a few bits that I didn't quite follow in the PEP, so I may just be missing the issue, however.

The important thing for localization is to allow the strings to be swapped at runtime by loading them from some kind of database in a way where the arguments are at least numbered, if not named, so as to let them appear in different orders for different languages. The interpolation syntax does not seem, to me, allow for this: it is an "implicit" syntax offered during parsing to take a constant string and immediately build it out of parts, as opposed to the "explicit" % operator or .format methods: it is extremely clear with these two models how one would load a template that has the strings in a different order, or which used a different subset of the strings.

    >>> '%(one)s %(two)s'%{"one":"hello", "two":"world"}
    'hello world'
    >>> '{one} {two}'.format(one="hello", two="world")
    'hello world'

The PEP that is looking at this problem has been deferred: at the end of the document they kind of indicate that they didn't really think about the i18n problem correctly and so don't actually have a good solution to present, and are going back to think about it more.

https://www.python.org/dev/peps/pep-0501/.

If you need to localize, you still have format(). You just happend to have a shorthand to do it for the 80% case, just like you have @decorator as a shortcut for decorated = decorator(decorated) or [x for x in z] as a classic loop alternative.

Yes, having so many solutions is not ideal, espacially when it comes to teach the language. However, it would be foolish to avoid improving the usability of Python just to avoir having "one more way to do it".

I do wish they'd deprecate Template though. It's more than useless.

string.Template is the one used for i18n, it is simple for the end user so has a use.

I suppose they need to add an API so that f-strings can be automatically presented to gettext or another translation database layer before substitution is performed.

It isn't for the most part, the GP is incorrect, except for the confusion.

Being a die-hard Python user on a daily basis, I concur. I think these core devs should not invent too many new syntax and ways to solve problems which don't need to be added to the language.

I am not sure why i18n is a big deal, let another library deal with it.

Whatever, I have the choice not using this feature after it is accepted and implemented. Their PEP discussion on email always ended up in tangential. Problem I always have with string manipulation is dealing with long string, which for coding style I'd split into multiple +, and thus using format is pretty ugly.

You don't have to use + to join strings on multiple lines. Here:

    >>> (" foo "
    ...  "bar.")
    ' foo bar.'

The other day I spent some time tracking down a bug where someone forgot a trailing comma in a multi-line parameter list.

That was annoying.

Ok, but this only works for literals.

Yes, which is what I'm replying to.

I agree with you. It is already possible to use quote and plus in the same way as the bracket in the new way, with the added advantage that the original syntax is more orthogonal.

Is it really worth updating all the Python syntax formatting and analysis code out there just to save one character on an operator? I don't think it's a good tradeoff.

+ is a terrible operator for formating.

You got:

    price = 18.8
    date = datetime.datetime.now()

Now you want to save in a file:

    "It's 18.80 and we are the 01/04/2016"

With +:

    num, dec = str(price).split('.')
    msg = ("It's "+ num + "." + dec.ljust(2, '0') + 
          " and we are the " + date.strftime("%m/%d/%Y") + "\n")
    f.write(msg)

It is so long and ugly we have to break it on several lines to respect PEP8.

And it's a pain to write, or even worst, to read.

With print():

    num, dec = str(price).split('.')
    print("It's", num, ".", dec.ljust(2, '0'), 
          " and we are the ", date.strftime("%m/%d/%Y")), file=f)

A tiny bit better since we don't have to use that many concatenation tricks.

If we didn't have to format the float, print() would have converted it to string for us which is nice (with + you have to call str() all the time).

With "%":

    f.write("It's %.2f and we are the %s\n" % (price, date.strftime("%m/%d/%Y")))

Way better, but still hard to read.

With format():

    f.write("It's {:.2f} and we are the {:%m/%d/%Y}\n".format(price, date))

Now we are getting sometwhere...

With fstrings:

    f.write("It's {price:.2f} and we are the {date:%m/%d/%Y}\n")

fstrings have everything:

    - easier / faster to type than any other versions;
    - easier to read than any other versions;
    - as expressive as any other versions.

It's reduce work by making you think less, type faster, read faster (for you and your colleague) and it's easier to spot bugs.

The only place where you don't want fstring are for l10n.

I don't think it's even that bad for l10n, as you can treat the expression-like format specifiers as opaque keywords for the placeholders.

For example, your i18n tool can extract:

    "It's {price:.2f} and we are the {date:%m/%d/%Y}\n"

as a key, and then l10n can map this to:

    fr_FR: "C'est {price:.2f} et nous sommes le {date:%d/%m/%Y}\n"

(Here we can fix up the silly US date formatting, too, although really you shouldn't be localising dates in your format strings).

All that remains is for them to work out some way of mapping in that localised string, as you can do this:

    _("It's {price:.2f} and we are the {date:%m/%d/%Y}\n").format(price=price, date=date)

any more.

> (Here we can fix up the silly US date formatting, too, although really you shouldn't be localising dates in your format strings).

ISO 8601 (similar to Japanese format) is the most logical way to format dates: YYYY-MM-DD, easily sortable, not the silly way US and Europeans format their dates. ;)

https://xkcd.com/1179/

Date formatting preferences are done at the desktop level and apps should query for it. All frameworks I can think of have a way of doing so, and you can just tell them "I want a long-form date & time", "I want a short date", "I want a short non-numeric date".

If you are a fan of ISO8601 (which I am), you can then set this globally on your desktop, rather than expect the app developers / translators to choose for you.

Someone made a good argument that in order to support lots of languages, you actually need general functions not just strings to be put into .format() or interpolation.

That's because some languages have complicated changes in the text depending on eg the number of things: not just singular/plural, but more complicated. Russian is one example.

That's a little misleading.

In practice, what you need is a notion of a "context" for your formatting (often something that is aware of both the current i18n and l10n factors), and that context needs access to the fundamental objects that would be rendered in to a string.

Effectively, the "format string" becomes an identifier for the particular message you want to render, though often the objects themselves are definitive enough.

In that paradigm, interpolation/formatting/whatever might be the default mechanism employed by the context, but you don't want to have that as your explicit mechanism. You want one more level of indirection before you get to it.

Some platforms make the mistake of having that context be tied to the thread, out worse still, a global, and that falls apart once you have any multiplexing logic (and the way python currently works with posix locales fully captures how terribly you can do this). Either way, I'd argue out really is a different problem from interpolation, and if you are trying to solve it using interpolation, you don't understand the problem.

Can you clarify the last point? I do not get it. Can you make an example of where an extra indirect mechanism (besides interpolation) is _required_ for translation?

I generally localize for western languages and write in many programming languages. The combination of gettext + python format strings has been working really great for me, and generally much better than other systems I've seen and put to use. In fact, the simplicity of gettext provides a very fast turn-around, and with translators experienced with the tool I never had problems. Python format strings also work great in this context, as I can supply an arbitrary dictionary of elements that the translator might need. The only real problem has been plural forms in complex text strings, where ngettext is not always sufficient.

What method (name a project I can inspect) do you recommend as a good localization architecture?

You're looking for something like:

     l8n_context.format('file_not_found', file)

'file_not_found' is just an identifier to lookup the actual format template (likely a format string) that will be combined with the file object to render the error message.

There's not much difference. In fact, you cannot expect translators to know how the underlying object is handled or write data extractor out of the file object.

If translators are cooperating with you, it's very easy to provide the needed elements directly in the format's dictionary (that is: you extract the translatable pieces for them). It also means you don't have to worry that they're going to fiddle with mutable state.

I generally write everything in english, and do back-translation to my own locale (I also cooperate for translating external projects into my locale), so I eat my own dogfood here.

I know I do not want to deal with extra lower-level subtleties here. Translation is hard already by itself. It's impressive how a good translation of a simple UI can take so much time. If I had to inspect the object to know what I can get out of it I would get crazy.

I'd take a pre-baked dictionary any time.

I've also already used the string-catalog approach in the past (heh, XUL), and I'd personally take gettext any day.

But do you see how that extra level of indirection basically maks arguments about the different formatting styles for localization kind of irrelevant?

I do, yes. But the translator needs to be aware of the extra indirection to produce the text he needs, and between a custom layer and a standard formatting syntax, the second is definitely friendlier for anyone approaching translation, even when technically less powerful.

At some point the translator will have to format some string himself.

There's no reason why your "ID" can't be the template for your default language.

OK. How is that different from using functions? Conceptually, there's a function that does arbitrary computation under the hood.

(I think we are already agreeing, only that you express your point differently in Python specific terms. I don't care whether you stuff your code into a context object or something else. I was talking about having the full power of the programming language available, vs using a limited language like format strings.)

The important part is you have one more level of indirection before you figure out the layout of the message.

I'm a bit confused about all the doubt on such a common feature industry wide. No, it's not magic, it's a simple transformation, takes about 30 seconds to learn, and is becoming an industry standard among modern languages. See C#, Scala, JS, Swift, etc, not to mention bash and perl.

It also has little in common with i18n, the use cases differ too much. Perhaps in the future someone can figure out how to bring them together, but not today.

All of the languages you list have been criticized for adding too many language features.

Go and Rust are two notable new languages which require explicit formatting functions as opposed to relying on special string interpolation syntax.

Well, Rust formatting is magical, unlike that of Go: it has to be because it has type-safe format strings, which can't be expressed in the normal language. But the magic is encapsulated into a macro, so the complexity doesn't leak into the language per se. Someone could write a similar string interpolation macro and replicate Python's feature without modifying the compiler if they wanted--though the fact that nobody has so far indicates to me that it probably isn't needed, as the {} syntax is awfully lightweight already.

Interpolation is one of the features I require in a rapid prototyping language, it is useful every single day and not esoteric.

Python was and still is to some diminishing degree influenced by C, C++, and Java - none of which have language syntax for interpolating strings based on variables in scope.

I wouldn't say it was influenced by Java, seeing as all the core syntax (functions, classes, modules, etc) was already present when Java was first released publicly. The stuff added later doesn't come from it either (list comprehensions, generators, etc).

Yes, though it has more in common with Perl and Ruby, which do.

Reply on news.ycombinator.com