Positional-only Parameters for Python
lwn.net
lwn.net
I couldn't agree more with this sentiment from Raymond Hettinger. I first learned Python starting with 1.5.2, and coming from a background of C and Perl. One of the things that really attracted me to the language was how compact it was. It took no more than an hour to learn most of the language, a day to read the entire spec, and maybe a second day to read the entire standard library documentation. It suffered from virtually no gotchas... nothing crazy like Perl's scalar vs list context, etc.
I haven't had trouble keeping pace with Python and I even probably agree with most of its changes. But this change really seems superfluous. How does it fit into the zen?
Can a language ever reach a point of being done?
>>> from pathlib import Path
>>> Path("foo/bar") / "baz"
PosixPath('foo/bar/baz')
Of course, after my initial disgust, I now use it everywhere :)
In order to achieve OOW, you need really high-level features (assembly language programmers have every possible way to accomplish a task!), but also a very limited set of them (so nobody can pick the 'wrong' one). But as a general-purpose programming language, users also need power and flexibility.
It's an unstable tripod where as soon as one starts getting higher, you mess up the others. I can tell they've been trying to slowly raise all three (and deprecate/remove old and lower-level pieces from the bottom), but it's a delicate balancing act, and it's easy to run into other constraints, like complexity.
p1 = "/a/b/c"
x = "d"
p1 + "/" + x == "/a/b/c/d"
p2 = Path("/a/b/c")
# If + replaced / for Path:
p2 + "/" + x == Path("/d")
But either way, I can’t see the problem with python defining the exact same operator to mean the exact same thing as you’d expect in that specific scenario.
"a" + "b" == "ab"
"b" + "a" == "ba"
Not sure exactly what you're saying there? Haskell strikes me as far more likely to define an operator for that.
Although Haskell will define something other than /, which unless you jump through some hoops is going to be the fractional division operator [1]. You'd have to be crazy to try to give a "path" an instance of Fractional, though I'd be intrigued to hear someone's definition of the "reciprocal" of a path.
[1]: https://hackage.haskell.org/package/base-4.12.0.0/docs/Prelu...
..The closest you could come is an anti-path, where if you are at a/b/c, you could have c'/b'/a' that when combined together produced the root path of both these things. You could then have things like b'/a, which represents "rise up to the previous directory, which is named 'b', and descend into a" where the reciprocal is a'/b. I'm done fiddling with this for now, but it's possible you could work this into a sensible scheme with some concept of "reciprocal". It's useless in practice because who wants a .. operator that asserts the name of the directory it is rising from? (And presumably, what, errors if it isn't right? Refuses to rise like .. does at the root?)
However since Fractional instances also have to be Num instances [1], and there's no practical way to fill in most of those, it's not going to get you very far. And the Fractional requirement of fromRational, while you can try to write something that turns a Rational (which in Haskell is specifically two Integers being used as a ratio) into a path, it's going to be very silly.
[1]: https://hackage.haskell.org/package/base-4.12.0.0/docs/Prelu...
at least
Maybe the answer is that none of us work in all the domains where Python is used, so we are all speaking with ignorance/elitism when we decide which list of features are superfluous.
Explicit is better than implicit, Simple is better than complex, Readability counts, I'm sure others would match.
Positional-only parameters are useful in general and even necessary to implement / override APIs originally defined in / for C (which has always had access to positional-only parameters). Currently, defining an optional positional-only parameter is:
def foo(*args):
if len(args) > 1:
raise TypeError(f"Expected at most 1 argument, got {len(args)}")
if args:
arg = args[0]
[…]
* it requires various non-trivial assertions repeated every time
By contrast, with PEP 570 this becomes:
def foo(arg=None, /):
[…]
*args, **kwargs
Also, I mean, come on, if a function signature is:
ord(c)
If you need to rename a parameter that it's likely someone might be calling as a keyword, then bite the bullet and break backwards compatibility or just introduce a new function name if backwards compat is truly that important. (And besides, this has to be one of the more trivial checks for a static analyzer to find.)
> the number (and optionality) of positional parameters is completely invisible and implicit rather than readable and explicit
It's invisible in the function signature, but it should be in the function documentation or as a last resort, read the source. I find myself reading Python library source code not infrequently because it's not unusual to need to understand a function's behavior. Getting the function call correct is like the least hard part? But again, this really feels like an edge case.
> it requires various non-trivial assertions repeated every time.
In fact, that's a pretty trivial assertion if you ask me. ¯\_(ツ)_/¯.
FWIW, I don't find:
(arg=None, /)
def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):
def f(pos1, pos2, /, *, kwd1, kwd2):
Maybe I'd be more convinced with a cleaner syntax. I think the Per-Argument Marker option proposed in the PEP is a lot more obvious (not sure about using a '.' though) and think the argument against it is pretty weak: "While this approach may be easier to read, it has been rejected because / as an explicit marker is congruent with ∗∗ for keyword-only arguments and is less error-prone."
I note that "less error-prone" is stated without evidence and that "is congruent with ∗∗" is definitely against the zen.
But generally, as I said, the entire PEP is not convincing to me. Anyway, I'm sure I'm repeating the arguments against the PEP others have made and Guido has ruled in favor anyway, so I'm just screaming at the ocean.
Taking that a bit further and ignoring backwards compatibility entirely:
def foo([po1, po2], pk1, pk2, *args, {kw1, kw2, kwd1=default, kwd2=default, **kw}):
pass
def all_pos_only[po1, po2, po3=default, *args]:
pass
def all_kw_only{kw1, kw2, kw3=default, **kw}:
pass foo([po1, po2], pk1, pk2, *args, {kw1, kw2, kwd1=default, kwd2=default, **kw})
Now yes, I realize an author could already do that with ∗args and ∗∗kwds, but the fact that the author would need to handle the argument parsing is sufficient disincentive to prevent most people from doing so.
And generally, I think the disincentive rationale is self-defeating:
* the premise behind disincentive is you're dealing with someone who is (at least in the moment) careless
* now they're going to have to write more code to do a thing they've set out to do
* but we've already selected for an individual who is writing carelessly
In this case, it can only work if extra argument checks are the last straw that causes this individual to say "enough" and refactor that function. For what my experience is worth, I think this is unlikely.
I really like the possibility of optionally calling a positional-argument function with named keyword arguments, which can be more legible in some calling contexts. I would argue that in most cases most function arguments should be of the pos_or_kwd type. YMMV.
The reason Python added keyword-only arguments was so that library authors could freely reorder them (e.g. add a new keyword argument before another, or add another positional argument before keyword arguments) without breaking backwards compatibility.
The only reason to have positional-only arguments is to allow libraries to rename their arguments (without changing the semantic meaning) and guarantee that calling code doesn’t break. This doesn’t seem like a very strong need.
Oh, that’s definitely the case. My point was: why would you have a function that had three different kinds of params: pos only, keyword only, and pos_or_kwd. That would be insane. You probably wouldn’t. That it’s allowed (after this PEP) is an consequence of this change which is just more evidence to me that the PEP isn’t needed.
Seeing "foo(args)" as an autocomplete necessitates looking at the docs to find out what "args" gets used for.
TeX? Yes, I know there are still new releases occasionally, but as far as I know, it's feature-complete, and only occasionally gets bug fixes. I wish the developers of more languages, and software in general, would recognize when their projects were done. Unfortunately, that doesn't seem to be the way things are headed.
And this type of discussion makes me think about Django - it used to be a nice little simple framework, then it got a big community around it and grew every nature of features to such a point that I think nobody should actually use it. The code ends up being pretty awful and I think probably the rise of Flask coincides with Django getting too big and too complex over a period of years.
I am deeply unconvinced the proliferation of features is even remotely necessary or helpful.
But then, I admit I've become deeply skeptical of the whole "language as a proliferation of features" model of language development in general. Python is just a particularly frustrating example because it's a clear case of a useful simple language becoming a huge, complex language that isn't really that much more useful. I'm coming around to what doesn't go into a language being more important, because there's always another feature the language needs and always some reason to put it in. If you don't have a clear thesis on why and how you're going to reject things, you won't often enough.
The problem I had was that I had to explain "oh, well, this is a generator because of the yield, and the arguments don't show up in the documentation for the function because it's introspecting on the keywords you pass in dynamically, and this here is a generator comprehension with its own syntax, and this is what the decorator on top is doing on this code, which is why this is breaking" and honestly it kept going like that for a while. When it all worked it was fine, but when anything broke, it's become far too easy in Python, and arguably idiomatic Python, to erect a rather intimidating wall of features and automatic rewirings and fancy decorators and suddenly to fix their problem the novice is swallowing all of twenty years of language development in one shot... or they give up and do something else.
I don't have a current go-to recommendation, though I've found myself recommending "Go but stay away from any concurrency features for a good long time" a few times. Ignoring all the other controversies that HN likes to indulge in, that's not a half-bad intro to "real programming" for a modern language. It's not the only good choice, of course.
Your idea of a beginner is someone in a mature environment though, I guess my thought when I read your previous comment was a total beginner starting from a blank slate.
trust me, if you have no apriori regarding what a programming language is, perl will make a lot of sense
I may have the order a bit mixed up. I started with perl 4 and got to be reasonably adept, using it as my primary language from maybe 96-2000. I've used Python as my primary language since 2000. I'm pretty fluent in C, Java, Objective C and shell.
Anyway, I had to fix a couple hundred line Perl script the other day and it had me scratching my head for long enough that I was pretty close to rewriting the whole thing in Python. One thing I really love about Python is that it's trivial to test bits and pieces of code via the REPL.
One important use case for the existing facilities is to take a function whose signature used to contain positionals, and migrate a positional to be a keyword argument without breaking existing uses of the API.
I think `/` is a poor choice of symbol, too. If such a separator were to be a good idea, it should be * * by analogy with `*` for separating keyword-only args.
For example, concatenation, mathematical operations.
Keyword arguments are also more redundant (a function taking a single argument usually does not need that argument to be named, quite the opposite, even for some multi-parameter functions it makes no sense e.g. `max(a=1, b=2)` is downright no matter how you name the arguments)), and the lack of positional-only parameters means any old name to what was intended as a positional effectively becomes part of the API.
I think Swift has the better pick there, an argument can be either positional or named, not both, and positional-only parameters are used to great effect. Sadly Python's history of positional-or-named can't really be fixed.
Would you rather have:
thread_pool.shutdown(True)
or
thread_pool.shutdown(wait=True)
Even with single argument, it's not always clear what the argument means from the function name, so being explicit makes the code more readable.
EDIT: I realize you said "usually", but my point still stands.
wait = True
thread_pool.shutdown(wait)
The only argument I'm even remotely sympathetic to is "If a function uses a name for a parameter, that precludes callers from using it as a keyword argument elsewhere in the argument list." But the workaround for that rare case is to use:
*args, **kwargsGuido's objection that "writing len(obj=configurations) is not something we want to encourage" makes not one iota of sense. Allowing it is not encouraging it.
There's mention of one bug which allegedly could have been fixed with positional-only parameters:
https://bugs.python.org/issue9137
The bug here is that dicts have an update method which you can call two ways, by passing a single dict, or with keyword arguments, and if you try to call the single-dict version using a named parameter, it's not clear what should happen, and at one point, for OrderedDict, it could end up throwing an exception.
But the mistake here isn't confusing parameters, it's having a single method that you can call in two different ways. That sort of thing might fly in Perl, but it's a source of ambiguity and confusion that, IMHO, is not up to Python's standards.
Also, I don't like the idea that package developers' opinions on what's "readable code" outrank mine, when we're talking about my code.
You can always opt for dict args if you want to change a signature and maintain backwards compatibility with obsolescent arg names.
If I was BDFL this would be a dead issue.
In general, though, I don't see the issue with always needing to pick good names. It's very simple to use single-character names if one needs to, such as for a mathematical function, or simply a procedure where the letters have clear meanings or will likely never be used to start with.
Asides from the design features I thought were jokes when I first learned of them, such as one-line lambdas and whatnot, this ''beginner's language'' certainly has a great deal of complicated syntax sugar and other things being added to it.
sub fun($a, $b, :$c) { }
fun(1, 2) # This works!
fun(1, 2, 3) # Error: Too many positionals passed...
fun(:a(1), :b(2), :c(3)) # Error: Too few positionals passed...
fun(:c(3), :a(1), :b(2)) # Same as before
fun(:c(3), 1, 2) # This works!
sub fun($b, $a?, :$c) { }
fun(1) # This works!
sub fun($a, $b, :$c!) { }
fun(1, 2) # Error: Required named parameter 'c' not passed
fun(1, 2, :c(3)) # This works!
sub fun($a, $b, A@rest) { }
Slurpy named parameters are indicated also with `A` followed by a hash parameter:
sub fun(:$c, :$d, A%rest) { }
sub fun($a, $b, :$c, :$d, A@rest, A%rest) { } # Or:
sub fun($a, $b, A@rest, :$c, :$d, A%rest) { }
def fun(parameter1, parameter2, parameter3):
a = parameter1
b = parameter2
c = parameter3
... (rest of function)
Yes and no: the syntax was used in documentation (that's the "argument clinic" mentioned in the article) and Python functions defined in C were able to define positional-only parameters, but it's a new thing for Python code: before 3.8, it's not possible to define positional-only parameters in pure Python[0], with it the "argument clinic" syntax becomes actual Python syntax (rather than pseudo-python).
[0] well you can use *args and a bunch of hand-rolled assertions
def show((a, b, c)):
print 'a = %s, b = %s, c = %s' % (a, b, c)
show((1, 2, 3))
show((a=1, b=2, c=3)) $ python2
>>> def f((a, b)):
... return a + b
...·
>>> f((1, 2))
3
$ python3
>>> def f((a, b)):
File "<stdin>", line 1
def f((a, b)):
^
SyntaxError: invalid syntax