More intuitive partial function application in Python | Hacker News Reader

More intuitive partial function application in Python

github.com

102 pointscgrimm199449 comments

Current solutions:

  >>> list(map(partial(pow, exp=3), range(10)))
  [0, 1, 8, 27, 64, 125, 216, 343, 512, 729]

  >>> list(map(lambda base: pow(base, 3), range(10)))
  [0, 1, 8, 27, 64, 125, 216, 343, 512, 729]

  >>> [pow(base, 3) for base in range(10)]
  [0, 1, 8, 27, 64, 125, 216, 343, 512, 729]

Proposed solution:

  list(map(bp.partial(pow)(bp._, 3), range(10))

ISTM the only time you come out a little ahead is if you know in advance that a function is going to be partialed (so you can decorate it), that you will need to partial arguments in other than a left to right fashion (otherwise a plain partial will suffice), that you won't use keyword arguments (the current partial works with keywords), and that you really dislike the lambda keyword (which neatly covers all cases from simple to complex without a new notation).

That said, this is a clever recipe. Kudos to the author for putting it together. I don't think it really rises to the level of "better" or "more intuitive", but it is an interesting experiment.

I'm glad you like the recipe :-)

What if you have a function "f" that takes like 10 arguments and you need to pass it to a function "g" that expects as input a function takes 8 arguments.

  @partial
  def f(p1, p2, p3, p4, p5, p6, p7, p8, p9, p10):
     return "stuff"
  
  g(f(..., p4=1, p7=2))
  #  seems nicer than
  g(lambda p1, p2, p3, p5, p6, p8, p9, p10: f(p1, p2, p3, 1, p5, p6, 2, p8, p9, p10))

First mistake, making a function with 10 parameters.

We have data structures for that, if you need it. Just pass in a big struct/named tuple as one parameter. Bonus points if it is type hinted.

Nit: List comprehension looks better for expressions (preferable over lambda+map)

Great to see this! Thanks for working on it.

Could you compare it to functools.partial?

In the example on the Readme:

  import better_partial as bp
  
  @bp.partial
  def some_operation(x, p1, p2):
    return (x + p1) * p2

It claims that the bp approach is superior because:

  func = some_operation(bp._, 10, 20)

...is better than:

  func = lambda x: some_operation(x, 10, 20)

But is it better than just:

  partial(some_operation, p1=10, p2=20)

?

  partial(some_operation, p1=10, p2=20)

Suffers from inconsistent syntax compared to normal function call. This is probably a negative, but it could be a positive.

Yes! I definitely need to beef up the README. I wrote up a gist for some other commenters below which I can incorporate as part of the justification.

https://gist.github.com/chrisgrimm/64bca66f14528cfda6d865cc2...

I was gonna say cool for FP guys but alien to the Python world, but then i saw the f(..., x=1) syntax and decided you're a genius. Really nice idea.

In practice i worry about static checkers and IDE param hints though, how have you fared with these?

Not your fault at all, but mypy/pyright have very poor support for wrappers like this, and in IDEs it's common for their arg hints to devolve to a copout (args, kwargs) as well.

> Not your fault at all, but mypy/pyright have very poor support for wrappers like this, and in IDEs it's common for their arg hints to devolve to a copout (args, kwargs) as well.

You might be able to annotate it accurately with generics, ParamSpec and Concatenate[1].

[1] https://www.python.org/dev/peps/pep-0612/

I'll give this a look.

High praise :-) I'm glad you like it! I don't know much about static checks / param hinting. TBH, I coded this up because I wanted to imagine how functions should work in Python as more of a fun exercise.

Accordingly there are a lot of things I have to do to make better_partial a strong tool for the community. I'll probably have time to work on the project again later in the week.

Hey, I use functools.partial a lot in my projects and I've always felt like it was pretty limited in the types of partial function application it could accomplish. I put together a modified version of partial over the weekend that I wanted to share / get feedback on. Let me know what you all think!

`functools.partial` has been bothering me for a long time and I've always thought it'd be nice to have something like the placeholder `·` that people use in math (i.e. `f(·, y, z)`). It didn't occur to me that a decorator might actually get us there without having to introduce new syntax. Nice project!

What specifically bothers you about functools partial? I think the decorator is nice in certain ways. However, then you have to decorate your code and guess where you’re going toy need the partials or create wrappers just to add the decorator which seems undesirable for a number of cases. I agree that partial has warts when it comes to inspection but at least they’re easy to make anywhere you want without having to apply any changes to the callee. Nice to have another option for partials though so that when you need these features there’s a way to get them.

I'm glad you like it! This project was basically me trying to visualize how I wish functions generally worked like in Python.

https://toolz.readthedocs.io/en/latest/curry.html

Can these be serialized? One of my main uses for partial is passing things to multiprocessing pool.map. People often reach for lambda but it cannot be pickled/serialized so it fails.

I love the idea of using underscore to represent partial currying.

Somewhat related, here's a similar PR for native support of this feature in Julia: https://github.com/JuliaLang/julia/pull/24990.

In my opinion, I'd love to see this baked in as part of the language. Getting buy in from my team to use this in production is just not going to happen. But cool package though!

I'm sure it's nice, but adding another dependency to save some keystrokes in an edge case --- no thanks. If I want lots of partial application I'll use Haskell.

Also thought the benefit too small for the drag of another dependency.

Could this be used without the "bp."? Half the point of coming up with a nicer notation is for the notation to actually be nicer.

Well, you could do

from better_partial import partial, _

presumably. Though since _ has some meaning in Python maybe you'd want to do

from better_partial import partial, _ as x

instead (or something like it).

As pointed out, you could import _ directly into your namespace, or map it to some other shorthand name that doesn't interfere with your naming conventions.

Also note that the _ placeholder isn't the only way that better_partial helps you. You can also use ... to indicate that you want to omit all arguments except ones you explicitly pass as kwargs. For instance:

  from better_partial import partial, _

  @partial
  def f(a,b,c,d,e):
    return (a,b,c,d,e)
  
  f(_,_,3,_,_)(1,2,4,5) == f(..., c=3)(1,2,4,5)

This is not a new idea. siuba (a dplyr-like Pandas alternative) is one project that does it, and it also has a "pipeline operator": https://github.com/machow/siuba

(mtcars >> group_by(_.cyl) >> summarize(avg_hp = _.hp.mean()) )

(Edit: ok, it seems siuba doesn't have the "parameter fixing" thing.)

Related: Coconut - a functional superset of Python - https://github.com/evhub/coconut

range(10) |> map$(pow$(?, 2)) |> list

There were others but I can't remember which now.

Interesting, but I'm wondering how does partial application for external functions works. Do you need to redefine the functions using the decorator?

I like how clever this is. It's a little too clever for production code for my tastes, but it might be nice to use in exploratory programming.

I really appreciate that :) Definitely give it a spin and let me know what you think!

This is great. I will use it with my python chaining approach: https://github.com/tpapastylianou/chain-ops-python

My only grief is that decorator, which forces you to wrap existing functions anyway (same way I had to define lambdas in my example anyway).

Do you have any insight on that?

While it does solve the problem with functools.partial() forcing you to use keyword arguments in some cases, I don't think currying is a common enough operation to warrant a whole lib, or even a new syntax.

I'd rather see partial() promoted to a universal static method attached to any callable, so it's easier to discover, and use.

The work you've done is nice, but using a package for this issue gives off bad code smell. I'd much rather refine my functions and their relations rather than require an extra dependency that reduces readability for my coworkers.

> but under the hood my partial decorator is working all kinds of magic

I'm sure it is! That's why I'd be leery of using it in production code.

whats wrong with functools.partial? I don't see that anywhere in the arguments.. all you say is "I find functools.partial unintuitive".. why? its a simple wrapper.

here's an example of one of the behaviors of functools.partial that I find unintuitive / undesirable:

https://gist.github.com/chrisgrimm/64bca66f14528cfda6d865cc2...

Reply on news.ycombinator.com