Show HN: Python library for functional programming
github.com
github.com
> # or if you don't like backslash continuation
got me thinking. I've always felt backslash continuation is a Python wart - but using brackets in this way only feels like a minor improvement.
It's nearly always this scenario that makes me want to break a line - i.e. splitting a long chained method call on a "."
Now - starting a line of code with a "." is not valid Python as far as I'm aware and therefore allowing it would not introduce any ambiguity or significantly complicate Python's parsing rules. "If a line begins with whitespace followed by "." then consider it as a continuation of the previous line.
Can anyone comment on whether this is a terrible idea or not?
I'm not up to snuff on my compiler theory, but I think at the least this would require upgrading to a two-token lookahead parser, and it might also upgrade the python grammar from context-free to context-sensitive. Accordingly, I wouldn't expect this to get implemented -- IIRC ease of parsing was/is one of the big design goals.
Absolutely agree. I really like the multi character piping at the end of lines that would get rid of the .
R has %>%
F# uses |> and for functions >>
I would love to see R and all languages just pick up and use F# |>
In [1]: class Foo:
...: def __rmul__(self, other):
...: print("%s foos!" % other)
...:
In [2]: .42 * Foo()
0.42 foos!PS: I'm not aware of any syntax changes in this area since Python 2.7, so I could be wrong.
It’s a great language for most programmers to get things done in, but it belongs to the school of ‘limited abstraction is for your own good’. For example, no multi line lambda expressions. The argument given is that it would mess up whitespace syntax. Haskell handles this situation, but it does indeed introduce a bit more syntax.
If you need to FP in a general purpose scripting language, I would be looking at Elixir or Javascript right now.
So this will soon be a stable API? Great work. The documentation is extensive.
I am personally a big fan of toolz[0], which is clojure inspired. This object method chaining Scala inspired approach is different, and interesting.
It might be fun to make sort of comparison matrix of the difference in approach between this library, toolz, and the venerable fn[1] which also has a Scala influence (and apparently already implements the '_' lambda form).
> There are theoretical and practical advantages to the functional style:
> - Formal provability
This is bollocks. To prove things about your programs, you need a formal semantics for the language you're using.
How precisely does modularity, composability and ease of debugging improve from this module? Not at all. It may have benefits, sure. But to claim these as general attributes is just sophistry in my personal opinion.
---
@lmm
> In practice no practical language provides formal semantics for the entire language
Programming languages are mathematical objects and always have a formal semantics, regardless of whether someone has taken the trouble to write it down or not.
What I'm saying is - the formal semantics that most languages have preclude the existence of compound values. Without those, you can't define interesting functions, and thus you can't program in a functional style.
> Programming languages are mathematical objects and always have a formal semantics, regardless of whether someone has taken the trouble to write it down or not.
Not true for the usual plain-english meaning of "programming language". Plenty of things we call programming languages take a "the implementation is the spec" view, or simply can't be assigned any useful semantics in a way that's at all aligned with the defining characteristics of the language in question. Even in, say, Haskell, io explicitly lacks formal semantics.
> What I'm saying is - the formal semantics that most languages have preclude the existence of compound values. Without those, you can't define interesting functions, and thus you can't program in a functional style.
Python tuples are perfectly cromulent compound values. Python has a bunch of builtin classes for which the equality operators that are available in the language are all kind of useless, sure, but that's true in almost all languages.
And do you think the implementation isn't a mathematical object itself?
> or simply can't be assigned any useful semantics in a way that's at all aligned with the defining characteristics of the language in question.
This is a contradiction in terms. By definition a semantics is a mathematical descrpition of the meaning of a programming language, i.e., the behavior of its programs. Sometimes a semantics exposes significant differences between the mental model of programmers and what a programming language really is, but the solution is not to turn a blind eye to reality.
> Python tuples are perfectly cromulent compound values.
So long as they have object identities, they are not values. Values are merely represented in memory - they truly exist in the language's semantics.
Only if you're taking the Tengmark "mathematical universe" view. The implementation is not necessarily any more mathematical than, say, a table.
> This is a contradiction in terms. By definition a semantics is a mathematical descrpition of the meaning of a programming language, i.e., the behavior of its programs. Sometimes a semantics exposes significant differences between the mental model of programmers and what a programming language really is, but the solution is not to turn a blind eye to reality.
The meaning expressed in a language is some alignment of what the writer intended and what the reader understood. If different readers don't reach a shared understanding then there is no meaning. It would be desirable for our programs to always have meanings that aligned with their behaviour, but we shouldn't turn a blind eye to the fact that many programs don't.
> So long as they have object identities, they are not values. Values are merely represented in memory - they truly exist in the language's semantics.
Any practical programming language will give you some way to find the memory address of a particular value, so values that we usually consider as semantically identical are not actually indistinguishable in practice. We deal with this by only assigning semantics to a subset of the language that excludes those operations. Python without "is" may be more obviously a subset than, say, Haskell without unsafeCoerce, but both are subsets.
The meaning of a program is its behavior, period. If reality doesn't agree with your views, it's your fault, not reality's.
> Any practical programming language will give you some way to find the memory address of a particular value, so values that we usually consider as semantically identical are not actually indistinguishable in practice.
Standard ML doesn't give you any way to get the memory address of a particular value, so values that are semantically identical are actually indistinguishable in practice.
> We deal with this by only assigning semantics to a subset of the language that excludes those operations.
That doesn't make sense. Semantics is a property of the real programming language you are using, not the fictitious one you wish you were using.
That's not the usual meaning of, well, meaning.
> Standard ML doesn't give you any way to get the memory address of a particular value, so values that are semantically identical are actually indistinguishable in practice.
Just tried it to be sure:
fun address i = Unsafe.cast(ref i) : Int32.int ;;
val a = 4 :: nil ;;
val b = 4 :: nil ;;
address (a) ;;
address (b) ;;
address (a) ;;
val address = fn : 'a -> Int32.int
val a = [4] : int list
val b = [4] : int list
val it = ~175003712 : Int32.int
val it = ~174998232 : Int32.int
val it = ~175003712 : Int32.intI hadn't considered doing a comparison matrix, but I haven't done a '_' lambda since it seems redundent with the one in fn.
I'm personally using Toolz so far, probably because it's the most popular of the maintained ones.
Would be happy to see both some standardisation (a PEP?) and consolidation between the various projects.
My big issue with method chaining is exactly the one of requiring the line-continuation-escape, or enclosing the whole thing in parens. You also need to wrap the initial value, then possibly unwrap it at the end, and you're limited to the methods you define (mind you, this seems like a pretty comprehensive list).
For this reason I decided to emulate clojure's threading macros instead, as a pair of functions:
pipe_{first,last}(
init_val,
some_unary_callable, # so far so familiar
(some_n_ary_callable, *args), # puts the piped value first or last, depending on {first/last}
...
)
With respect to _, this would make these two equivalent: a = lambda x: x + 1 b = _ + 1 a(1) == b(1)
Is there an expanded example I could look at for the pipe stuff, I haven't used clojure before
(f(v) for v in (g(u) for u in (h(w) for w in x)))
But like I said, I'm no FP expert. My objections are basically aesthetic. Maybe if I used it, I'd learn to like it.
It is one of the most aestetically pleasing attributes of use in the functional languages
Again, this is a purely aesthetic argument -- the intent of the library seems unclear to me from its syntax. If it's beautiful and clear to you, I'm not going to object to your using it. Aesthetics aside, I don't think there's anything wrong with this library.
class Funk:
def __init__(self, f):
self._f = f
def __or__(self, other):
return Funk(lambda *_, **__: other(self(*_, **__)))
def __call__(self, *args, **kwargs):
return self._f(*args, **kwargs)
pipeline = Funk(h) | Funk(g) | Funk(f)
pipeline(x)
pipeline(y)
I'm kind of surprised nobody's done this before. Unless they have. Any pointers to more fully fleshed out implementations of this idea?
As it turns out, implementing that different syntax in pyfunctional wouldn't be too hard or API breaking I think. Mainly it would require 1) wrapping/exporting functions to a module you could bring into scope (eg `from functional import functions as F` or `from functional.functions import *`), 2) Writing wrapper code to provide something functionally similar to `pipe`.
On libraries, I swear I saw something a while back, but my googlefu just now didn't help me find it.
Pipetools is all about implementing a "clever" syntax for data flows, whereas PyFunctional looks like it's much more about providing a library of functions for manipulating data in functional ways.
I would expect that PyFunctional could probably implement the pipe syntax if they wanted, as just a layer of syntactic sugar on top of their core, however that would break OR semantics in places so I'm not convinced that syntax is a good choice, as "clever" as it may be.
Here's one way you might write one of the examples in Coconut:
([1, 2, 3, 4]
|> map$(-> _ * 2)
|> filter$(-> _ > 4)
|> reduce$((x, y) -> x + y)
)
if foo.is_success():
...
else: # foo.is_failure()
...
People are often satisfied by the veneer of something better, because, they were told it was better but don't truly understand why. This is super common in the programming world, I'm afraid.
{ok, integer()} | {err, string()}(0) Functional programming is expressing computation as the evaluation of functions. A language is “functional” to the extent it allows you to do functional programming.
(1) A function is a mapping from values to values. To express computation as the evaluation of functions, you need a rich universe of values: not just primitive ones like integers and object references, but also tuples, lists, trees, etc.
(2) Values are timeless entities that are distinguished from each other structurally. For example, it makes no sense to distinguish “this list [1,2,3]” from “that list [1,2,3]”, because both have the same structure - the same parts. Thus, what Python calls “tuples” and “lists” are not really tuple values and list values.
---
@_9jgl
Let's see how equal they remain in a timeless fashion:
xs = [1,2,3]
ys = [1,2,3]
zs = ys
send_to_another_thread(xs,ys,zs)
print(xs == ys) # who knows
print(ys == zs) # True
print(xs is ys) # False
print(ys is zs) # True
---
@lmm
> This is by no means the only way to write Python
This is not the problem. The problem is that Python doesn't have compound values. Values are a matter of semantics, and semantics is a matter of how the language is defined, not how you choose to program in it.
Would you mind expanding on this? I'm not sure I really understand what you mean – if we have two lists a = [1,2,3] and b = [1,2,3], then a == b, so I don't see how Python is distinguishing them from each other.
Edit: although I guess that just using a tuple would be better for this maybe
I don't see what encapsulation you're talking about. Maybe and Either are very much concrete, not abstract, data types.
> Where some situation means that null already has a meaning, the alternative is to create a wrapper class for your situation (not general), or a general wrapper class which is basically what this is.
Unless you want to take advantage of existing plumbing infrastructure such as a monad transformer library, custom wrapper classes for your particular situation are precisely the right tool for the job.
By itself you might as well use nulls or throw errors although you could argue that it's clearer having a maybe or an either.
The monad laws already fail miserably in Haskell when you account for nontermination. Haskellers can only cope with this by eschewing partial functions as a matter of coding style. Now, eschewing partial functions, not to mention effectful procedures, is essentially impossible in a unityped language like Python. Are you sure you still want to play the monad game?
Of course, the decision to tolerate leaky abstractions is very much subjective. But constantly dealing with abstraction leaks is, in my experience, far more burdensome than not using abstractions to begin with. This is especially the case with mathematical abstractions whose meaning is given by equational / rewriting rules.
foo.fold(lambda successResult: ..., lambda failResult: ...)
Do you seriously consider this less of a hassle than branching on a frigging bit?
I'd love to have a reactive state libraries like Mobx on server-side.
Mobx is also reactive, but has a different focus: It is not so much avout the low-level event dispatching, but reacting on data changes, recalculating derived values (but only those affected, and being lazy on non-observed dependent values). And Mobx determines dependencies of calculated values automatically.
Also, Vue [2] has Mobx functionality built-in, with an even better syntax, but closely tied to the framework.
Alas, both are JavaScript and targeted at client-side / user interface. Is there something similar for Python? (See also my question on SO [3].)
Not the same thing, of course, since Mobx is much more sophisticated than Redux, but in my team we're currently experimenting with aioredux for state management of client data, which drives the React+Redux frontend. It's working surprisingly well so far, although the biggest hurdle is team familiarity with functional aspects of Redux, and general structuring and responsibility of reducers.
The library is no longer maintained, but considering the simplicity of the implementation and of Redux itself, we felt it wasn't a show stopper since we could always take over maintenance ourselves.
I'd also be interested to know of Mobx Python implementations.
The fundamental difference between reactive state management (Mobx) and the Redux approach is that Mobx has some automatisms that are vital to what I'm looking for:
- automatic recognition of actual value dependencies (i.e. minimal update chains)
- automatic determination of the order of updates (so you don't have to think about and design your reducers along their dependencies ... because this is an annoying task which can and hence should be automated ... it is prone to inefficiencies, and sometimes errors, when done by hand.)
I was interested to see that coexisting with Pandas is supported, but the documentation seems either scant or unfinished, I can't tell which:
https://github.com/EntilZha/PyFunctional/blob/master/example...
Fair enough on docs being a bit scarce. The main intent is to being able to easily make a pandas dataframe a sequence of tuple/namedtuples, and convert a sequence of tuple/namedtuple to a pandas dataframes.
And the backslashes shouldn't be mandatory for wrapping method chains.
So for parallelization operations that are embarassingly parallel (map/filter etc) are paralelized with multiprocessing. If the data is heavy and serialization is expensive it might be slow, but for operations where the bulk of the work is done in parallel it can help a lot.