Pyffi – Use Python from Racket | Hacker News Reader

Pyffi – Use Python from Racket

soegaard.github.io

67 pointsgus_massa10 comments

Pyffi automatically exports Python modules as scheme objects and that's quite practical. We worked on this briefly while building the Gambit interface to CPython (https://github.com/gambit/python), but ran into some quirks with the current Gambit macro and module systems that forced us to postpone. It is definitely something we want!

Our syntactic interface based on Gambit's infix parser (SIX) is quite ergonomic and allows you to write expressions that look just like Python, within Scheme code (and vice-versa). It's much more natural than directly[1] passing strings to 'eval', although you can certainly do so (the whole low-level API is available). Here's the 'cal' example:

  > \import calendar
  > (display (\calendar.month 2022 09))
     September 2022
  Mo Tu We Th Fr Sa Su
            1  2  3  4
   5  6  7  8  9 10 11
  12 13 14 15 16 17 18
  19 20 21 22 23 24 25
  26 27 28 29 30

[1]: SIX expressions are compiled to strings and eventually evaluated by CPython's eval.

Neat! Nicely done.

fwiw, I wrote a Lisp in Python whose sexprs ended up looking similar: https://github.com/shawwn/pymen

That’s the runtime for it, which is self hosted, so it supports that dot syntax but doesn’t actually use it in its own code. (If you add a new feature like dot syntax to a self hosted lisp, then you start using the feature, and later decide it was a bad idea, it becomes incredibly difficult to revert — So the compiler sticks to a minimal subset.

But it can run code almost exactly as you’ve written, minus the backslashes. It also has:

  (for x in (list 1 2 3 4)
    (print x))

  (for k, v in ((globals) (.items))
    (print k v))

That was a fun one to get working… k, v is read as (, k v) which then compiles to k, v in the output code. Also shows method call syntax there; (globals) compiles to globals(), and (.items) compiles to .items(), so it becomes globals().items(). You can’t use dot syntax in a normal way when the functional position isn’t a symbol — it’s the result of a function call. Curious whether you solved that; I don’t really like my solution, but it’s the best I could think of.

List comprehensions

  (list i for i in (range 10) if (% i 2))

Compiles to

  [i for i in range(10) if i % 2]

Async functions

  (async def foo (x)
    (async for k in x
       (yield k)))

Uhh.. what else..

with statements:

  (with (open “foo.txt”) as f
    (f.read))

Exceptions, though I forget the exact implementation. It’s something like

  (try (f)
    (except ValueError as e (print e))
    (finally (print 1)))

classes

  (class (MyDict dict)
    (def find (self k)
      (get self k)))

(get self k) compiles to self[k]

And countless other small details. Notice there’s no return statements anywhere; you can write those manually if you want to, but by default they get automatically inserted due to the lowering pass.

Etc etc. I’m depressed and writing this on an iPad, so I’ll try to go sleep. But I hope your project helps make python more accessible from lisp — it’s a near and dear problem to me, so I love seeing people solving it!

Here is how the dot notation works in Pyffi:

    1. o.f                       access field f of object o
    2. o.f1.f2                   access field f2 of object o.f1
    3. (o .m a ...)              invoke method m on object o wth arguments a...
    4. (o .m1 a1 ... .m2 a2 ...) same as ((o .m1 a1 ...) .m2 a2 ...)
    5. (o.m a ...)               invoke method m on object o wth arguments a... 
    6. (o.m a ... .m1 a1 ...)    invoke method m1 on resultof object (o.m a ...) with arguments a2 ...

The notation works nicely for method chaining:

    (foo .somemethod arg1 arg2
         .someothermethod arg3)

Beautiful! How do you access a property on the result of a function call?

On the other hand, that case is pretty rare. Maybe it’s worth ignoring.

That's a good question.

The problem is that syntactically calling a method with no arguments and accessing a field will look the same, if .f is used to access a field:

    (foo .m1 arg1 .m)  ; m is a method name
    (foo .m1 arg1 .f)  ; f is a field name

I guess, one could introduce `_f` for field access.

Right now one can use `getattr` or one can bind the result of `(foo .m1 arg1)` to an indentifier `o.` and then use `o.f`.

Hey, just read your comment.

You seem to be using Scheme macros (try, async, list comprehensions, etc) to reproduce Python semantics in a Scheme syntax. That's one level on top of what we offer, i.e. I think it's very interesting, but as a library. Otherwise you're not writing Scheme anymore (your 'list' example looks good, but it's not Scheme).

We actually chose not to implement list comprehensions in the infix syntax because our grammar is a hybrid to allow us to have multiple target languages. You can already see how choices at this level can be mutually exclusive: the Python import syntax is not the same as the JavaScript import syntax.

Anyway, this is a vast space to explore, we just want to keep writing Scheme _with_ another language. Interesting, thank you.

> Pyffi automatically exports Python modules as scheme objects and that's quite > practical.

I'm quite pleased how well that feature works.

> We worked on this briefly while building the Gambit interface to > CPython (https://github.com/gambit/python), but ran into some quirks with the > current Gambit macro and module systems that forced us to postpone. It is > definitely something we want!

I just watched your video from Scheme 22 and I liked how you handled "passthroughs" and threads. Currently pyffi if more "one sided". The motivation was mainly to use Numpy (and friends) from Racket.

Yes the core of the contribution is the threading mechanism. Writing a low-level integration of CPython and $SCHEME is not trivial, but any implementer can do it. The syntactic interface we have clearly separates Scheme and Python code and we don't have to have new semantics for e.g. method accessors (you just write them as you would in Python).

It’s also worth noting that Clojure has libpython-clj (https://github.com/clj-python/libpython-clj) which offers an interface with Python from another lisp. Here are some advanced ML and dataviz examples using that lib: https://github.com/gigasquid/libpython-clj-examples.

I like libpython-clj. I used it in my Clojure AI book that you can read for free https://markwatson.com/books/clojureai-site/ (or buy a copy https://leanpub.com/clojureai).

For Common Lisp I have used py4cl and I also recommend it.

Reply on news.ycombinator.com