Hacker News Reader

Comment by lkirk | Hacker News Reader

A big use case for python is glue for high performance c/c++ code. How is racket at this? Also, is there a cython alternative for racket? I would also argue that, though racket may have more batteries included, there are definitely not as many externally developed libraries. Especially important are numpy/scipy/pandas/pysam, the list could go on... That's not to say that I'm against racket, I really like the language and its level of design and documentation.

View on HN

I've used the Racket FFI for some important tasks, and it's very easy to use. However, it's not as fully baked as the Python one (I've found bugs due to GC interactions) and it's weirdly slow (a few too many levels of wrapping). That said, I still use Racket extensively and enjoy it.

This is the write up I was looking for as this is the use case I've had in mind for Racket of late after reading Felleisen's LOP paper.

Would love to know more about your experiences with doing Racket FFI. How hard were the bugs around GC to discover and workaround? What were you building in Racket?

See my responses to a sibling comment for Github bug reports. I had a great experience with them: very fast triage and fixes, plus a lot of help developing a workaround so we could target a range of Racket versions. (Distros still ship some old versions!)

These bugs were found while working on Herbie <http://herbie.uwplse.org/>. We use the FFI to bind to your machine's math libraries so we can properly evaluate how accurate a floating-point expression is on your machine.

You found bugs in the FFI mechanism itself, or in FFI bindings that someone had written for a particular library?

In the interaction between FFI, GC, and hash tables, so yes, in the FFI itself, not in particular bindings:

https://github.com/racket/racket/issues/2702

https://github.com/racket/racket/issues/2263

There were workarounds (in Racket you can instruct the GC not to move your FFI objects) and I believe the underlying issues have been fixed.

I must add that I got immediate and comprehensive support from the core developers, who not only fixed the bug but also suggested the workaround (so I could continue to support all our target versions of Racket).

Thank you for your great bug report on this, and I'm glad to see it was in good hands.

While it's nowhere near as mature as Racket, those interested in this use case might also be interested in Clasp, a Common Lisp implementation built on LLVM specifically to enable the combination of Lisp-style dynamic development approaches with C++ libraries.

https://github.com/clasp-developers/clasp

https://www.youtube.com/watch?v=mbdXeRBbgDM

There's a nice FFI for C stuff, but writing glue for C++ is more work.

FFI into C++ is a lot of work in any language. A lot of projects just define a plain C wrapper for the C++ code they want to call, and call that C code from whatever their glue language is.

However, Python does have CLIF[1], which is the nicest solution for calling into C++ that I'm aware of.

[1] https://github.com/google/clif

Have you seen cppyy?

https://cppyy.readthedocs.io/en/latest/

Not in enough detail to have an informed opinion.

There's also Boost.Python, which boasts "seamless interoperability between C++ and Python": https://www.boost.org/doc/libs/1_70_0/libs/python/doc/html/i...

(In general, the Boost C++ libraries are well-renowned in the C++ world.)

I think the biggest thing, for me is Cython. I've not seen anything quite like it in other languages. It allows you to compile python code to c, with gradual typing. It also allows you to write c code inline w/ your python or interface with other C/C++ libraries. https://cython.org/ Other languages will be pressed to beat its utility (esp in the scientific computing world)

pybind11 is a modern version of the Boost Python approach to Python bindings. Header only and can be installed trivially from PyPi. I like it well enough that I even use it for binding C code. Especially because it has nice numpy array support (albeit a little underdocumented).

In my experience it’s far from what I’d call seamless but it’s definitely good enough

On Windows it is much better, thanks to COM and now UWP, which improves COM a lot regarding what is exposed across the ABI.

Yes, bare bones COM is full of boilerplate, however there are more productive ways to use it.

How does CLIF compare to PyBind11[1]?

[1] https://github.com/pybind/pybind11

Nicest for Python, I assume.

Many languages have FFI. Lua, PHP, etc. https://en.wikipedia.org/wiki/Foreign_function_interface

I don't understand the choice of Python for things like gluing together C programs. Seems like a performant mistake, at the very least.

It’s the 80:20 rule, which naturally doesn’t always holds. Often, however, a small part of the code is responsible for almost all of the performance issues.

By picking the performance sensitive areas of the program to code in C one can often code the rest of the program in a slower more convenient language.

Reply on news.ycombinator.com