Hacker News Reader

"What I find interesting about this is the way he did codegen by passing in mock objects that just emit llvm it in the simplest possible way and that it actually works while being incredibly succinct."

It's interesting, but it doesn't work in general, which is why you don't see the technique in common use. The problem is that it only catches methods called on objects, but it can't catch anything that isn't a method called on an object. Unfortunately, that includes for loops, if statements, some aspects of exceptions, and just generally "all flow control".

So rather than being a powerful and general technique, it's a powerful, but very very specific technique that only works in certain toy problems. Unfortunately, once you leave those toy problems behind, it doesn't even help you solve the problem because the paradigm this forces you to operate under makes easy things easy, but medium things hard and hard things pull-your-hair-out impossible. You're better off writing a real compiler... or doing this in a language that does expose enough of what's going on to make this technique work, although you tend to end up in either a Lisp or Haskell, which come at this from very different angles but both have the capacity to make this work. (Lisp via syntax tree manipulation, Haskell via Free Monads and other similar techniques.)

The AST is accessible in python, I wish more projects took advantage of it.

The Unity 3D engine is doing interesting things with this subset idea, where you can write most of your game in regular C#, then if you need a chunk of super fast stuff, you can use a subset of C# that can be easily vectorized, or parallelized, or GPU-ified. You can just do that all inline, no need for separate files or build processes. It is pretty neat and letting them get some crazy performance numbers.

https://unity3d.com/learn/tutorials/topics/scripting/using-b...

How are they at odds with each other? Julia is more expressive and faster than python. It supports all the same high level programming constructs, and better versions of the same (like it's lambdas, macros, multi dispatch).

The one issue is compilation time, which is much and improved and will be further obviated through more caching and planned multi level compilation.

Julia is a much bigger language than Python, by a long shot.

Bigger?

More complex, requiring much greater cognitive resource, “larger”.

I find the opposite to be true. End users don't need to worry about types, interfaces are generic and multiple dispatch and abstract typing obviates the need all kind of weird special methods and type system stuff

What part of Python has those things that you are describing?

https://docs.python.org/3/reference/datamodel.html

Then we have metaclasses, abstract base classes and protocols.

Also Super and self everywhere.

Even more complexity with typing, which is necessary if you want to approach similar type safety to julia ie to make the comparison more fair.

Try PyPy. IME it makes normal Python code 5-10 times faster.

Except when dealing with things that tend to require speed, PyPy is short in some respect in that the code won't work.

This has happened to me 5 or 6 times now. At this point I just stick to stock python.

Just curious - do you remember any specific case what went wrong?

I can't remember the very specifics, but one particular instance that sticks out is when I was creating some images on the fly for tobii eye-tracking software. I was using pillow, I remember... I think it might have been pillow which wouldn't work well with PyPy. I'll try to dig out my code later and see what specifically made things not work with PyPy in a few days

Aye.. There are some bugs left to be ironed out in PyPy. But many times you just need it to work and the correctness of what you're calculating isn't too critical. Then it is pretty awesome.

Yeah I like this technique too. To put a name on it, it's a form of metaprogramming that could be called "staged programming" or "multi-stage programming", and there are a bunch nascent systems that do things like it:

https://scala-lms.github.io/ -- Scala LMS is similar in that you replace type T in your algorithms with Ref<T>, and you get a code generator (compiler) rather than an interpreter.

This links to a CACM article by the authors of Scala LMS: https://cs.stackexchange.com/questions/2869/what-are-staged-...

https://andrewkelley.me/post/zig-programming-language-blurs-... -- Zig has a code generator for printf using compile-time metaprogramming.

http://terralang.org/ -- You can write arbitrary Lua to generate Terra code, sort of like he's writing Python to generate LLVM code. I think this is more general and doesn't necessarily work by retaining the structure of say solve_linear_system(). I'm not quite sure. But the problem domains they tackle are very similar to this (numerical code).

TensorFlow also has two stages of computation -- you define a graph of objects in imperative Python, using Python operator overloading in the same way that LLVMCode in this example does. And then later the separate TensorFlow language is evaluated in parallel / distributed on GPUs.

In one case you're defining an SSA graph of ops in LLVM; in the other case you're defining a graph of TensorFlow ops.

Note that the semantics of TensorFlow language are distinct from the semantics of Python, e.g. https://ai.google/research/pubs/pub46196

One interesting language in the same vein is Spiral, with "first-class staging" for machine learning:

https://github.com/mrakgr/The-Spiral-Language

You can also do a lot of this with constexpr / templates in C++, although the syntax is horrible.

As you allude to, these staged programming systems can have TWO languages or one. In the case of Scala and Zig it's one language, but in the case of Lua/Terra it's two. Arguably in the case of C++, it's two languages and not one.

Another thing to search for is "partial evaluation". I still have to understand the difference more precisely, as alluded to in the link above.

https://en.wikipedia.org/wiki/Multi-stage_programming

The Python/RPython language pair in PyPy is also related:

https://morepypy.blogspot.com/2018/09/the-first-15-years-of-...

Speaking of LMS, the Tensorflow for Swift project is doing graph program extraction from LLVM for the same purpose. Super neat stuff.

https://github.com/tensorflow/swift/blob/master/docs/GraphPr...

Yes I think it's interesting, here's me trying to understand it better:

https://www.reddit.com/r/ProgrammingLanguages/comments/9y4eh...

At first I was a little critical but I get it a little more now. Of course the real test is trying it, but I'm not working in that area at the moment, though I've used TF before.

What I find interesting is the recurring theme of two languages -- in Swift's case they say you can use a "static subset". I think that means all the imperative control flow, and then static dispatch (as in templates), but NOT dynamic dispatch (as in OOP).

So in this example you have Python/LLVM, then there's Lua/Terra, Python/RPython, Python/TensorFlow, or Swift/static Swift, etc. Except in the latter case I guess it's all mixed together and you don't explicitly have a graph building stage.

> “That said, like all things that try to accelerate Python, it only works for a subset of the language”

This is not true for Cython.