Comment by stephth | Hacker News Reader

Comment by stephth | Hacker News Reader

Parent Full thread

I often wonder why there is so little interest in a high level language that would compile to pure C++, with a syntax that makes code more succinct and friendlier, just like CoffeeScript compiles to JavaScript. We have a clear problem: higher level languages make programmers more productive, but they have performance issues. Because of those, developers that need performance fall back to C++ (look at videogame development, C++ is there to stay).

So why not make a language that is high level, but compiles to pure, cross platform, C++? Think of the productivity win: a higher level language that rarely needs optimizations. Obviously it would have to come with some lower level semantics, like memory management (although it could maybe be solved somehow like Apple did with Automatic Reference Counting, which is basically a preprocessor) or static typing (although the compiler can sometimes be instructed to guess the type, see the := operator in ooc [1]), but even if some lower level semantics are unavoidable, it still seems like a huge win. Yet every time I saw the idea mentioned online it's been either mostly ignored or treated as stupid idea, and the very few projects that attempted to go in that direction never took off. Maybe I'm missing something? If so I'd love to know what it is.

No matter how fancy and awesome higher level languages are, we keep going back to C++ for performance, and that's probably never going to change (at least until processors are stupidly fast). So why not make it easier? The CoffeeScript approach has largely been proven to work.

[1] http://docs.ooc-lang.org/language/syntax.html#declarations (note that ooc is a good step in this direction, it compiles to C99, but then takes step back: it depends on a garbage collector)

The weave bits, and Pyrex bits, found in this article are actually snippets of C/C++ written inline in the Python code (or very simplified Python), and able to access variables and such defined in the Python earlier (with caveats and when used with care), and later Python code is able to use the resulting compiled functions.

So, it's high level Python code, except when you need to write low level, super fast code.

The reality of making a very high level language compile down to high performance C/C++ is a much harder problem than "compiling" CoffeeScript to JavaScript. Both are quite high level, and CoffeeScript is merely very concise syntactic sugar for a number of common patterns in JavaScript, and maybe a few higher level constructs tacked on for good measure. But, JavaScript is effectively Scheme with C-like syntax; it was designed to be built up in this way, just like Lisp and Scheme (and it's only the lack of control over the parser and syntax that forces it to be a compiler at all; all the capabilities are there in JavaScript: closures, first class functions, code as data, etc.). Building DSLs is what this kind of language is for. C and C++ is not for building DSLs, and you don't magically get C performance by converting high level code to low level code.

One example is B::C, a Perl to C compiler: http://www-rohan.sdsu.edu/doc/perldoc-html/B/C.html

In short, the CoffeeScript approach is not at all comparable to a high-level to low-level compiler. It is comparable to a, say, Lua to Perl compiler, perhaps. From one very high level language to another very high level language.

It's not lack of interest. Weave and Pyrex, which are discussed in this article, are proof positive that people want the speed of low level and the convenience of high level languages. It's just not possible to get it via the means you've described. JITs seem to be the currently fashionable way to get closer to that goal, though it's still miles away.

Oh, and there are Lisp compilers that compile to lower level languages like C or compile very fast Lisp. Lisp is quite high level. So, when it goes fast, it is impressive. I'm not knowledgeable enough to know whether they could be used in the same contexts of SciPy, Weave, Pyrex, etc. I imagine there are big math libraries for Lisp, though...and they're probably pretty fast.

As for ooc and garbage collection: How would you write a compiler for a high level garbage collected language without including a garbage collector? And, do you believe that garbage collection is the primary reason dynamic languages are slower than compiled non-GC languages? (Hint: It is not. GC is very far down the list of resource users in every garbage collected language I'm aware of.)

It's just not possible to get it via the means you've described.

Why not? it might harder to implement than coffeescript, but ooc is proof that it's possible. Remove GC and dependent libs, and you still have a substancially higher level language than C (or C++).

As for ooc and garbage collection: How would you write a compiler for a high level garbage collected language without including a garbage collector?

I didn't say it should be garbage collected: Obviously it would have to come with some lower level semantics, like memory management. The coffeescript approach is it shouldn't have dependencies. By high level, I meant higher level than c++. it will always be lower level than a language like python, without dependencies we can't escape all the semantics underlying in the target language. That said like I mentioned before, memory management could possibly be handled by something similar to Apple's ARC (and to answer your last question, I don't think it's that black and white, look at how Apple is investing on ARC instead of GC; also GC's unpredictability make it a no-go for low level routines in videogames).

"Obviously it would have to come with some lower level semantics, like memory management."

Why? Garbage collection is nowhere near the top reason high level languages are slow, and there are GC languages that are fast.

"By high level, I meant higher level than c++. it will always be lower level than a language like python"

So, Java or C#, then? JIT languages that can be as fast as C/C++ in some circumstances, and faster in others (and slower in still others). And you don't even have to manage your own memory, so it's seemingly higher level than you're asking for.

I think GC is fine for high performance computing. Might not be fine for real time systems or games, but that's not what this article is about. It's about high performance scientific computing in Python. Having a GC sweep every now and then, even at unpredictable times, isn't going to cause things to come to a crashing halt or ruin the performance of a batch run.

PyPy might be something you'd enjoy reading about. It is an alternative implementation of Python (mostly written in Python, and using a JIT), with the goal of being fast like C. This may be the closest thing to a CoffeeScript like tool where the goal is to make a very high level language faster through translation or "compilation" or whatever.

"Obviously it would have to come with some lower level semantics, like memory management."

Why?

Once again: this approach means zero dependencies. And in a way, you answered it yourself:

Might not be fine for real time systems or games

And there are more examples where it might not be fine, here's a major one: building a C++ library. It's true, this article is not about real time systems (I do apologize for diverging...), but this language could be one solution to problems approached in this article (it would replace the C++ example).

I am interested in projects like shedskin or pypy, but they have dependencies. The bottom line to me is: a higher level language that compiles to C++ code with zero dependencies. Think about it: this language could be used for any application, deployed in virtually any codebase and any system.

"... people want the speed of low level and the convenience of high level languages."

http://basic-converter.org

Shedskin compiles python to C++. It's getting quite good too. You can compile up to 10,000 lines within an acceptable time now. It is a subset of python though, so more dynamic things do not work.

You can also get it to compile as a CPython module. The latest release supports the mmap module, so it is possible to quickly share large array data without copying. Shedskin 1.0 is coming out maybe within a number of months after years of development.

The C++ code it generates depends on a Python runtime, doesn't it? That means you can't use it to write - for example - C++ libraries.

Reply on news.ycombinator.com