Hacker News Reader

Comment by gnulinux | Hacker News Reader

I write compilers for fun (hobby) and I just transpile my IR data structure to C++ (I'm also experimenting with other targets like Haskell (similar to agda compiler) and Rust (work in progress)). My friends usually treat this approach "too amateur" and a "real language" should compile all the way to assembly. The reason I don't do that is if you transpile to a well-known language and then manage that compiler to go all the way to machine code, you get insane amount of optimizations for free. Like C++ compilers are so good, you rarely need think about low-level optimizations, as long as your program is asymptotically sound (i.e. you don't keep random inserting into an array) it gives pretty good performance regardless.

View on HN

> My friends usually treat this approach "too amateur" and a "real language" should compile all the way to assembly.

Get new friends—no, seriously. I used to think like them, and hung out with similar minds. It is gatekeeping[0] in a way that, if internalized, will blind you to opportunity.

A few things I missed because I scoffed at them for not being "pure enough":

1) The entire Javascript ecosystem

2) Bitcoin, circa 2009

3) Countless SV startup opportunities from 2001-2015

4) ML and data science (e.g. the terrible tooling)

The success I have found always came with an open mind to people's approaches, even if they were not the ones I settled on. It took me way too long to learn this, but I hope I can change a few younger people's paths.

[0] https://www.urbandictionary.com/define.php?term=Gatekeeping

Nim compiler also generates C, so I'd say you are in pretty good company there.

The Idris 2 compiler compiles to Chez Scheme; the Idris 1 compiler compiled to C, and the former actually produces significantly faster executables than the latter.

I sometimes see programmers have very strange, irrational purity misgivings a particular case was someone who did not wish to fork off `grep` for some task, but actually wrote his own implementation which broke input into lines and used and manually searched for substrings; — he was initially convinced that this would be faster, or was at least “cleaner” than simply forking off `grep`, and piping to it.

The reality is that modern `grep` implementations are so optimized that writing something that was as efficient would probably exceed the complexity of his entire project.

I have never understood why many programmers seem to have an irrational dislike against forking off POSIX utilities when they target such platforms, and piping through them, which are often far more optimized than anything they've ever written.

One reason is that disparate impls of POSIX utils don't behave as similarly as one would hope. It's easy to cross the line from using flags that are guaranteed to be available to using flags and behaviors only available on your machine.

An example of a mistake I've made like this is relying on `find` on linux to default to searching the pwd if it doesn't have a positional arg. Another is `sed -i` requires a file extension on MacOS, but doesn't on Linux. You can catch these with tests to some extent, but how many people bother running their CI on MacOS just to avoid breaking other devs?

That said, I did work on a production system which happily shelled out to `dig` for DNS lookups for several years; everybody who saw it had a visceral negative reaction, but it also worked well enough that there was no good reason to spend time replacing it.

> One reason is that disparate impls of POSIX utils don't behave as similarly as one would hope. It's easy to cross the line from using flags that are guaranteed to be available to using flags and behaviors only available on your machine.

Certainly not, it's as easy as looking up the specification for anything else; what is and isn't POSIX is well documented.

> everybody who saw it had a visceral negative reaction

This is a reaction rooted in emotion, not reason.

> This is a reaction rooted in emotion, not reason.

Yes, humans are driven by emotion, not reason. Even when there is superficial rationality, it's mostly post hoc rationalization, not actual rationality.

> I have never understood why many programmers seem to have an irrational dislike against forking off POSIX utilities when they target such platforms, and piping through them

Many programmers have trust issues with code, and find it easier to write and comprehend a minimally-functional, narrow-scope implementation of their own than to get to know a general purpose utility enough to trust it. Also, lots of programmers would rather spend time on the experiencia of hacking out their own version than hacking out an interface to someone else’s code.

(And, yes, there's a whole lot of illogic and inconsistency in how this tends to manifests. But I think that there are very understandable reasons why a tendency in this direction is not only associated with predisposition to become a programmer, but also, among programmers, with tendency to develop deeper and broader skill as a programmer.)

If your explanation would be the genesis, then the same distrust would manifest when calling library functions, which does not seem to be the case.

The distrust seems to manifest from forking off a separate process invoked by it's name, not relying on externally written software. — it seems to be a distrust of forks and pipes.

I think what in many cases also plays is a fundamental misunderstanding of performance bottlenecks, with many programmers believing that forking off a program and piping to it is measurably slow, while in reality if the target program be well optimized it will be faster than a single heap allocation.

Not to be like your judgmental friends, but if you're going to use the approach that gives optimizations for free, why not just target WebAssembly / LLVM IR / Cranelift IR?

On the other hand, why yes? The OP found something that works for them and seems to give them the level of optimization they want. Presumably they also know C++ well, but possibly none of the technologies you mention well enough to "just" target them.

Because I don't know web assembly, llvm ir. I'm pretty experienced in C++ and Haskell and they have "good enough" performance for my toy projects, so why complicate things?

Besides you can compile C++ to webAssembly and llvm ir. (don't know about cranelift).

But then you miss all the fun with doing a nanopass compiler.

Maybe that's not entirely true though, you can still generate C/C++ in the end I guess.

Reply on news.ycombinator.com