Hacker News Reader

Comment by gavinray | Hacker News Reader

I mean, to me it is a transpiler because it is a "frontend" to several other languages.

You choose whether you want to emit C, C++, or JS. There's an experimental, fledgling community LLVM frontend for Nim, that is what I would call a "compiler":

https://github.com/arnetheduck/nlvm

What Jetbrains calls Kotlin -> JavaScript, is "transpilation"

What Google calls Java -> JavaScript in j2cl, is "transpilation"

But this is all pedantic IMO.

View on HN

What if it just emitted C as its IR instead? MLton (the SML compiler) uses C as its primary backend (while they also have an x86 one) for portability and ease of implementation. By doing so, it bundles a small runtime needed for SML (garbage collection, closures, and such). Would you also consider that a transpiler? I personally wouldn't because SML to C is a drastic change, especially from the abstraction point of view. C just makes a convenient backend at that point.

But since it also has native backends and C is its only other high level language, would you still consider it a compiler since you specifically mentioned "several other languages"?

I think it's kind of fun to look at this philosophically, since these are broad terms without perfect definitions.

I just tried MLton with the "-verbose 2" flag and it seems to use "amd64Codegen" by default, on MacOS at least.

Oh neat! I'm assuming then that C is the default for anything that doesn't have a specific backend for. I believe I heard that they had an x86 backend, but it was a pain to work with (maybe not as fast). Those are probably old concerns though, since their x86 64 backend must be more performant.

Speaking about MLton, I've often heard that it's "special" as it's a " whole-program optimizing compiler", but I was under the impression that it's already what compilers do. Is there a difference between MLton and let's say Clang, GCC or something like that?

GCC will compile one set of files at a time into .o files, and then those are linked together later on, so each "module" (using that term loosely) is compiled separately.

MLton takes in the entire program source, including the standard library, at once. Because of that, there's a ton of information we know, which helps a ton for things like dead code elimination. Because of this, there's a lot of memory required, since we have the entire program's syntax/parse/any other type of tree in memory, and it's not going to be as fast as something like GCC since we're recompiling everything every time in MLton instead of only recompiling the changed modules.

Despite MLton being a bit slower to compile, it's truly an incredible compiler. I had Dr Fluet, the lead maintainer, as my compiler professor in college, and he's a wealth of knowledge. The compiler we wrote in class was in SML and compiled with MLton. The amount of optimizations that occur is insane, it made damn fast executables, and that applies to most big compilers like GCC, Clang, Rustc, etc. The big difference in optimizations is how much information we have at optimization time. Languages like C are a bit looser and thus information may be harder to derive. Rust has a stricter set of rules, so we have more guarantees about a program to base our optimizations around.

Thank you for the explanation. While that explain what makes MLton special, I now wonder why other compilers aren't doing the same. There is probably lots of money to make by optimising some performance critical code, especially C/C++. But you seem to say that MLton is one of a kind. I don't understand that part.

The full program optimization is more applicable in a more formally specified language like SML I believe. Big projects take quite some time to compile, which is definitely a pain while developing.

So I'm out of my depth at this point, but this post from the SE Stack Exchange was interesting in that it highlights that those cross boundary optimizations can be handled at link time. I'm not sure how much is possible compared to MLton. MLton will have more access to type info, I'd assume, but maybe some metadata is stored in a .o file that I'm just not aware of.

I don't know much about this serious level of optimization, but I assume big C compilers get optimizations that are equally, if not more impressive and do just fine.

https://softwareengineering.stackexchange.com/a/370242

> What Jetbrains calls Kotlin -> JavaScript, is "transpilation"

What about the new beta Kotlin/JS compiler that does Kotlin -> frontend IR -> backend IR -> JavaScript?

Oh man, good food for thought.

I guess... "Compilation of Kotlin, then emittance of JS"?

Because I believe this uses the shared "FIR" IR that the JVM/Native backends use right?

This is why the whole argument/topic is so subjective/pedantic haha.

Yup, the backend IR is shared between JVM, JS (beta) and Native. The new frontend IR, I realize on second thought, will come later[1] with the K2 compiler frontend.

The old Kotlin/JS compiler is definitely closer than the new one to what might be called transpiler.

[1] https://blog.jetbrains.com/kotlin/2021/10/the-road-to-the-k2...

>High-quality gdb/lldb debug information with source stepping, type information etc

Very nice. Last time I experimented with Nim the lack of this was the single biggest dealbreaker.

Well there's major drawback here:

  > nlvm does not:

  - understand C - as a consequence, header, emit and similar pragmas will not work - neither will the fancy importcpp/C++ features
  - support all nim compiler flags and features - do file bugs for anything useful that's missing

One of the unique and best things about Nim (IMO) beyond being multi-target, is that you can do things like say "emit this exact C/C++ string", or "import this type from an existing header".

With this LLVM backend, that doesn't work, and it doesn't support JS (which probably isn't a big deal to most people).

That means you can't do stuff like this:

  type
    VectorIterator {.importcpp: "std::vector<'0>::iterator".} [T] = object

  # std::vector<int>::iterator x;
  var x: VectorIterator[cint]

https://nim-lang.org/docs/manual.html#implementation-specifi...

https://nim-lang.org/docs/manual.html#foreign-function-inter...

https://nim-lang.org/docs/manual.html#implementation-specifi...

With the '--debugger:native' flag, the Nim debugging experience isn't too bad. Not on the level of Java or C#, but you can set breakpoints, step in/over/out of code and inspect or watch variables without having to look at C code. There are some issues with type visualization (to see the contents of a string you have to evaluate something like '(char*)variable->data') but Nim ships with a gdb plugin that should help (I haven't actually tried it).

It could definitely be better, and hopefully some day it will be, but I use Nim in a debugger quite frequently and it usually works out pretty well.

It was roughly 3-4 years ago the last time I tried it. Happy to hear they made good progress since then.

C is also a frontend to several other languages, which is why you can make portable code with it. Nim is the same. Instead of compiling to x86 for Windows and ARM for Linux, it compiles to C and JS. The idea is the same.

well what does make emitting C different from emitting LLVM IR? neither backends are meant to emit human readable code, the LLVM IR is also a "language", etc...

Hence the "it's pedantic" ending part of my argument.

To me, if the thing that gets emitted is another regular programming language, you've done transpilation.

If the thing that gets emitted is IR, bytecode, object code, essentially anything that's not something you would read/write yourself and is a step further towards being executable, it's "compilation".

I see transpilation as lateral/horizontal movement on the "path to being able to run it" chain, and compilation as vertical movement towards the executable end-goal.

But, again -- "pedantic", and just my $0.02.

Reply on news.ycombinator.com