Hacker News Reader

Comment by catnaroek | Hacker News Reader

Too much wishful thinking during the second half of the talk, though.

Also, while fancy runtime systems can improve the performance of dynamic[0] languages, it doesn't come for free: the price to be paid is the loss of elegance. For instance, a JIT compiler could inline a virtual method that seems not to be overridden anywhere, but if later on it turns out that the virtual method was overridden somewhere, the “optimization” has to be undone. How can anyone in their right mind trust a language that requires such dirty implementation tricks to achieve decent performance?

[0] By which I mean “less amenable to static analysis”, regardless of whether the language has a static type system. For instance, Java and C# are dynamic languages in this sense.

View on HN

   trust a language that ...

Where's the problem?

JIT compilers work. Most high-level languages require compiler tricks for achieving performance from Scala to Haskell to Prolog.

It's useful to distinguish between (1) having a clean, easy to understand semantics and (2) having a fast implementation. Use all the hackery in the world to get your language fast, as long as it's abstract semantics is easy and canonical.

> Most high-level languages require compiler tricks for achieving performance from Scala to Haskell to Prolog.

To make things perfectly clear: I'm not against optimizations being performed automatically by compilers or runtime systems. What I'm against is unclean designs: deliberately performing an unsound optimization and then rolling it back is an unclean design.

> Use all the hackery in the world to get your language fast,

The language implementation is a program itself, and I don't have any good reasons to trust a hackish language implementation any more than I trust other hackish programs - that is, not at all.

> as long as it's abstract semantics is easy and canonical.

Hah! This thread is about JavaScript.

   is an unclean design.

It's not unsound. Otherwise you'det g incorrect results. You could say the design is wasteful, because you optimise and then throw away the optimisation. But it's hard to do better for some kinds of languages.

   trust a hackish language implementation

I agree. And indeed JIT compilers are hard to get right. But in practise even JIT compilers are much higher quality than applications: ask yourself, how many of the bugs in your code turned out to be compiler bugs, vs how many were ultimately your mistakes?

> It's not unsound. Otherwise you'd incorrect results. You could say it's wasteful, because you optimise and then throw away the optimisation.

The optimization is unsound. If it weren't, it wouldn't have to be rolled back occasionally.

If you're talking about the combination of the optimization and the rollback mechanism, it's not unsound, but it's inelegant. A runtime system designed this way only understands your program in a statistical sense (based on concrete execution profiles, which may vary from one run to another), never with the full certainty that static analyses (type checking, abstract interpretation) can give you.

> how many of the bugs in your code turned out to be compiler bugs, vs how many were ultimately your mistakes?

Of course, most were my mistakes. But the very reason why those bugs made it into the final executable is the lack of powerful static analyses in the first place. Curiously enough, when I use languages that make static analyses possible, I write programs with less bugs and they perform better without relying on fancy runtime system tricks.

---

Sorry, can't reply to you guys, because “I'm submitting too fast”. So my replies go here:

@mafribe:

> That's an orthogonal issue.

It's not. Static analyses gather valuable information that can be used to emit efficient code.

> More powerful static analysis is also more time-consuming.

So perform it ahead of time!

> One of the design goals of Javascript JITs is to make web-pages as responsive as possible. That rules out complicated static analysis.

Of course, a browser can't spend much time statically analyzing JavaScript programs, but programs can be statically analyzed (gasp!) before they're deployed.

---

@smallnamespace:

> Why does that matter to anyone?

Because this implementation technique is unnecessarily complex, and a far simpler alternative exists: Know beforehand what your program has to do. Think before you write code.

> Java and C# are both statically, strongly typed languages where JITs are the dominant implementation.

Their type systems can be easily subverted, so they're not “strongly typed” in my book.

---

@smallnamespace: Oops, sorry, I accidentally swapped my two replies to you: this one and https://news.ycombinator.com/item?id=12481956 .

> it's not unsound, but it's inelegant. A runtime system designed this way only understands your program in a statistical sense (based on concrete execution profiles, which may vary from one run to another), never with the full certainty that static analyses (type checking, abstract interpretation) can give you.

This is a false dichotomy. You can always build a JIT that uses runtime statistics to speed things up, even in languages that are quite amenable to static analysis, for the simple reason that type systems cannot capture all relevant runtime context. Java is a good example here, since it's strongly statically typed.

By your reckoning, all optimistic heuristics that CPUs do like branch prediction and prefetching are also similarly 'inelegant', because they can be wrong and require rolling back.

Optimistic heuristics have a long history in computer science, and IMO it seems strange to single one particular use case as being particularly evil.

   The optimization is unsound.

The JIT compiler is sound, w.r.t. to the source language's semantics. That's the only thing that matters for the programmer.

   but it's inelegant.

Elegance is in the eye of the beholder. I was blown away when I first encountered JIT compilers.

  lack of powerful static analyses

That's an orthogonal issue. More powerful static analysis is also more time-consuming. One of the design goals of Javascript JITs is to make web-pages as responsive as possible. That rules out complicated static analysis.

> How can anyone in their right mind trust a language that requires such dirty implementation tricks to achieve decent performance?

Isn't this a rather broad brush with which to paint all JIT language implementations, including Java, C#, and, say, PyPy?

Could you name a JIT compiler that doesn't do this kind of thing, yet offers performance comparable to AOT-compiled languages?

(FWIW, I'm not saying it's impossible. It's perfectly possible, but you'd need a source language that offers much better static guarantees than the typical language that a JIT compiler is written for.)

---

Sorry, can't reply to you directly, because “I'm submitting too fast”. So my reply goes here:

> for the simple reason that type systems cannot capture all relevant runtime context.

Type checking isn't the only kind of static analysis out there. And there's no need to use statistics to optimize anything at runtime when your ahead-of-time compilation step already emits optimal target machine code.

> Java is a good example here, since it's strongly statically typed.

Java is as dynamically typed as it gets: `instanceof`, downcasts and reflection, all conspire to reduce the usefulness of static type information to zero.

> By your reckoning, all greedy optimizations that CPUs do like branch prediction and prefetching are also similarly 'inelegant', because they can be wrong and require rolling back.

Yes, indeed. It's more elegant to know beforehand what exactly you have to do, and then do just that and nothing else.

> that _doesn't_ do this kind of thing

Why does that matter to anyone?

> you'd need a source language that offers much better static guarantees than the typical language that a JIT compiler is written for

Java and C# are both statically, strongly typed languages where JITs are the dominant implementation.

Almost by definition, it's impossible to write a JIT compiler that outperforms AOT compilation without looking at runtime data, because AOT compilers have a lot more time to look for difficult static optimizations. The reason JITs can keep up is because they have access to information that an AOT compiler does not.

> And there's no need to use statistics to optimize anything at runtime when your ahead-of-time compilation step already emits optimal target machine code.

This is simply untrue. For example, it's not possible to statically determine whether a function should be inlined or not. However, a JIT can see that it's used in a hot loop and dynamically inline.

For any language, no matter the type system, runtime information will always be a superset of compile-time information. There will always exist optimizations in a JIT that aren't possible in an AOT compiler.

> Java is as dynamically typed as it gets: `instanceof`, downcasts and reflection, all conspire to reduce the usefulness of static type information to zero.

Idiomatic Java code doesn't use these features heavily. Just because it's possible to wipe out type information doesn't mean that the vast majority of code that an AOT or JIT compiler sees won't be strongly typed.

> For example, it's not possible to statically determine whether a function should be inlined or not.

MLton has absolutely no problems inlining functions, even higher-order functions, at compile time. This is only difficult in languages with virtual methods, because they can be overridden anywhere. If anything, that's an indictment of virtual methods, not AOT compilers.

> However, a JIT can see that it's used in a hot loop and dynamically inline.

What if it's a virtual method call that's known to be overridden in several places? You can't inline it, even if it's in the middle of a hot loop.

> For any language, no matter the type system, runtime information will always be a superset of compile-time information.

Runtime information is always anecdotal, specific to one particular run of a program, so...

> There will always exist optimizations in a JIT that aren't possible in an AOT compiler.

... for every “optimization” a JIT can perform, there will always exist a program for which the “optimization” will have to be rolled back after it has already been performed, because it turned out to be unsound.

> Idiomatic Java code doesn't use these features heavily.

Language implementations must work correctly whether you write idiomatic or unidiomatic code.

> Just because it's possible to wipe out type information doesn't mean that the vast majority of code that an AOT or JIT compiler sees won't be strongly typed.

Most code I write in Python could be given static types too. That doesn't make Python a statically typed language.

And “strongly typed” doesn't really mean anything.

> MLton has absolutely no problems inlining functions, even higher-order functions, at compile time

Of course, but how does it know which functions to inline? If you inline everything, then you will blow through your cache.

> What if it's a virtual method call that's known to be overridden in several places? You can't inline it, even if it's in the middle of a hot loop.

That's not true--a JIT could optimistically replace with a concrete realization.

> Runtime information is always anecdotal, specific to one particular run of a program, so...

That's a benefit. No matter what AOT compiled code you have, it is possible to speed up execution if you know what code paths you will take.

> ... for every “optimization” a JIT can perform, there will always exist a program for which the “optimization” will have to be rolled back after it has already been performed, because it turned out to be unsound.

Yes, but so what? As long as it improves performance in the average case, and the worst case is bounded, then that is a net win. You can equally well write deliberately obfuscated code that an AOT compiler has trouble with.

> Language implementations must work correctly whether you write idiomatic or unidiomatic code.

Implementation is correct. Only reflection is slow. If you don't want that, don't write reflection.

> Of course, but how does it know which functions to inline?

Small functions and higher-order functions are the most natural candidates. (The two categories greatly overlap in most cases.)

> That's not true--a JIT could optimistically replace with a concrete realization.

You'd have to roll back an unsound optimization in the middle of a hot loop. I'm pretty sure that's not what you want.

> it is possible to speed up execution if you know what code paths you will take.

That knowledge can be encoded statically in many cases, if only you used the right languages.

> As long as it improves performance in the average case, and the worst case is bounded, then that is a net win.

This is only the case when your program wasn't close to optimal to begin with.

> You can equally well write deliberately obfuscated code that an AOT compiler has trouble with.

Yes, but languages amenable to static analysis will actively get in your way if you try to write such obfuscated code. The static analysis either tells you that your program is gibberish, or outputs nonsensical gibberish of its own. So the path of least resistance is to write code that the static analysis knows how to optimize. Which is not the case in dynamic languages (including pseudo-static ones like Java).

> Only reflection is slow. If you don't want that, don't write reflection.

So. basically, you're telling me to ditch Java's entire library and framework ecosystem?

> Small functions and higher-order functions are the most natural candidates. (The two categories greatly overlap in most cases.)

But then you're just guessing. Isn't that also inelegant?

Let's play devil's advocate: how do you decide the cutoff on function size for inlining? Well, you would profile a bunch of programs with various cutoffs... now all you have is a heuristic, and MLton will inline some functions that it shouldn't, and it will fail to inline other functions that it should.

It will do worse than a JIT at this, because the JIT has more information.

> That knowledge can be encoded statically in many cases, if only you used the right languages.

Sure, but you won't ever succeed in encoding all of it, which is why runtime techniques can have a place.

> This is only the case when your program wasn't close to optimal to begin with.

That's simply untrue, and you can prove that formally -- given a machine M and a program P that produces outputs on a set of inputs I, it is always possible to come up with a program P' that produces those outputs with fewer steps on some subset of I, in return for taking more steps on the rest of I (except in the trivial case where the running time is completely independent of input).

You can view a JIT as iteratively replacing P with P' after it sees which inputs I are most common, and this is true no matter what M, P, or I are. In particular, there exists a version of P' that is faster than the statically optimized version of P on your program's input.

> Yes, but languages amenable to static analysis will actively get in your way if you try to write such obfuscated code.

I don't see how that isn't equally applicable to writing code to fool your JIT.

> So. basically, you're telling me to ditch Java's entire library and framework ecosystem?

Framework code doesn't generally run inside your inner loops, so I don't see how that should affect either your AOT or JIT compiler much.

>Let's play devil's advocate: how do you decide the cutoff on function size for inlining

AOT compiler writers have been tuning inline heuristics for more than 40 years. Sure sometimes you have to help the compiler with annotations or PGO, but in the large majority of cases things just work.

In fact AOT can deal much better with the massive code explosion due to aggressive inlining than JIT compilers which have a very tight time budget for optimisations.

> In particular, there exists a version of P' that is faster than the statically optimized version of P on your program's input.

Sure, but I'm interested in what the program does on all meaningful inputs, not a specific one. Otherwise, I'd just precompute the answer and hardcode it.

> I don't see how that isn't equally applicable to writing code to fool your JIT.

AOT compilers are supposed to provide feedback to the programmer about what the program means (e.g., inferred types, type errors). JIT compilers are not.

> I don't really understand why optimistic heuristics bother you so much.

Um, because they can be wrong, and then you need to fix errors, which makes the system more complex?

> Seems like you just have an aesthetic preference.

Yes, for simplicity, and for thinking before writing code.

> Um, because they can be wrong, and then you need to fix errors, which makes the system more complex?

Yes, but I don't understand why compiler complexity concerns you, so long as the whole thing works. AOT compilers are also extremely complex, and are also full of heuristics.

You seem really hung up on the fact that an optimization can be rolled back at some point, but why should you care? JIT optimizations can be 'wrong' in the same way that caches can miss. It the right engineering solution to eliminate caching, over some belief that one should never be 'wrong' anywhere in a program, even though the eventual answer is always correct?

This might matter in system with real-time performance demands, but then you should be equally concerned about the garbage collector, for example.

> Yes, but I don't understand why you compiler complexity concerns you, so long as the whole thing works.

Because I find it easier to trust simpler systems than complex ones.

> AOT compilers are also extremely complex, and are also full of heuristics.

Yep, those heuristics are annoying too. (But less so than the ones JIT compilers use, because at least they don't involve temporarily breaking my program.)

> unless you've profiled it and see this having a real adverse effect on overall performance?

How many times do I have to repeat that what annoys me is the excessive complexity?

it's interesting that C# has recently moved away from JIT and reflection for client apps (.NET Native), though.

Reply on news.ycombinator.com