Scala Native with Denys Shabalin – Software Engineering Daily Podcast | Hacker News Reader

Scala Native with Denys Shabalin – Software Engineering Daily Podcast

softwareengineeringdaily.com

56 pointslihaoyi41 comments

I wasn't familiar with Scala prior to listening to this episode, so I enjoyed that it provided both a background on the traditional, JVM-based Scala, as well as the new Scala Native project, which I'm more inclined to use.

There is a third backend, Scala.js https://www.scala-js.org

Based on the same principle as Scala Native, this one outputs javascript.

I've written a couple of projects where both the backend and front end is written in Scala, using one project, with different sub-projects for frontend/backend.

Scala.js will probably not convert Javascript developers en masse, but its very handy for Scala developers that prefer to write Scala to Javascript.

Im actually quite interested in why teams choose to use Scala if youre not fitting it into an existing JVM shop.

The only competing production-capable languages are Haskell, OCaml, and maybe F#

And Swift if targeting Apple devices is enough.

I'd include Rust on your list (if we're defining production-capable at a level where Scala Native qualifies), and maybe even Idris.

I do think Scala still has advantages: higher-kinded type support which OCaml and F# don't have, strict evaluation unlike Haskell, and better IDE/tool support than any except F#.

I wouldn't say Idris is "production ready" in terms of libraries and tooling. Unless dependent types is a must, I would choose Purescript instead if I wanted something Haskell'ish but strict and with nice JavaScript support.

Is PureScript really any more production-ready than Idris? That wasn't the impression I got.

It seems a bit more mature, at least in terms of package management and available libraries (https://pursuit.purescript.org). I'm not very impressed with the editor/IDE support though.

Even if you aren't fitting it in to an existing JVM shop, the available libraries on the JVM beat anything else for variety and quality. There are specific niches (e.g. visualization of data) that you can find better options, but if you don't know exactly what your project is going to entail in advance, JVM is the safest choice.

Why would you compile Scala natively itself instead of compiling JVM bytecode?

JVM startup is really painful for short to medium time running apps, this is the niche we’re trying to address.

I meant, "compile bytecode to machine code." The fact that nobody does it is a sign that it's not a good idea. I guess I was just wondering why.

Scala is a relatively simple language in the end of the compiler pipeline. We just take lowered ASTs and emit our SSA representation from there because it’s easy it to implement it this way. You can totally do something similar from the bytecode.

Excelsior JET does it, and it works Scala. So you can already compile JVM targeted Scala to native code that way if you want to. JET is fairly expensive, though.

Lots of people do it.

Besides ExcelsiorJET named in a sibling answer, the JDKs from Aicas, IBM, PTC, STMicroelectronics, Google with their ART implementation and a few smaller vendors for the embedded market.

Also Oracle just added initial AOT on Java 9, with plans to extend it to other platforms, and eventually (long term plan) rewrite everything in Java as part of Project Metropolis.

Oracle also had AOT compilers before Sun's acquisition, like JServe and JRockit.

Why does it seem no one does it?

Sun was religiously against AOT, leaving it to third party commercial vendors.

And since the rise of FOSS most developers think it is outrageous to pay for compilers, only Fortune 500s buy such compilers.

Are they all proprietary/closed-source? When I was first learning about Clojure I really wanted a native JVM compiler, but as a hobbyist I was not about to go all-in on an enterprise compiler. The only FOSS solution I found was GCJ.

That, and "the JVM startup is slow!" seems to be a popular argument against Clojure and Scala here on the Intertubes.

How about generating .so, callable from other environments, for example CPython?

This is on our roadmap, stay tuned.

Interesting question - I hadn't thought about compiling the JVM bytecode to machine code. It looks like there is a Java frontend for LLVM that achieves that: https://llvm.org/svn/llvm-project/java/trunk/docs/java-front...

One of the things that appeals to me about Scala Native is its interoperability with C, which I suspect is something that the LLVM Java frontend doesn't provide.

Doesn't the JVM have some kind of native FFI?

I suspect the built-in Java Native Interface (JNI) won't work when compiling bytecode to native code, since the JNI methods require a pointer to the JVM (passing a pointer to the JVM to C functions or calling methods on the JVM class in C++). Perhaps an FFI library like this could work, though: https://github.com/jnr/jnr-ffi .

Scala may be written in functional style, which is a poor match for JVM (e.g.: uglyness of `@tailrec` annotations, absence of mutually-recursive functions, JVMs GCs are not usually optimized for that many ephemeral objects, etc).

Native code is not much better as a compilation target for FP, but at least you don't have to fight through JVM constraints first (GHC compiles to pretty efficient native code).

I remember that shortly after??? Scala native was first announced, Kotlin native made the news. How do these two compare today? Which one is better suited to create native executables now?

The alternative Kotlin compilers are currently at the "proof of concept" stage, but aren't usable, or supported much by the community.

Scala's JavaScript compiler (http://www.scala-js.org/) on the other hand is production ready and supported by the whole ecosystem, with plenty of libraries being cross-compiled to both the JVM and JS.

Scala Native piggybacks on top of the work made for Scala.js — it's harder, due to not being able to piggyback on another platform this time, so they have to deal with things like concurrency or garbage collection, but it's progressing nicely.

Scala Native isn't production quality yet, but it's at that stage where it's fun to play with it and the Gitter channel has some friendly folks you can interact with: https://gitter.im/scala-native/scala-native

Any reason to use Scala native when Rust exists?

Garbage collection.

You can think of Scala Native as Go on steroids. We don’t try to compete with Rust at all.

I am not sure Akka already runs on Scala Native, but if/when it does, that would be my first reason.

The language is more powerful, Rust still doesn't have HKT support.

The maturity of the eco-system.

Not having to deal with the borrow checker.

In any case, I would say in such cases OCaml and Haskell are also good candidates.

Rust is in a different league altogether.

Scala, Haskell, OCaml are garbage collected languages. This makes them easier to use, but some problems are out of reach.

Embedded development, kernels, native apps requiring performance without frames dropped, real-time systems actually, including consumer stuff like decoding video, these have been the domain of C/C++.

Of course, there are projects out there using these languages, pushing the boundaries, like MirageOS (https://mirage.io/) which is built with OCaml. But those are like the exceptions confirming the rule.

Java is doing quite well in such domains, including real time control of battleship weapon systems and missile radar controls.

The ongoing NLL work proves ergonomics are still not there, and it doesn't cover all scenarios like internal references as done typically for callbacks in GUI widgets.

It will surely improve, and the best thing for polyglot developers is that we can easily pick the best tool for the job.

Powerful is a strange choice of word. Did you meant to say more expressive?

I feel like if anything, Rust sounds more powerful to me, greater performance, control over memory, low level unsafe access, etc.

Higher-kinded types (worth it for this alone), existing library ecosystem, better tool support, politics. I do hope Rust reaches a stage where it can a substitute for Scala, but without HKT it will never be as productive.

There is plenty of Scala developers that needs to write for example command line tools etc that benefit greatly from the startup times of a native application.

Still requires Java to build.

Yes, that is true but eventually this could be lifted. There is ongoing work getting the compiler compiled with Scala Native as well as other work related to pulling the code related to build out of the sbt build that requires the JVM. This could make a native build chain possible.

Those are great news, thank you!

Why is the transcript a PDF? It's unreadable on mobile. Do they think people are going to print it out, like Knuth reading his email?

Are you using a rooted Android phone? There are open PDF readers that you can flash as an apk into your rom image, if you’re trying to avoid spammy adware from the market. Apple supports PDF files on iPhones natively, by default.

there are various tools online to view PDFs without even downloading them. For example, google docs has one: https://docs.google.com/viewerng/viewer?url=https://software...

Reply on news.ycombinator.com