Scala Native 0.3.7 | Hacker News Reader

Scala Native 0.3.7

github.com

77 pointsjoshlemer52 comments

I really like (and work in) Scala, and Scala native is an interesting project.

However, its use cases seem to be encroached on all fronts.

If you want native, performance and don't need the JVM ecosystem, Rust and go both take very different but good approaches to that problem. I would have a hard time recommending Scala Native over Rust.

If you want browsers, Javascript is rapidly evolving and becoming a better lanuage. Typescript adds an excellent type system given the constraints its working with. Webassembly opens the door to use your own favorite. I would have a hard time recommending Scala.js over Typescript.

If you want JVM, Java is adopting traditional Scala killer features at a glacial pace, lambda's in 8, pattern matching in 11+ [1], Data/Case classes in ? [2], but adopting them nonetheless. Kotlin has succesfully performed a Blitzkrieg maneuver on Scala's "better Java" territory and has full support from Google in Android. It's also much more approachable for Java devs. I have a hard time convincing JVM devs that Scala is a good choise over Kotlin. At the place I work in all new projects are Kotlin, and Scala is used for the projects that were started in it.

Python and R are mainstays in the data science world, with Spark serving as a sort of Scala bastion.

I like scala, and follow its developments and 3.0 (Dotty) version somewhat, but I am slowly seeing it dwindling into irrelevance.

[1] http://openjdk.java.net/jeps/305 [2] http://cr.openjdk.java.net/~briangoetz/amber/datum.html

I could never give up higher-kinded types: there are plenty of ad-hoc solutions to various effects, but a single reusable solution[1] means you can build up libraries that not only handle all the common effects, but your own custom ones as well. Even if you don't use them directly, you benefit from the ecosystem they make possible. Few other languages let you use something like treelog[2], fewer still let you fold a custom tree type while managing an effect as nicely as with matryoshka. Typeclass derivation is reflection done right, with compile-time safety and corresponding performance. I've never seen another language that made it so easy to put a database transaction boundary in exactly the right place but in a type-safe way so that you can never forget, or even have transactions implicitly applied to web endpoints that need them in a 100% statically-compile-time-checked way, because your endpoint routing is just plain old code (the way it is in akka-http[4] or rho[5]). Hell, even when it comes to something as basic as result types: Kotlin or Typescript can't do them (nicely) at all, and Rust's can't offer nice syntax without a macro (or a magic trait in newer versions of the language).

There are plenty of things wrong with Scala - in many ways I'd like to see a clean replacement - but now that I think in HKT I could never go back to a language that doesn't have them.

[1] http://philipnilsson.github.io/Badness10k/escaping-hell-with...

[2] https://github.com/lancewalton/treelog

[3] https://github.com/slamdata/matryoshka

[4] https://doc.akka.io/docs/akka-http/10.0.4/scala/http/introdu...

[5] https://github.com/http4s/rho

> dwindling into irrelevance

Scala won't be the next Java. However, _some_ Scala-like language will be the next Scala, probably EPFL Dotty [0] or Twitter Reasonable Scala [1] Short term, also Lightbend Scala is evolving (collections redesign [2], macros EOL [3])

Just upgrading existing Scala projects will keep most Scala devs employed.

[0] https://github.com/lampepfl/dotty

[1] https://github.com/twitter/rsc

[2] https://news.ycombinator.com/item?id=16461022

[3] https://news.ycombinator.com/item?id=16519366

There's one case missing: Build a command-line tool with concise and maintainable code; Performance is not important.

Scala Native would win over Rust and Go, which are too low-level. Scala Native would win over python/ruby because of the ease of deployment (e.g. compile to target CPU) and static typing.

However, Crystal probably wins over Scala Native...

Go is probably a winner for the time being for CLIs, just the fact that I can cross compile for 3 platforms with a very simple script makes it very attractive.

Specifically it can cross compile to a static executable. It’s quite pleasant to just copy a compiled Go program and have it just work.

I agree. Golang is the winner at the moment because it supports compile to Mac, Linux, and Windows. The deployment of golang app is just so good (no extra installation) that I can ignore its lower level of abstraction.

Other native languages should be able to do the same in the near future (I hope).

Other native languages are able to do the same since ever.

I would rather use OCaml for it, there are even published books how to do those kind of tasks with OCaml.

https://ocaml.github.io/ocamlunix/

And it has an eco-system that goes all the way back to 1996.

I'm not familiar with OCaml. Its syntax looks like a high-level language (which is good).

Can we compile code into a single binary and run it on a user's machine without any extra installation? Does it work with Windows?

Yes, all languages in the ML family compile to native code.

OCaml can also compile to bytecode, usually used on the REPL or just to speed up the development cycle.

Your single binary might depend on glibc by default. If you want a pure static library then you need to customize the toolchain to use another C library like musl.

And yes, it does work on Windows.

I think that the Docker CLI bring written in go is some pretty big counter evidence to saying go is to low level to write CLIs. I can't think of a single reason that would be true.

I wasn't clear about "low-level". I mean "lower level of abstraction". Lower level of abstraction often means higher amount of detail programmers have to take care of.

For example:

1. Rust is on a lower level of abstraction than Go is. Because, in Rust, when we use strings we have to think whether we should use `String` or `&str`. In many types of applications, we just want a string. We don't care which type of a string we need. It's too much detail.

2. Golang is on a lower level of abstraction than, say, Ruby on the aspect of error handling. Maybe, for a command line tool, I don't want to explicitly handle exception on every statement. I just want it to exit if an error occur. It's too much detail.

3. Python can be thought of as being on a lower level of abstraction than Ruby. For example, in Python, there are 3 ways of getting the first element of an array or null (https://stackoverflow.com/questions/363944/python-idiom-to-r..., well, which one is better?) In Ruby, the function is already encapsulated in main library (e.g. `.first`), and people don't have to care of this kind of detail.

Actually, regardless of the endless discussions we might have about its features, I would say that thanks to Docker and Kubernetes, it already won the hearts of many sysadmins.

> I would have a hard time recommending Scala Native over Rust.

Why? Scala is so much nicer.

> I would have a hard time recommending Scala.js over Typescript.

For this too, Scala is so much nicer. The base libraries are far better than in both Rust and TS.

> I have a hard time convincing JVM devs that Scala is a good choise over Kotlin. At the place I work in all new projects are Kotlin, and Scala is used for the projects that were started in it.

That sounds strange. I know Scala's got some warts and all, bit I still can't imagine Scala devs wanting to downgrade to Kotlin.

> Rust

The type system is very good. Denys Shabalin is doing amazing work but Scala Native is still an early mostly 1-man project and not production ready, rust is a production-ready language with a large number of backers. The rust ecosystem is much larger than the Scala native ecosystem.

> Typescript

Types solve my biggest gripe with Javascript. It's singleton and union type support is leagues ahead of Scala and Dotty and a decent patch for some of its omissions. It's a much closer fit to browsers. The ecosystem is gigantic. Frontend devs know Javascript, you can convince them to learn Typescript but not Scala and the ramp-up time on Typescript is super low.

> Switching to Kotlin

The Scala devs aren't happy with it, but ultimately it's about the project not the language. It's mostly new hires and previous Java devs that have a negative impression of Scala and prefer Kotlin.

> It's singleton and union type support is leagues ahead of Scala and Dotty.

What? I don't understand it's better than dotty which has a proven core? Typescript's type system wasn't (isn't? I don't know) even sound.

> Java devs that have a negative impression of Scala and prefer Kotlin

To each their own but if you learn Scala you can never go back to Java/Kotlin, there is not even a contest.

Edit: newlines added

Typescript is incredibly unsound and at the same time incredibly useful. It rejects the notion that types are just for hard correctness guarantees and focuses on tooling.

So 'tooling' is it's real selling point? Wow... Last I checked it still used the same mean arrays and dicts, too. Scala's default libraries blow/blew it out of the water.

That's not what tooling means. What you are talking about are called "libraries", which, using the same libraries with better tooling, is one of the core strengths of TypeScript.

Those were 2 separate thoughts. I was not conflating tooling with libraries. And I'm still amazed that you seem to think tooling would be a main selling point or 'core strength' for it.

Yes. The compiler catches most type errors (depending on how faithful you are to the type system), and code completion and refactoring are great also.

You should really check out how to do ADT's in TS. It's incredibly simple and intuituve thanks to the union and Singleton support.

    type Shape = Point | Circle
    interface Point {
      type: "Point"
      x: number
    }
    interface Circle {
      type: "Circle"
      center: number
    }

    let x: Shape = foo()
    if(x.type == "Circle")
      log(x.x) // Error

It's enterily generic and flows through the whole type system. It makes nullability trivial and `Option` superfluous, since `Option<T>` is an alias for `T | null`. If you check it for null once the type will immediately become `T` from then on.

Try something like this with Dotty's enums, it's super easy to hit the limits there.

“Option<T>” is not an alias for “T | null”, which is apparent when you try to puzzle out how to represent “Option<Option<T>>”.

Indeed, and this is exactly why union types are not a replacement for algebraic data types. (Not disagreeing, just making the point more explicit.)

You've already hit a limit when you're having to pepper your code with C style union type checks, imo.

You don't get it, you don't have to pepper the code. If you already have a `Circle` the check is unneeded. If you have done the check the type is constrained. You do exactly as much or less checks as you would do pattern matches on sealed traits in Scala, except that it works as you except in all instances, not just with sealed traits.

> You don't get it, you don't have to pepper the code. If you already have a `Circle` the check is unneeded.

That's not even what I was talking about...

You're the hero we deserve.

But as a commercial choice, the depth of available talent with JS experience vs Scala just makes Scala.js an absolute non-starter. Even if you found someone they will be potentially double the salary. So you're stuck cross training developers to write something they could have done anyway with less cognitive load.

I admire what scala.js is and I've dabbled with it for my startup for my own indulgence but it's window for viability is tiny.

What makes scala "nicer"? Why is Kotlin a "downgrade"? Are you possibly just giving your opinion?

> Are you possibly just giving your opinion?

Well yes, in the same way everyone here is... But no, if you look at the features of the languages there's a clear difference.

Pattern matching.

Higher kinded types.

I've heard the standard C++ criticism that everyone programs in a different subset of the language leveled at Scala. How true do you think that is?

I have no experience with C++, so don't know how it compares.

There is some truth in the statement, in that a Scala team will often gradually adapt more advanced features of the language. I wouldn't call them different languages, but rather say Scala is a "Java N+5" in that sense. Java 8 is objectively much more advanced than Java 4, but it's very much the same language and a Java 4 dev will get used to and see value in Java 8's new features. Same with Scala .

The main schism is what some people jokingly call the "compiling Haskell with scalac" crowd. They are hardcore functional programmers who value FP ideals like category theory over all else. There even is a forked Scala compiler [1] (luckily the fork and mainline reconciled, and now every change in the fork is a PR for mainline). The crowd is mostly exemplified by libraries like ScalaZ [2] and to a lesser extent Cats [3] and shapeless [4]. Some even advocate replacing the whole Scala standard library with their own. Can you at that point still say you're using Scala? You can probably read that I do not personally like that style, and consider it a detriment to the community†, so consider my statements in light of that opinion.

The main reason why I don't like the style is that Scala is not powerful enough when compared to Haskell or Julia, requiring lots of magic and boilerplate to make it work. Maybe this will ultimately be resolved, as there have been gradual improvements over versions, and this will be the final niche Scala will occupy long-term, as I have not seen any contenders yet.

† I have tremendous respect for the maintainers of the library and their skills, but I don't think the schism is healthy.

[1] https://typelevel.org/scala/

[2] https://github.com/scalaz/scalaz

[3] https://github.com/typelevel/cats

[4] https://github.com/milessabin/shapeless

As a maintainer of one of the aforementioned libraries, I think the amount of dogma the proponents of them have is a bit overblown. :)

At Verizon I worked in projects that were pure FP/heavy Scalaz, as well as working in scala projects that were basically java++.

Both of them were projects I had architectural input on, but depending on the teams comfort level, we decided upon what style of Scala would be used.

The language is deeper than some others, and while a java programmer might be able to pick up python in a few days and vice versa, there's definitely going to be more of a learning curve to pick up 'FP style Scala'. However, we had people who had no prior Scala experience come in and dive in successfully to even our most hardcore FP projects. As long as you are OK with a bit of ramp up, the disparity between various styles of Scala are a bit exaggerated in my experience.

Curious why you brought up Julia here? We really don't seem to have the dogmatic FP advocates you're describing here, and the language is much more oriented to an imperative style than a functional style anyway.

The irony is that can be applied to any multi-paradigm language, but since many are afraid of C++, only it gets the fame.

To pick on my favourite scripting language, Python, you can go from "I just learned to print hello world" style up to "look at me, master of meta-programming with multiple inheritance and dynamic code generation".

And this not counting all the changes that have happened throught all Python releases, with semantic impact how the code gets written.

Yet many assume it is a "simple" language.

very true

I would also throw in Haskell and OCaml in the native front.

And one can use Kotlin with Apache Spark, e.g.:

http://tomstechnicalblog.blogspot.hk/2016/11/using-kotlin-la...

Kotlin == (Scala syntax) - (Scala power) + (Groovy wackness)

Kotlin's the latest in a long series of leading alternative languages for the JVM, each having a turn taking second spot behind Java. First there was Beanshell adding syntactic shortcuts for property boilerplate, then Apache Groovy came adding closures, then Scala took the adoption lead adding full static typing and heavy functional capabilities, and now Kotlin's come into second JVM spot with simplicity by removing cruft. So think of Kotlin as "alternative JVM language Two point Oh".

> and now Kotlin's come into second JVM spot

Not according to Tiobe.

Does Kotlin have anything approaching Akka or akka-streams?

Akka has a Java API:

https://doc.akka.io/docs/akka/2.0/java/index.html

Not sure about the relative ergonomics of Akka-in-Scala versus Akka-in-Kotlin (via the Java API).

I mean...you can just use Akka in it.

Had my first experience playing with Scala Native a couple of weeks ago, for hacking together a simple vote-counter (https://github.com/tunesmith/condorcet-counter) and am enjoying it just because I'm able to do all the scala goodness I want (test cases, types), while also be able to run it quickly from the shell.

I couldn't figure out the IntelliJ tooling, though, that's still lagging. I have to turn off the scala-native plugin to get the IDE to recognize the code, and then turn it back on whenever I want to generate the executable.

Out of interest, how will Scala Native work with projects like Apache Spark, which rely on shipping JARs around in order to run the same code on multiple nodes? Would you build "native" JARs containing object code?

I don't think Scala Native would be a good fit with Apache Spark. Starting with their work in 2.0, the Spark framework already started doing things like working with sun.misc.Unsafe for direct memory control over certain operations. In general they put a lot of effort into getting as close to metal and hyper-optimizing the framework itself. Given that, trying to do the same yourself on top of that framework seems counter-intuitive in my opinion.

Surely a goal with Scala Native is performance and things like "direct memory control" (why else move away from the JVM?), which would very much dovetail with the work being put into Spark?

The JVM has had native-like throughput for long-running batch processing jobs for years - of course you can always tune further, but it already fits nicely with what Spark is used for. Scala Native makes more sense for things like command-line utilities that need to start up quickly, or in the future maybe real-time systems that need to avoid GC.

Reply on news.ycombinator.com