Hacker News Reader

Comment by azinman2 | Hacker News Reader

I'm curious to know the target user. There's no JVM interop, so one of the main reasons to use Scala -- an evolution of Java the language -- is out. Plus with Kotlin even that reason is being outdated, especially as you lose all the type conversions between Java arrays and Scala arrays, etc. Their C struct example has a lot of awkward syntax, so it seems that it's a bit 'bolted on' versus inherently designed for this use case. We have both Swift and Rust as great higher-level languages that compile to native applications with C interop, with more robust ecosystems and faster progress (Scala moves at glacial paces in comparison)... both appealing for different use cases.

I used to be a Scala enthusiast before these alternatives existed. What place does Scala now have in the world?

View on HN

For me Scala was never about using a Java++.

If you want Java++, the best Java++ is still Java with a bunch of libraries. You don't need a new language for that.

Kotlin's succes is effectively because Google was late to port Java 8 to Android. A fine language btw, but it will either become more like Scala, or it will wither away eventually. My bet is in it becoming more like Scala, as evidenced by arrow-kt.io

Scala has a blend of features that makes it the most expressive mainstream functional programming language, most importantly ... implicits, by which you can model typeclasses and higher kinded types.

And in Scala we've got an active community of FP libraries. Actual FP, emphasizing referential transparency, with abstractions built for suspending effects.

It's a little known fact but the kind of FP programming we do in Scala tends to be more pure than what happens in OCaml, F#, Erlang, Clojure and others.

Also, we have a Scala to Javascript compiler, all the tooling works with it and we've got an ecosystem of libraries cross-compiled to Javascript.

As for Scala Native, it has its place. The problem has been lack of manpower to bring it to the maturity that Scala.js is at.

> It's a little known fact but the kind of FP programming we do in Scala tends to be more pure than what happens in OCaml, F#, Erlang, Clojure and others.

I have only dabbled in Scala, but IMHO it's more suited as an OCaml with first-class modules. Why not use Haskell, if you don't need the JVM?

Compared with OCaml, Scala can express type classes and has higher-kinded types. Many abstractions that are easy to express in Scala are in fact hard to express in OCaml.

For-comprehensions are worse than Haskell's do-notation or compared with F#'s computation expressions, but the list ends there. In practice they work well and at least Scala has monadic for-comprehensions. How many other languages can claim that?

As for performance, I encourage you to take a look at the state of the art. Working with our IO Monads has better performance than many abstractions that people now take for granted, like most Future/Promise implementations.

I build systems that are super stressed by incoming traffic and Scala's performance is awesome. Worse than bare metal Java of course, but when you compare it with high level Java frameworks or with Python/Ruby/Javascript/what have you, you'd actually be surprised by how better Scala FP is ;-)

As for Haskell, I like the language, but it's super fragmented, everyone has its own Prelude and favorite language extensions, extensions that are sometimes in conflict with other extensions and the tooling is a mess. Also laziness is cool, but performance gets hard to predict. We have the same problem in Scala when we are using lazy abstractions actually, the difference being that Scala is opt-in instead of opt-out.

With Scala, even if you're targeting JS instead of the JVM, you still use the same tools.

And I even like the language better. Scala 3 will fix many annoyances I have, including adding untagged unions and GADTs that are based on ADTs, in other words nicer than Haskell.

PureScript I would like though, as it fixes some of Haskell's problems, maybe one day it won't be limited to JS.

> As for Haskell, I like the language, but it's super fragmented, everyone has its own Prelude and favorite language extensions, extensions that are sometimes in conflict with other extensions and the tooling is a mess. Also laziness is cool, but performance gets hard to predict. We have the same problem in Scala when we are using lazy abstractions actually, the difference being that Scala is opt-in instead of opt-out.

Not an expert in Haskell, nor Scala - only dabbling. However, the above could be repeated verbatim after replacing Haskell with Scala, couldn't it? You can write Java++, Haskell-- or any combination in between. Use one of the several FP stacks or build tools.

I think both Scala, Haskell and C++ can compete for the name of the most complicated, fragmented and diverse language and ecosystem in existence.

I think people are too quick to judge languages.

If I tell you that Haskell has a fragmentation problem, I feel good about telling you that because I actually have experience with it.

However Haskell is still a language that I'd pick over Java, Go, JavaScript or any of the mainstream languages for that matter and I'd be willing to put my reputation on the line for that choice.

I also worked with C++. To see people compare Scala and Haskell with C++, kind of rubs me the wrong way.

And unfortunately such opinions stay on the Internet forever, are picked up by beginners and repeated verbatim.

So no, there's a huge difference between Scala, Haskell and C++. For one the former wouldn't lead to stupid and disastrous bugs like Heartbleed, whereas C++ can and routinely does. By comparison when well typed Scala or Haskell programs compile, the defect rate tends to be really low.

I'm seriously thinking about starting a course in how badly and easily you can fuck up in C/C++, for people that make unfair comparissons with them, making life hard for their managers :-)

Also just to get this out of the way... the problem in Scala are actually the Java refugees going gung-ho on OOP. And the FP people, because they tend to be more experienced, are the ones cleaning up the mess.

And I like that. You know why? Because Scala makes it possible to introduce FP gradually, without an organization wide commitment. In the company I'm working for, a language like Haskell would never fly, but Scala does.

Also in case you haven't noticed, FP is spreading.

- in Typescript/JS you have fantasy-land and fp-js

- in Kotlin you have Arrow

The difference is that you don't get the nice type system to support those abstractions well, so you also get the weird HKT and type class encodings to go with it. Although in fairness the Arrow guys are working on compiler plugins for better supporting type classes and the HKT encoding that they use.

So if you're conservative about adopting FP, language choice will not really protect you ;-)

Thanks for the answer.

To be clear, I wasn't comparing languages in terms of correctness of code you could potentially produce. Of course C++ is the worst on this metric. However, it has its own niche in low-latency setting where I can't see anything else replacing C++ any time soon.

"So if you're conservative about adopting FP, language choice will not really protect you ;-)" Well, there is always Go and Cobol

Otherwise, exactly on the target. I hear java is getting continuations and tail call elimination soon

Haskell and Scala share complicated and fragmented programming styles. But Haskell has additional dimensions of complications (and thus fragmentation) due to:

1) Many compiler-option language extensions: https://downloads.haskell.org/~ghc/8.2.2/docs/html/users_gui... --- these add new language syntax and deep features. They radically change the meaning of the code you're reading.

2) Multiple competing standard library collections ("preludes"): https://guide.aelve.com/haskell/alternative-preludes-zr69k1h...

3) Haskell's language spec ceased development a decade or so ago. So the above diversions are never rolled back into the language. Instead, all forward development to Haskell is done in the form of new compiler options.

I'm not too familiar with Scala, but I'm pretty sure it doesn't suffer from these problems.

There are even more sources of Haskell fragmentation, such as many competing methods of error handling, input validation, and data records.

(I'm a Haskell user who wishes this all weren't the case.)

re: “ As for Haskell, I like the language, but it's super fragmented, everyone has its own Prelude and favorite language extensions....”

I really like Haskell, but this aspect of the use of Haskell also bothers me. I use a small subset of Haskell and language extensions, a small subset that I am happy and productive with, but it takes me a lot of effort to understand other people’s code.

> everyone has its own Prelude

In my Haskell experience (including some professional experience) this isn't the case very often at all (maybe not even as often as it should be).

(Which is not to say that there aren't some problems with fragmentation elsewhere in Haskell's ecosystem)

> extensions that are sometimes in conflict with other extensions

Could you name two widely-used extensions that conflict with each other in a non-trivial way? I've used Haskell for several years and I've never come across any.

Regarding purescript, I thought I saw a post on the other day that said it now has C++ and Go backends.

Thanks for the detailed reply!

> It's a little known fact but the kind of FP programming we do in Scala tends to be more pure than what happens in OCaml, F#, Erlang, Clojure and others.

What do you mean by pure exactly? IO Monad? Simply no side effects?

Yes. Many libraries in the FP Scala community are concerned with modeling effects in a pure way.

Oh interesting view. It's funny since Scala's author himself doesn't consider the IO monad to be "pure".

> “The IO monad does not make a function pure. It just makes it obvious that it’s impure.”

https://groups.google.com/forum/#!topic/scala-debate/xYlUlQA...

It's a pure function because it doesn't do any IO. It returns an action that, when run, performs the IO. Because it's a pure function, we have referential transparency, we can safely replace the function call with its computed value (the action) anywhere in the program.

In a typical IO function, the first 10% of the code returns a monad-bind that holds the remaining 90% of the code in a closure. The pieces are pure, but the whole isn't meaningfully pure.

A function returning IO preserves referential transparency.

    fn() + fn()

    // ----

    val a = fn()
    a + a

Are the two programs above equivalent? For a function returning IO, they are. For a function that's side effectful, they are not.

(As an exercise for the reader, think of our function here returning Future/Promise instead of an IO, how would that change the program?)

This is a definition of purity that you can work with. The trap that people fall in is to talk about fiction essentially. What is "meaningful" anyway? If you don't give it a definition, is it meaningful to talk about what's meaningful?

I agree that a function returning IO is opaque. You can't reinterpret it, it's no longer a plan that you can change the interpreter for, but something concrete. There are of course better abstractions for dealing with effects in a way as to make them less opaque ... like the Free Monad, or Eff.

Many of them have been expressed in Scala as well.

Purity is not a goal in itself, its purpose (its meaning) is to help me answer useful questions about the behaviour of the program.

In a typical IO function, `fn()` only gives me 10% of the code in `fn`: the part up to the first bind. To get the rest, I need to `do` it, and this is where purity is no longer meaningful:

    a <- fn "a"
    b <- fn "b"

Can I swap these two lines ? The IO monad tells me there might be consequences, but what are they ? Another example:

    foo fn

Will `foo` trigger the IO effects of `fn` zero, one or more times ?

I believe these are meaningful questions, and yet, a program built from pure functions can answer them no better than an imperative program.

> "Purity is not a goal in itself"

Agreed. However having referential transparency for IO does help. Let me explain...

The program you mentioned has one clear property: those two computations will execute sequentially.

This sequence of executing operations becomes independent of evaluation via IO. And if you want parallelism, you need to make it explicit.

    val a = fn(1)
    val b = fn(2)

    for {
      r1 <- a
      r2 <- b
    } yield r1 + r2

For IO this always has the same execution characteristic. "B" will execute after "A". You have a clear happens-before relationship.

As an exercise:

1. think of what happens if that function returns a Future/Promise

2. think of what happens if that function is synchronous, but still side effectful

In both cases the actual order of execution is not guaranteed. In the second case the ordering is only guaranteed from the point of view of the current thread only. And this is interesting b/c the only way to force ordering in a multi-threaded context, besides memory barriers which are a runtime trick, is to create a data dependence. In other words, while this code doesn't guarantee visibility with IO, it does have stronger ordering guarantees than any side effectful code you can throw in there.

You're a little unfair here btw. If the two operations here are not dependent on each other, then the Monad's flatMap/bind isn't the operation you want, since Monads literally describe sequencing. And you want sequencing in case order of execution does matter.

In other words, when you see a sequence described via bind/flatMap, you have to assume that the order is relevant, because the author could've done this instead, which is self explanatory:

    (a, b).parMapN { (r1, r2) =>
       r1 + r2
    }

And now they are executed in parallel, with no related evaluation gotchas (exposing the API in the Cats library here).

So to say that IO doesn't give you any useful property here is factually untrue. The obvious gain is that you look at an expression and know exactly how and when it will execute.

I encourage you to learn more about IO:

https://typelevel.org/cats-effect/datatypes/io.html

Also see my comment here:

https://news.ycombinator.com/item?id=20744549

"implicits" seems a polarizing feature

I've been using Scala Native for two years now, and I find that it's C interop is state-of-the-art. The way I would pitch it is: Scala Native gives you the capabilities and performance of C, embedded in a Scala DSL.

In particular, pointer arithmetic and memory allocation feel really smooth, and struct syntax is getting reworked for the upcoming SN 0.4 release.

The affinity with C, and with systems programming in general, isn't something you necessarily get with Swift or OCaml to the same extent. And even Rust, although it is outstanding for some systems tasks, isn't 100% mature on concurrency and async use cases yet.

In contrast, Scala Native works really nicely for low-level concurrent programming - I'm writing a book about SN for Pragmatic at the moment [1], and the entire second half is about building up a fully concurrent backend service framework, from scratch, using C libraries like libuv and libcurl. We're also preparing to release an official libuv binding for Scala Native 0.4 [2]

1: https://pragprog.com/book/rwscala/modern-systems-programming...

2: https://github.com/scala-native/scala-native-loop

Scala is great for statically typed functional programming. It can be as pure as Haskell, but with the escape hatch of not requiring that purity at the language level. With Scala 3 coming out next year it will become more approachable to developers that find Haskell and OCaml off putting (and has already solved the "python 3 problem"!).

As for scala-native, it gets past the major draw backs of the JVM: startup speed, memory size, and Oracle. From applets in 2002 to kubernetes sidecars and aws lambda today, the JVM refuses to optimize for these things.

What a joke.

Currently Scala.Native still performs poorly than Scala JVM.

Since 2000 anyone that cared about AOT in Java had quite a few commercial Java vendors that offered AOT on their JDKs, Oracle and IBM were two of them.

Project Panama, Valhalla, Metropolis and Graal are proof that Java community does care to optimize for these things.

> Currently Scala.Native still performs poorly than Scala JVM.

This is factually incorrect [1]. 0.4.0 series is shipping with brand new optimizer (as of M1) and soon a parallel garbage collector (M3). Both contribute to major performance overhaul compared to the current stable 0.3 branch (see benchmarks on the link below).

[1] https://github.com/scala-native/scala-native/releases/tag/v0...

Benchmarks against all major Java implementations?

So developers can choose to have the batteries included or to pay a vendor. I hear signing contracts with IBM and Oracle is a great experience. Is that the joke?

> proof that Java community does care

Oh, the community wants them. It's been Sun then Oracle and now the OpenJDK project that has never prioritized these things and that's the runtime 99% of JVM shops run.

The sad joke, when comparing with other professions, is wanting to be payed while refusing to pay others for the tooling one uses.

I happily pay for a number of excellent tools. I also have no interest in doing business with IBM and Oracle again.

My take as a continuing regular Scala programmer:

Scala's position in the standard JVM world hasn't changed too much. Kotlin has taken up some of the "better Java" part of Scala, but otherwise Scala still maintains its niche of statically typed FP (mixed in with OO) on the JVM.

Scala on Android has withered (Kotlin really took off here). Scala Native unfortunately also has kind of withered too, but GraalVM picks up the slack there in allowing for native compilation.

Scala.js (compiling Scala to Javascript) has become surprisingly mature.

> What place does Scala now have in the world?

serious answer: chisel3

other serious answer (at least to me): the ability to write incredibly terse code. For instance Rust, as great as it is, doesn’t have partial application.

    iter.map(|x, y| f(x.as_ref(), y))?

is so much less satisfying than

    iter.map(f(_, _))

I’m pretty sure you could recover that behavior using a compiler plugin, though.

Also, don’t get me started on implicit conversions... I’m an avid golfer.

Other than not having (usable) macros, the language itself has very nice amenities. SBT was like an early Cargo, even.

I lost so many weeks trying to decode the fk of the code written in scala with so much "magic" in it. And I'm working with a colleague to "unmagic" it in our source base.

Yeah that's exactly my feeling on Scala after debugging some "magic" code, it feels like a functional perl where you write whatever you want and good luck if someone else will need to read it later.

Spoken from true experience, I've also been staring down at some Scala magic and wondering how in the earth did we end up in this hole.

I personally believe that people who bring functional programming into commercial settings are wrong to do so. There's a real trend lately to go "big" and "functional", but know that it causes actual business problems.

I understand that all of these tools have their uses, but the most important thing is ease of finding decent developers. Lots of people can do JavaScript well, but few can do Scala, so you're going to have a smaller pool of talent. More popular languages are also going to have more developed patterns, more native libraries, and better tooling options.

If you need Scala, please just write Java. You'll always be able to hire a Java dev. Everyone hates writing Java, but everyone can write Java.

And related, if you're using any kind of NoSQL for your business data, you're wrong. I understand NoSQL for caching and document storage, but not everything is a document. Most things are actually best in SQL tables. Especially if you're a startup.

>I understand that all of these tools have their uses, but the most important thing is ease of finding decent developers. Lots of people can do JavaScript well, but few can do Scala, so you're going to have a smaller pool of talent.

Isn't that a side benefit? It keeps out the riffraff.

I don't think it's fair to include F# in that list, what with its emphasis on explicitness. After all, it's not like functional programming requires writing obscure code any more than complex object hierarchies pushing events and callbacks all over the place until you have no idea how you got where you are.

What actually causes trouble in an enterprise setting is not any particular paradigm itself, but "magic." All that flexibility of Scala or Lisp is great in the small, but it doesn't scale easily. You are effectively building new DSLs in them, and then everyone that comes to the project has to first learn the specific variation that you ended up with.

The problem is that people usually neither acknowledge they are building a language, nor think about design of that language to make it easy to learn and use, including documentation. So new people come and are completely lost. If they are experienced in the given base language, it is only easier for them to find the starting point to reverse engineer what was done, but that work still has to be done.

This actually happens in Java and other less "magic" languages as well, but then nobody blames the language: they blame frameworks and development practices in a particular shop.

Additionally, the whole mindset of "easier to hire a Java dev" only makes sense for juniors, if at all. Developers are meant to be problem solvers, and any particular language or framework is but a small part of the overall toolset they should have. Sure, if they are experienced in a particular language that you are using, they will have easier time starting up, but that is only a short term benefit.

As a non developer, but someone who codes, the hire for Java mindset always confused me. If I can pickup an entirely new language (with the exception of Haskell) in a few days, so should a professional. Then, I noticed how low the bar is for an average developer when it comes to understanding their tooling, basic commandline knowledge, and problem solving. Sure, the best developers can solve a business problem with pretty much anything if they have to, but the average one chooses a major language and framework and hacks things together with stack overflow. We've all done that before, but then again good problem solvers are also capable of picking up a book and learning something new like Lisp and a lot of developers simply aren't going to do that ever.

The average developer cannot pick up an entirely new language in a few days. Maybe I'm incompetent, but it takes me a few months before I start to feel comfortable with a new language and its ecosystem, and a year or two before I feel like I really understand the nuances and know just where to look when something goes wrong.

I can easily find someone with 5+ years of full time Java development (or JS, Python, and Ruby), and that means they're going to come fully equipped to dive into a problem, using familiar patterns and libraries. I don't want to hire people to make them learn Scala as quickly as possible and hope that they're able to figure it out very quickly.

> the average one chooses a major language and framework and hacks things together with stack overflow

Call me average, but I spend most of the day googling and reading results on SO. I think most developers do the same. Even if I already know how to do something, I usually google it to see if there's a better way to write it. If you're working with Scala (or some other less popular language), you're going to have less results doing these things, because less people are going to have hit your special problem. There is no amount of amazing language features that make this trade-off worth it.

And it's not just true for languages, it's true for frameworks too. Using common stuff (language choice, framework choice, library choice, compiler choice, etc) makes it so much easier to solve the problems you'll inevitably run into.

> The average developer cannot pick up an entirely new language in a few days. Maybe I'm incompetent, but it takes me a few months before I start to feel comfortable with a new language and its ecosystem, and a year or two before I feel like I really understand the nuances and know just where to look when something goes wrong.

I have to agree that in most cases few days are not realistic, but I'm convinced you could apply Pareto principle and state that you can get most things done with knowing just enough of the language.

A year or two to understand nuances sounds like not making a deliberate effort to learn about them and instead just waiting until you get that through osmosis by just working with the language. It works, but it's slower than deliberate effort to identify differences from what you know already and learning specifically about them.

At any rate, arguments like that neglect that most languages are very similar, and after learning a few quite different ones the differences with most others become largely superficial.

> I can easily find someone with 5+ years of full time Java development (or JS, Python, and Ruby), and that means they're going to come fully equipped to dive into a problem, using familiar patterns and libraries.

Yes, and no. You assume that all people using the same language use similar patterns and libraries. Assuming recent experience with Python or Ruby that might be close enough to true in those languages, but I heard that in case of JS the ecosystem is incredibly diverse.

Setting aside any particular language, any sufficiently large codebase will have its own idiosyncrasies that add to the entry barrier.

> I don't want to hire people to make them learn Scala as quickly as possible and hope that they're able to figure it out very quickly.

You seem to be optimising for "very" short time horizon. It strikes me as short-sighted. You also seem to underestimate how much knowing the JVM as the runtime environment gives you.

> If you're working with Scala (or some other less popular language), you're going to have less results doing these things, because less people are going to have hit your special problem.

There will of course be fewer results, but you seem to assume that has to mean you will regularly run into trouble. Numerically, most Google results are bogus or even misguided anyway. You might try to argue that having more people using given technology increases the odds of finding a good solution, but that requires relatively stable signal to noise ratio across different technologies. I don't think that is necessarily the case.

Additionally, Scala might be poor example for this one, as it is actually used quite a lot in industry. It is nowhere near as esoteric as Haskell, for example.

> There is no amount of amazing language features that make this trade-off worth it.

This is a very dogmatic statement that I strongly disagree with. It is ironic to read it on Hacker News, given the emphasis from Paul Graham on how much velocity he gained by starting his career (were those e-shops?) in Lisp and being able to iterate much more quickly because of that. It is especially ironic given your focus on short term earlier in the post.

> And it's not just true for languages, it's true for frameworks too. Using common stuff (language choice, framework choice, library choice, compiler choice, etc) makes it so much easier to solve the problems you'll inevitably run into.

Straying from the common path should be weighed carefully for sure. But never forget that if you are far enough from the lowest common denominator for which it has to optimise, you might actually get way more mileage by using something else. Right tool for the job and all that.

"I personally believe that people who bring functional programming into commercial settings are wrong to do so. There's a real trend lately to go "big" and "functional", but know that it causes actual business problems" I can certainly see where you are coming from, given how you are lamping up FP with distributed systems. There are a lot of packaged "solutions" out there that over complicated to be introduced all at once.

> everyone can write java

I can almost guarantee you they cannot.

From my recollection working with Scala many years ago... Scala is in some ways like C++, it gives a grab bag of cleverness. You can shoot yourself in the foot going wild with template metaprogramming and excessive abstraction and cleverness in C++, or go wild with FP abstraction in Scala.

Or you can define a sane coding standard and subset that your team agrees is readable, and stick with it, and enforce it through code review.

> you lose all the type conversions between Java arrays and Scala arrays

This is what killed scala to me. It talks up a big story about Java interop, but the reality is that the separate collections really drive a wedge between the scala ecosystem and the rest of the JVM world so it does not have the kind of seamless iterop that Kotlin and (especially) Groovy do.

I have my share of grievances against Scala, but the difference in collection libraries only feels painful if your app can't tolerate the performance hit of re-copying a collection (which to be fair does sometimes occur on hot codepaths). Otherwise it's a simple .toScala or .toJava call away.

Also arrays are one place where Scala and Java do happen to share the same collection.

A good example of why platform languages always reign on their turf.

It's place is when you want a functional JVM language and would prefer not to use Clojure.

As you accurately pointed out for a "better" Java you probably want Kotlin - which has better compatibility with Java, is supported by big players (JetBrains, Google) and because of being the blessed language for Android is growing in popularity very fast.

> It's place is when you want a functional JVM language and would prefer not to use Clojure.

I think GP meant the place of Scala Native, where being a JVM language is irrelevant.

Scala is suited best for those who like to do FP in an enterprise environment.

With Cats or ScalaZ you can get Haskell-like syntax, and the advantage over Haskell is that Scala runs in the JVM and it provides interop with Java libraries.

But this is Scala native we’re talking here. No JVM.

Reply on news.ycombinator.com