The Road to Scala 3
scala-lang.org
scala-lang.org
For all the complaints about the language that always pop up in these threads: yes, the language lets you shoot yourself in the foot (with great power comes people who don’t apply it responsibly), but it’s precisely that power that makes it so useful and exceptional when judiciously applied.
I like to say that scala is as if java and ruby had a love child and it all worked out. Unfortunately, that does mean that java people will feel it’s too fast & loose, and the ruby people will feel stymied by its static typing. But I just love that where other statically typed languages generally say “no”, scala says “well, I trust you know what you’re doing...”.
The only substantive tool complaint I have these days isn’t with compiler speed - it chugs through the codebase plenty fast - but with tool startup time. Hopefully the more recent java versions and maybe some native packaging will eventually help with that.
For what it’s worth, in my own journey learning Scala, I simply didn’t have the luxury to blame, and I just forced myself to try and figure out how other people, like the finagle team for example, could be so productive with such an obviously newfangled invention, until I realized… I was wrong.
Scala is actually a wonderful language! People move companies to continue working in Scala! In fact the ideas and lessons that I’ve learned from rethinking how I used to do things are so unfathomably valuable to me that I don’t think I will ever look at code the same way again.
And before you downvote me please don’t mistake my words for criticism. I truly empathize with how some of the people here may feel and I’m very happy to go into specifics if anyone requests that. I don’t write on HN a lot because a lot of times I think there is just this therapy - or a version of therapy - that amounts to people reinforcing their need to blame things in order to continue excusing the poor performance, and I don’t want to get caught up in it or be made to feel bad for encouraging others to reconsider their virtues.
And by the way we all, at times, have poor performance. (Anyway, I dictated this into my phone, so also excuse any weird words or punctuation. I love you all. Some typos were subsequently edited)
Yah, I have noticed lately a division between two types of programmers.
The first is the "infrastructure oriented" programmer. They assume that the rank and file programmers will be expected to write bad code. They look at problems in the codebase as a sign that there is a missing piece of infrastructure. They 1) wait for the pain level to get high enough on the team, 2) identify a class of problems that is not getting solved well, 3) install or implement a new service/middleware/library that solves that class of problem elegantly, and 4) port all of the crappy code over to the new reality.
The second is the "pattern oriented" programmer. They assume that sloppy code will create areas of rot around it that become cost centers, and that "low hanging fruit" cleanup tasks will generally pay for themselves. When there are problems in the codebase, they go back and refactor. They advocate for team discussions of patterns, establish precedent for the team to follow, and expect other members of the team to learn and apply those patterns correctly.
I'm not really sure whether these two types of programmers should work together. Each group's primary assumption:
1) Programmers will inevitably do sloppy things, and'
2) "Low-hanging cleanup" pays for itself
... is correct in practice. The problem is they are somewhat at odds in terms of pointing to a high level strategy.
If you are committed to ripping out and replacing large parts of the codebase when they become unworkable, there's not really any point to trying to keep up with tidying.
And if you tidy, you won't really need to rip out large parts of the codebase.
So maybe we should just each try to find a team that fits with our values, rather than argue about it.
The child was then adopted by Haskellites and raised in a fundamentalist commune.
I feel Kotlin is probably a more accurate result of this mindset, but reasonable people disagree.
What part of ruby or java has anything morally or technically even approaching the concept, use case, or meaning of implicits?
Dependency injection for implicit params, for one? Also, you can almost see implicit conversions as one take on type safe dynamic typing. Almost.
But of course Scala also brings functional programming and its own ideas to the mix.
Spark on Scala 2.13 https://issues.apache.org/jira/browse/SPARK-25075
Scala 2 and the Java 9 Module System https://news.ycombinator.com/item?id=20912258
Do it badly, and you end up with the unmaintainability of Python and the clunkiness of Java. Awful.
Do it well, and you end up with the convenience and interactivity of Python and the typesafety, performance, and toolability of Java.
This lets you implement your code quickly the first time, and have it run blazing fast on a hot JVM, with the compiler having your back as you grow your codebase in size and complexity. Sure beats trying to prototype in Java, or porting half your Python prototype to C when you realize it’s too slow.
Scala has a bad reputation, well-earned due to an early culture of crazy experiments, crazy operators, and crazy tools. But those days are behind us.
You can no longer get stuff into the standard library “just because”, and a lot of the early experiments in that category (xml, parallel collections, parser combinators) has been consciously moved out.
Operator-heavy tools like SBT have largely replaced the crazy operators, while operator-heavy libraries like Dispatch have been largely replaced by less-operator-heavy equivalents.
SBT used to be awful, but it’s improved a lot. And you don’t need to use it: I haven’t touched an SBT build in years by now, in both OSS and proprietary contexts.
There remains a lot of different ways to write Scala, more than most languages, but you don’t need to write Haskell-in-Scala unless you really want to.
I’m personally very happy with my Python-like language with Python-like libraries, with static typing for maintainability, orders of magnitude better runtime performance, excellent parallelism and concurrency, one of the best compile-to-JS experiences in the world, and the best tooling (IDEs, profilers, monitoring, etc.) on the market.
As someone who maintains optimized programming language interpreters, distributed backend clusters, three-tier web apps, command-line tools, and many other things on a daily basis, I appreciate being able to do all this in one language rather than juggling 5 different languages (and 5 different sets of libraries, and 5 different sets of tools, ...) to satisfy each use case.
Scala may be diverse and fragmented, but it’s not as diverse or fragmented as the polyglot Python/Ruby/C/Go/Javascript codebase it would likely take to replace Scala for my daily work
Hey lihaoyi, completely agree with you on this. (Coming from someone who recently stopped writing scala).
That said, in my opinion, if you got to write “python-like scala” with static typing and hof on a regular basis, you should count yourself as one of the lucky ones. In the context of a diverse team, with interests and experiences ranging from Java to Js to python To all the way to Idris, how would you reconcile how different team members want to write Scala?
What I’ve seen this lead to is just because you understand one portion of the codebase, doesn’t mean you can understand another one (both on similar levels of domain complexity), and just because intellij works great on one module, doesn’t mean it’s going to work at all in another one.
EDIT: and I have to say, I love your contributions to Scala library ecosystem, and have used many of them. :) they really do reflect the simplicity of python-like scala.
After all, if you are a Go shop, what’s stopping someone implementing their code in Haskell? Or someone in a Python shop deciding they want to use Clojurescript on Node.js for scripting? Or even within a language, if you are a Spring shop and someone wanted to implement their service on WebLogic?
Scala definitely had a culture of crazy experiments and bifurcation in the past, but it’s always been a matter of team culture rather than of the language itself: something that applies to all languages, especially as polyglot codebases become more common
Without Scala, but with the same lack of coordination, quite likely you would end up with 4 separate silos in your polyglot codebase, written in JS, Java, Python, and Haskell/Idris. Would that really be a better state?
scala is a lot like perl, a powerful tool in the right hands, but gives you more rope to hang yourself if you dont know what youre doing
I really appreciate that I can walk the full gradient from immutable-functional to impure-imperative with one language, depending on the context and the requirements. It takes some discipline for a team (code reviews, regular feedback, discussion on approaches) but I'd say that's true for any language to some degree.
Then there is the ecosystem that's getting better and better with every year, with more and more libraries building on and onto each other.
SBT has been a sore point in the past, that has been ameliorated quite a bit in the past three years. SBT 1.3.x together with GraalVM native is a game changer! And then there are lots of other build tools that less ambitious and more approachable than SBT, if SBT simply isn't your cuppa.
Scala.js is also quite amazing. Scala is definitively underrated in that area. Being able to stay in one tool for a full stack application and being able to shift code seamlessly up and down the stack, depending on the current requirements has been an eye opener for me.
Biggest Scala pros - Terse expressive syntax with a strong type system - Welcoming community and creators - Amazing libraries and rich ecosystem - World-class tooling!
I the future I hope that Odersky and the core team will continue to identify "implicits" use cases and provide dedicated syntax for them, just like they did with extension methods in Scala 3.
Scala.js and Scala Native will support Scala 3 eventually.
SBT isn't nearly as bad as it used to be. It's an open question how much more it will "improve".
I'm looking forward to it! Now that the Dotty experiments come to an end and development of the Scala language shifts away from Scala 2.x to Scala 3 I think this becomes feasible!
Also you mentioned command line tools. Any chance you could elaborate on how you distribute them?
Thanks
Scala missed its opportunity with Android. I think you just have to concede that to Kotlin at this point and just focus on moving in a direction that is not occupied. Which is why I mention FP with a strong focus on types. There's not much competition there besides Haskell. And since Haskell for the JVM will likely never be a thing (a few have tried, but none have gained traction), Scala is still there. And with the direction Scala 3 is going and all the improvements it brings, it's actually looking quite positive. At least in my eyes.
And besides, I don't think Scala has to be a "worse Haskell" in everything that it does. I think ZIO is a good example of a great Scala library that actually resulted in something really special by embracing what Scala can do: https://zio.dev
Will Scala 3 help?
I'm not sure there are any real blockers for Scala on Android anymore. I imagine it's mostly a resource problem now. With Kotlin's momentum in that space though, I'm not sure too many people are calling for it anymore. Who knows though, maybe after Scala 3 is finally out the door priorities could shift.
not really, it's still the problem Android being Java < 8, and Scala on Java 8 features.
https://github.com/scala-android/sbt-android/issues/334#issu...
Scala had, and has, a large bespoke standard library, which is big enough that if you tried to ship it with even a trivial app, it exceeded limits not just on memory, but on numbers of classes, methods, and so forth. (And even if those limits have been raised on newer phones, older ones are still a nontrivial market segment.)
So the way people got around that was by using "tree shakers", which would grovel through the app and its libraries, and chuck out portions of the library which weren't actually used. This works, but significantly complicates the development toolchain.
Between that and the official support for Kotlin as the "better Java" dialect of choice, you have to really like Scala to make it worth the effort...
What sunk Scala on Android was largely the completely lack of interest from the core team.
Everything that Scala on Android achieved was pretty much _despite_ the actions of Scala's upper echelon.
- It's hard to get a good message out, when core members spread FUD by bringing up problems that were solved year ago (like the big std lib issue).
- The core team's perception of Scala was always inward – new platforms existed such that Scala developers could run their code in more places, not because it could attract new developers from these existing ecosystems to Scala.
(Scala.js partially overcame this by herculean efforts of individual members of the Scala.js community that got the word out to the JavaScript world about Scala.js.)
- Core Scala pulled the plug from Android support in the backend. So Scala on Android was pretty much left to die on an outdated Scala version.
- Something like TASTY (an interchange format that can be recompiled on the fly to target new backends), which could have dealt with the problem above – was promised for years, but still hasn't shipped.
- The core team was against even acknowledging the existence of Scala on Android on the website – good luck in convincing that Scala on Android works, if the official website/documentation doesn't mention it with a single word.
- Core Scala treated the main contributor with so much public disrespect, that I'm still surprised he didn't outright quit back then. It takes balls to publicly blame the main contributor of Scala-on-Android for not working on the project full-time, but also conveniently managing not to tell him this crucial "requirement" for half a decade. And with "full-time" I mean "unpaid full-time".
It's a sad story, because Scala-on-Android shipped with some amazing features like hot-code-replacement years before Google managed to ship it. The typed resources stuff was also pretty impressive.
I remember getting the feeling that C# is holding it back though. F# can't have any feature that it wants because interop in both directions seems to be a priority. If I recall correctly, that's what is holding back higher-kinded types in F#.
That, and the last time I tried F# on .NET Core things were still very rough. This was when .NET Core was still fairly new though. It might be fine now.
not as much C# the language, as CLR, the runtime. Because of reified generics, which is the way things are done in the .NET world, and which the runtime needs to support, it would also require explicit support in the runtime for Higher-kinded types or Type classes. And that won't happen without C# having these features (and having them first).
F# is really really great, but it is like an unwanted child of Microsoft :(
Also, core FP principles are not that hard (immutability, expressions, HOF, functional error handling, pattern matching), but when you go full / pure FP, that is a steep cliff and an end goal in itself that has serious time /monetary/ complexity costs. The real goals should always be a healthy balance of reasonably fast business functionality delivery and a good code base.
That's why I specifically mentioned ZIO. Things improved greatly after migrating to it. It feels like it plays to Scala's strength. It infers well and we don't have a single implicit import to make it work. Well, with the 1 exception of the optional Duration syntax so that we can write "5 seconds" instead of "Duration(5, TimeUnit.SECONDS)".
I wonder about the extent to which this is true. One of my primary considerations in picking the language for a product is economic fit. Once you get outside of the top 10 or so, I think you need an especially strong set of niche-specific positives to offset the negatives of a small community.
As an example, for fun I've been working on a mobile app using Dart and Flutter. They are clearly targeting the niche of "mobile app developers who want to be cross platform and don't really care about it being a perfect iOS experience". Which is a pretty big niche, and Google has invested a lot in the platform, so I'm enjoying it. But library support is about 10% of what I'd want, so for things where in Python I'd have a few decent packages to pick from, in Flutter I'm often deciding between an under-maintained 0.0.7 and rolling my own.
If I were doing this for real and with funding, this would at least be a risky bet. Maybe we save 20-40% on our mobile dev costs and , maybe we get screwed by Google's fickleness and have to rewrite everything. Does that offset the extra recruiting and ramp-up time costs? Are good mobile devs willing to work in a less marketable language? Will Google dessert-topping-and-floor-wax it to death in service of internal political goals? Etc, etc.
Scala's in a similar bucket for me. I tried it out and liked it well enough. It's certainly reliable; I wrote my home lighting system in it and the only failure I've had in 3+ years was when Kubernetes shat the bed. But I'm of a similar mindset to Eckel on this: when it's good it's good, but it has a lot of cliffs that I found it very easy to stumble over. [1] Add on top of the hiring problems, and I'm wondering in what niches the economic case is strong enough. Especially so if it's going to keep dropping in popularity.
> I wonder about the extent to which this is true. One of my primary considerations in picking the language for a product is economic fit. Once you get outside of the top 10 or so, I think you need an especially strong set of niche-specific positives to offset the negatives of a small community.
I think you have to take into account the ecosystem in that equation. Scala may have less than 5% programmer mindshare but it has access to the massive Java ecosystem underneath. So languages like that I think get to be viable at a significantly lower threshold than languages that require the whole ecosystem to be built from the ground up (even if that is mostly by writing wrappers that FFI to C libraries). The risk is heavily mitigated by that as well, as a retreat back to Java from Scala is far more doable than from say, Go or Rust to Java if things turn sour.
Scala is disadvantaged by the fact it philosophically departs from the Java ecosystem (in terms of FP, HOF, etc), because that pretty much ruins the interop story backwards into Java (that is, without effort, your idiomatic Scala code / library is not going to be accessible idiomatically to Java programmers). This is unlike Kotlin and Groovy both of which are built intentionally to fit smoothly into the bigger Java ecosystem (although Groovy is more so than Kotlin).
Of course, there could be special-purpose stuff that's very nice not to have to rewrite. PDF generation comes to mind, as it's an enormous pain in the ass, and there are some good Java libraries for it. But it's often not much more work to encapsulate something like that as a service or a CLI tool.
Not to say people shouldn't do it, of course. But I'm not sure what Scala's niche should be as it shrinks. I originally saw it as a better Java, but I think Kotlin did a better job of is winning there. I get the impression that Clojure is doing better at "functional language on the JVM". Which is already a niche inside the FP niche. And given how much other languages have been learning from functional approaches, I don't know whether that niche is growing. I look forward to seeing if they find one.
The killer problem for me is that you can't write a good library in Scala and have Java users adopt it. Because it uses its own collections etc, unless you build highly non-idiomatic Scala code, you will always end up with a huge impedence mismatch there. So while you can use libraries from java, you can't contribute to them, and your audience is always limited. Contrast with Kotlin or Groovy where you can write idiomatic Java-consumable libraries much more easily (esp. Groovy).
It's more than social reasons. The JVM has technical limitations that ruin idiomatic Haskell. Lack of a mechanism for efficiently and modularly compiling general tail-call elimination.
For instance, if you traverse too long of a list, you'll blow your stack. You can try it and fail yourself with Eta.
FP in Scala has the same issues, but they have some extra utilities to work around it, but it's something you gotta be aware of.
It’s too bad, as Scala did the most of any language to bring me into the fold of statically typed programming. I miss it sometimes, but then I remember all the times Scala wasted my time, and get back to writing Go like a happy little camper.
Put your ear to the ground, and you too will hear it.
My employer is doing the same.
[1] https://github.com/rajasekarv/native_spark
[2] https://medium.com/@rajasekar3eg/fastspark-a-new-fast-native...
I think the situation is similar to c++ where too many features have been added over time.
While with c++ there is a body of compiled recommendations on what not to use and how to write c++, for scala that doesn't really exist.
On the flip side, Scala can be a very pleasant stack to work with if you have right people on the team.
> While with c++ there is a body of compiled recommendations on what not to use and how to write c++, for scala that doesn't really exist.
Principle of least power is what you need.
Not everything is a science paper.
Sure, some companies are moving to other languages, that might be growing quicker than Scala ever was. But this isn't zero-sum.
Last time I tried to create a Kotlin project in IDEA, it couldn't provide type information on hover. The Kotlin dialect of Gradle was barely supported enough to be called a "third-class citizen". It's super embarrassing.
Can't comment on the Gradle stuff though, I vastly prefer Maven.
Scala is not tied to a single IDE, it works great in VScode, Eclipse and Intellij and anything else that supports LSP, which is an open standard. And it works really well - including type inference and very advanced implicits stuff.
And recently Scala got excellent incremental compilation support by Bloop from ScalaCenter, which not only blows Kotlin out of the water but even Java with Gradle. The last year I develop purely in Java and I use Scala bloop for day-to-day development, because I couldn't stand multi-minute Gradle compile times. Getting incremental compile times counted in milliseconds (!) or single seconds at worst is something very hard to give up on.
I don't know about pace, but in absolute terms, Go is far behind even the oldest versions of Scala.
But logically, what you suggest implies that either generics aren't absolutely needed, and shouldn't be in Go, or it's absolutely needed, and Go has been lacking them for a long time.
Languages are in a big part matter of tastes, and it does not make sense to criticize their chosen approaches beyond objective technical merits for your tasks.
After 5 years of programming in Scala, professionally, writing and debugging a cli tool in Go-Lang felt like a breath of fresh air. I agree with about go marketing, in that I think the creators of go found a larger “market” of devs and understood them better.
The whole “we don’t need generics” fiasco was hilarious, though.
I wonder if generics and proper dependency management were on this survey you’re talking about...
Generics and dep management were on the top of that survey last year. They are the primary focus of the Go team right now. Russ wrote Go Modules for dep managment, the implementation specifically addresses pain points from the community. Generics are a hot topic and looks like we're close to a finalized design.
Don't get me wrong the Go authors have overridden the community a couple of times and there was some serious outrage. Don't think that was a Google thing though, more of an "We know better" from the authors.
I've never found a more sophisticated type system anything but helpful. It's easy for beginners to start, but when you don't have a good type system and you have a complicated project things quickly become a nightmare.
How do you implement it in Go? It can be done in Java with generics, but if I make the problem a little bit more generic, a little bit more complicated, then I don't think Java can do it.
Of course, this is not representative to the whole market and heavily biased towards big companies with big backends that need to scale, but neither are Tiobe, GitHub or SO rankings.
https://github.com/melling/scala
Recently, however, I’ve noticed that Scala does to be be quite alive (3.0, ScalaDays, Cats, Shapeless, Scala.js) so I’m taking another crack at it.
I’ve done a lot in Swift and it’s my current favorite language, but it’s not widely supported.
When I look at Scala (especially 3.x), I feel like it’s like a Swift++. Go is a great language but I want to move up a level of abstraction.
By the way, Scala 3 is now feature complete:
https://www.reddit.com/r/programming/comments/edcenz/dotty_s...
I spent so much time learning what all these math concepts apply in programming but now I don't remember what any of these mean anymore and I am still shipping production applications ( in clojure). Sigh, what a waste of my life doing scala :( .
The older I get the more I want things that are explicit and easy to understand. Tons of sites and videos and blogs exist that break down the features of Go without a ton of math lingo. I never see Scala materials free of that. Even when I agree that I need to understand the terms of art I haven’t been in a math class in 16 years and it’s slow going. With Go I was productive in two days and proficient in less than two weeks. Yes I still shoot myself in the foot with channels and race conditions when I’m rushing but at least it’s a super fast compile cycle to iterate and my IDE doesn’t require over 2gb ram like scala / idea does.
The flip-side here is that a language that doesn’t support basic programming language theory concepts dooms its users to repeat the mistakes of those who discovered them.
I also would personally disagree that any statically types language without generic types and without non-nullability (e.g. `Option`/`Maybe` and `Result`/`Either`) is severely lacking in the ability to express programs that are easy to understand.
Tensorflow almost won the ML framework wars despite being clearly outclassed by Torch and it took until recently for Torch to overcome the Google brand name
I’ll add to this and say that Go is a language that is very well-suited for Google’s issues. Namely, take a bunch of engineers with a particular area of familiarity (e.g. familiarity with C and/or Java) and throw them at code, scaling up by number-of-engineers where necessary.
My issue with pointing to this and calling it “practical” and “easy to understand” is twofold:
(1) Go implicitly makes its ease of understanding dependent on an having existing background knowledge of concepts that the HN crowd is likely to have, themselves, which biases people but isn’t particularly easy to learn on its own merits (versus, say, Python)
(2) Most companies cannot afford to solve engineering problems by throwing more bodies at it, but with Go your option is either that or leveraging external code generation
As far as “taking the time to get each piece right”, this stuff isn’t cutting edge technology development! Go isn’t doing dependent typing or borrow checking or scoped effects!
Everything that people lambast the language over not having is old hat, it’s like defending a car manufacturer who chose to use carburetors over fuel injection by saying that they’re trying to figure out how to do it right.
Counterpoint, regarding “reducing the cost of features”: Go is spreading a simplified understanding of what garbage collection performance means to a generation of programmers by providing almost no configuration over their GC settings. I’ve had people say that Go’s GC is obviously the best, without any point of reference with which to compare it.
What Go has done here is pick a reasonable set of trade-offs to make in their garbage collector, hard-code most of these settings into the runtime system, and package that into the philosophy of “configuration is bad”.
I will say Rust it isn't nearly as easy to enter as Go, that should be somewhat expected by their desired feature set. Likewise, they aren't as concerned with the cost of features as Go is.
I would somewhat agree with the GC stuff, I think having a couple more levers there with sensible defaults is probably ideal.
> Go is taking the time to get each piece right and I'm glad for it.
Have you ever taken a look at the error handling in Go standard libraries? Rob Pike is constantly badly reinventing monadic error handling patterns and doing it wrong; He even admitted to this himself.
Error handling is an active topic of consideration, but I kinda like how it is now.
Have you ever maintained production Scala code? I have and it was an absolute nightmare compared to Go.
(ns dev.scratch.time-ex
(:import
(java.time Duration)
(java.util Date)))
(let [from (Date.)
until (-> from (.toInstant) (.plus (Duration/ofHours 12)) (Date/from))]
[from until])
I find very little friction in using Java directly from Clojure code. I can't speak to comparisons with Kotlin as I have no experience using it.
yea kotlin prbly is a better choice if you mainly want to use java's libraries and write idiomatic java.
Clojure is not 'better java' like kotlin, its a completely different paradigm.
When using a Java library in Clojure, many times its just better to just write some Java for your Clojure program specifically for the purpose of calling it from Clojure.
Can you describe the kind of situations that you faced issues in?
P.S.: Also, Clojure has first class interop... It's a part of its fudamental design.
https://old.reddit.com/r/Clojure/comments/5twabp/new_clojuri...
What? Calling Java from Clojure couldn't be any easier. The same as calling JavasScript from ClojureScript
https://old.reddit.com/r/Clojure/comments/5twabp/new_clojuri...
Using Java from Clojure I'd say is as good as Kotlin. Using Clojure from Java isn't as good as Kotlin, but it is still pretty good. Only certain things get messy, like in frameworks where you need to extend a class and override methods to set some behavior. Or frameworks that make heavy use of annotations. Since Clojure isn't OO, doing those are a bit trickier.
Although I still think that it is a lot of fun to write clojure, it's a breeze with its focus on stateless, functional code.
I tried to get into writing a GraphQL API in scala once, and the library made excessive use of the implicit parameters. That was too much for me, there was always something missing and it wasn't immediately apparent where things are taken from or why something doesn't work in one place but does in another.
Personally I am making top money on Scala gigs, and I don't see that changing anytime soon. The people I meet on these gigs share my opinion. Most people wouldn't take a Java job even if it paid more, same for Go or Node ( both of which I have used in production in the last month, and wouldn't take over Scala, with the exception of serverless)
Teams in Media, Government and Finance are leveraging Scala to do things that take Google and Netflix twice as many devs because they use less expressive languages.
Yes, Go is great because you can scale a team from 1 to 10 to 100 with little effort. With Scala you get the same stuff done with 20 people.
The people that hate Scala are either Java devs that didn't get it because they wanted to write inheritance based OOP code and would still be bad devs in any other lang, or are hiring managers that struggled to get the right people.
This is actually an advantage when you have the right hiring process as you end up with better developers - they don't need to know Scala, but need to be open minded to new ideas and motivated to learn.
With all that said, the church of FP and the hascalator community is a problem for Scala. Overly dogmatic & academic functional programming styles don't have a place in production systems.
Leverage the type system, immutability and the Future/Option monads and you eliminate huge swathes of bugs. If you are looking to write "Java on steroids" or would rather be writing Haskell then you are probably contributing to Scala's perceived problems.
If you are a startup or small business then Scala isn't a good fit - you're probably better off starting with JS or Python and moving to Scala when you have more users and need to deliver something solid. FAANG don't need to use Scala because they can throw money & devs at problems and therefore favour languages that fit that modus operandi.
For all the companies in between, Scala is great fit because when you get the right people you can outmanoeuvre any competition.
One of the earlier comments described Scala as a love child between Java and Ruby that turned out great. Despite not having used Ruby in anger I agree with the sentiment as I believe the best way to write Scala is as a statically typed Ruby for the jvm. It's supposed to be simple and elegant, and the best Scala devs realise this. To re-iterate my earlier statement, if you are trying to write something "oop" or "pure fp" then you are getting it wrong.
Not a FANG, but close, Twitter is a huge Scala shop and was one of the first to run Scala workloads on GraalVM in production.
Twitter’s Quest for a Wholly Graal Runtime by Chris Thalinger: https://www.youtube.com/watch?v=PtgKmzgIh4c
I adore Finagle/Finatra and wish it had the mindshare over play.
Definitely a large Scala shop.
I like what I've used of the language at work, although I tend to write a lot of JavaScript on a day to day basis, but to say the right hiring process makes Scala an advantage is not quite right in my own experience.
From what I know there are very successful teams that use mostly Scala, at least at Apple and Netflix.
Now Amazon... I've had some AWS recruiter approach me specifically because I was looking for a new Scala job, who assured me they had a few products 100% in Scala. He then directed me to a team that didn't give a damn and was looking for a generalist. Not the first time I experienced the broken recruiting process at Amazon.
Try looking for Ruby or JS devs. It's a lot easier to teach someone from a dynamic background how to leverage a type system than it is to teach an imperative developer to write side effect free code. State is a crutch.
If there is one thing this profession has taught me about hiring, it is that it is very difficult to predict who the top performers will be. Blanket assumptions will often prove to be not true when searching for the cream of the crop.
If I had to hire and teach people for Scala, I would probably go with those familiar with TypeScript or C#.
Isn't Scala minus the problems you mentioned just Kotlin?
Personally, being able to solve problems with the abstractions you can achieve within the language over adding very specific language features to solve them appeals to me more.
Having said that, Arrow has Option: https://arrow-kt.io/docs/apidocs/arrow-core-data/arrow.core/...
Honestly I prefer maybe monads to null coalescing
Kotlin can be introduced very easily into most Java codebases as interoperability is a tenet. Scala not so much. Holistic interop between Scala and Java can fill an entire intermediate-level book.
At the startup, we had a difficult time hiring for Scala. A number of candidates we moved forward with who were well-versed in it wanted to explore the language and embrace the FP aspects, which led to incomprehensible code for the engineers used to procedural OOO. In full retrospect, this was an issue with our hiring standard; Scala should not have been a specific part of our bar. It was a startup, we had crushing customer demands, there was no time to teach every engineer what an IO Monad was. It was a constant battle and perhaps the most frustrating aspect to me leading all the way up to acquisition. None of these issues would have existed if we chose e.g. Go; we would have quickly found a team of engineers who wanted to build applications instead of tinkering with the language. To me, that's the best part about Go -- it's boring.
Java is another boring language, but it's also a very verbose one, even when fully utilizing Lombok/G-libraries/etc.. Kotlin fixes the verbosity issue to a large enough degree for my needs without introducing enough interesting features to attract those darn academic types. To me, Kotlin is about as good as a language can get while remaining on the current iteration of the JVM. We have been slowly introducing Kotlin and plan to use it for most new non-Spark-related projects. I would only consider Scala for the same purpose if 1) interop was trivial to the degree that Kotlin's is, and 2) it was reduced to a reasonable subset of its full complexity. I believe the right steps are being taken with the language, but it still has a ways to go in those regards.
Incidentally, I find golang to be more verbose than Java. The amount of one-off functions you have to write to get around the lack of generics and/or map/filter calls on collections makes the code longer and harder to follow, and harder to debug. Not to mention error handling.
I generally agree with the gist of your post though.
What ended up happening with the startup you worked at out of curiosity?
We made two poorly judged business decisions: 1) build a marketplace without enough proof of demand, 2) treat certain types of financial instruments as securities ahead of a fed decision that never materialized (I call this the bizzaro-Uber play). Outside of the things we could have controlled, the sector we operated in did not meet its expected growth projections in the aftermath of certain participants not playing nicely. We were acquihired as part of the typical VC risk sharing.
The ultimate failure had very little to do with Scala as a tech decision. I was extremely proud of the products we built from an engineering perspective, but I don't credit Scala with that either. The high-performers on the team could have repeated the same technical success with any of the languages I discussed above. Perhaps our hiring was slower than our funding level would have allowed. Perhaps our worse-performing engineers could have contributed more if it was Go or Python. Neither of the alternative realities would have gotten us past the two aforementioned business decisions.
I've used Scala before. It has challenges that would make it a non-starter for many teams. It has some cool features, but those (IMO) don't outweigh the negatives that come with the language: it's overly complex (different people have very different styles for writing the language), the compiler is slow, it generates more garbage than Java, lots of implicit behavior, the standard library is not up to par, among others. Java is improving significantly in the coming few releases: pattern matching, records, switch expressions, etc. will bring to programmers the bulk of what Scala has to offer without the added weight. Kotlin is a much better contender for most teams anyway, and even that will be a challenge IMO.
What does Scala's pattern matching do that Java's (upcoming) doesn't?
> For example, does Java 14 Stream have any method equivalent to `collect` in Scala, which is a combination of filter and map?
Not that I'm aware, but then again, I don't see the large advantage compared to making two separate map + filter calls. Streams in Java are lazy, whereas calling `map` or `filter` on a concrete collection in Scala is eager, creating temporary intermediate collections only to discard them in the case of chaining calls.
case class Person(name: String, age: Int)
// a collection of 3 people
val c1 = Vector(Person("A", 5), Person("B", 6), Person("C", 4))
// now I want a collection of people who have age > 4 and then their ages are doubled
val c2 = c1 collect { case Person(name, age) if age > 4 => Person(name, age*2) }
data class Person (val name: String, val age: Int)
val c1 = listOf(Person("A", 5), Person("B", 6), Person("C", 4))
val c2 = c1.filter{ it.age > 4 }.map{ it.copy(age = it.age*2) }It removes a few things and adds a whole lot of new stuff to it.
It's a bigger, more complex language, even if you credit all the cleanups that were already in Scala 2 to Scala 3.
Scala 3 finally has an answer for that[1].
Besides that; yeah, Scala is amazing. I really see it shine for scientific computing when it moves beyond Python.
[1] https://dotty.epfl.ch/docs/reference/metaprogramming/macros....
Very excited about what's coming in Scala 3
* Lots of improvements to make the language simpler and easier to work with * Integrating some tried-and-true patterns from the current Scala ecosystem directly into the language * First-class tooling to solve some long-standing issues in the ecosystem (binary compatibility)
The team behind Scala 3 has done a tremendous job. Along with the whole movement of better tooling that's happening in the Scala ecosystem right now, I think the language will have a fun and productive future!
[1]: https://github.com/milessabin/shapeless [2]: https://scalalandio.github.io/chimney/
Scala can't beat Rust because you can't have safe concurrency (and also single-threaded mutation control) without linear types, and you can't have linear types on the JVM; furthermore, a GC and VM-based language is inferior to a native non-GC one providing the same guarantees.
Scala can't beat Kotlin because Kotlin is designed to be just a better Java and Scala also tries to do other things that reduce Java compatibility; pivoting would result in being behind Kotlin with no hope of catching up.
Furthermore, its core features are cute but fundamentally broken: class inheritance is an anti-pattern and functional features need either purity like Haskell or mutability control like Rust to work safely and efficiently; on top of that, the language is also complicated and hard to learn.
Why can't you have linear types on the JVM?
On the flip side, Scala will never be as good system language as Rust is, even after adding more improvements to the JVM e.g. value types, or even with Scala native. In these use cases where I need a tight control of resources I'd take Rust over Scala every time.
As for Kotlin - meh, it is just Java with nicer syntax, not interesting to me at all.
Does anyone know if the Scala maintainers have addressed this directly?
Then there is the total lack of access to the huge JVM eco-system, a less performant automatic memory system, and AOT playing catchup with AOT/JIT tooling available for the JVM, some of which with battlefield experience in production since the early 2000's.
And then there is the whole thing that JetBrains gladly sells CLion licenses for the whole development experience with Kotlin/Native.
* What Google does with Kotlin and Android is completely irrelevant to the Scala ecosystem and its users.
* On the back-end side, Kotlin doesn't intersect that much with Scala. I haven't met anyone migrating from say, Play to Spring thanks to Kotlin's support.
* In the big data world, if you rely heavily on Spark, Scala is still the preferred option on the JVM. Python is a much bigger threat there.
* Lightbend and the Scala center have been seriously working on tooling, which is the biggest issue to address compared to Kotlin (Jetbrains has a huge advantage here).
Hi, now you have :) Well, Vert.x and Ktor, not so much Spring Boot.
It's mainly because effective and readable Scala requires a lot more discipline and experience than effective Kotlin for a Java developer. If we'd started solely with experienced Scala devs then I suspect we wouldn't have had an issue.
There is, also, the problem of footguns - we've inherited a large Scala codebase from a sister company, and they, unfortunately, let the devs run free. So there's large amounts of hard to debug, hard to read, Scala magic going on. Implicit classes, inherited three levels deep! Macros! Why? Because they could!
Those who put them out are geniuses (props Cats ppl, ZIO too) - but it seems it is now too late. You know there’s irony that alot of the earlier docs were based on haskell books, too.
Its not the math. Look at haskell’s freenode presense vs scala’s. Scala’s a 6th of haskell. What haskell had over scala is docs and community.
Scala was just too late, and my coworkers are all whispering Kotlin. It’s a real shame.
Irony edit: i actually maintain a scala codebase at work, one of two under thirty other devs who do, and me and one of the major players in the team were talking about this only two days ago. Sort of a sad kind of funny.
And what makes you think Scala needs Android to be successful? Android is completely irrelevant to most Java developers.
This is a real feat of engineering, and likely to make migration actually happen and be fast and easy.
Compared to other 2 to 3 version changes, this is amazing.
Personally, I've found Scala to be by far the best language I've ever worked with. The complaints on the complexity of the library are ones they are (at least partially) trying to address in Scala 2.13. Dotty, though not revolutionary, looks like a nice set of improvements too.
It's not a "pure FP language" (though some might like it to be), but is a multi-paradigm language on the JVM. It has a lot in it, but moves relatively slowly. To me, those are its advantages, and for someone like me they are fantastic. I am in academia, which means that a lot of my software is written by me for courses and so I get to work on it in patches about twice per year. In Java-land, that'd mean major language versions are now faster than I get to revisit my projects. In Scala, I've had projects progress from 2.10 to 2.13 (and soon 3) with maybe two days' effort updating them so they feel fresh with the language, even though the language still feels more advanced than Java, Kotlin, and most of the "faster" competitors.
Scala.js works astonishingly well (I wrote my own react-like framework in about 1,000-lines of code, so I no longer need to fuss about whether React's just updated so everyone should be using function hooks now either). Graal looks like it's going to come along and solve the issues with native deployment soon too.
In short, while most of the complaints I hear are about Scala as a community, it seems one of its strengths is how it is robust to the winds and storms that blow through programming communities.