Ask HN: What is your experience with Scala?
Hello,
what is your experience with Scala language? Do you like it? Would you use it for your next project? Are you thinking of something else?
Cheers, Wiktor
what is your experience with Scala language? Do you like it? Would you use it for your next project? Are you thinking of something else?
Cheers, Wiktor
* Neither IDEA or Eclipse are particularly great (I routinely get types that can't be introspected, and both require inordinate amounts of RAM to work well)
* SBT is a productivity sink to do anything beyond the basics with
* The compiler is slow (and yes, it does matter)
* The compiler's error messages are poor
* I find the code in many Scala libraries impenetrable (which matters a lot while debugging).
The unexpected side-effects and the library situation frustrates me the most: I feel like I'm always dealing with "code in progress" written by people who're enamored with the features Scala provides over building something robust and easy to understand.
Obviously, my opinion. There are plenty of people out there who love Scala.
I.e when you get types that can't be introspected you definitly don't work with simple scala. I've been playing around with really big projects and even made some contributions to the Playframework project: (https://github.com/playframework/playframework) and even that has barly/no types that won't work. (Oh while working on Multipart I had Java Code that didn't worked on IntelliJ 15 (works in 2016.X now)). I know there are certain libraries that are akward to work with in any ide but these libraries should be avoided. And they are mostly based on scalaz and I would even consider specs2 as harmful. But nothing applies to scalatest its imo way faster.
While you see SBT as a productivity sink you can stick to gradle or maven, both are supporting Scala. Still even the slowness of sbt (which isn't so slow it just needs to do everything twice in IntelliJ when you use it inside the console and your ide) isn't so bad for me since it's really simple to make AutoPlugins (this isn't so simple in Maven/Gradle) What error message is poor for you? I barly had issues with them. My issue's were more some stack traces but on Java the same stack traces annoyed me aswell but that was on runtime crashes.
Actually its true that some libraries are akward. But my ruling for libraries is anyway: use as few as possible. You prolly don't need a library for leftpad but that applies to all libraries I even was on the golang Ecosystem for a while predates even the 1.0 dates I even have a book for ~0.7 and there were some libraries that were pretty akward and messy, things evolved but there are still some code around which you shouldn't barly touch and even for http, you prolly don't need a library for most of your stuff and still people use them.
P.S.: Nothing against the compiler slowness but from day-to-day you barly notices only a sbt clean compile could be slow but our app (which is quite big these days only takes a few seconds). Even play! compiles fast on my machine:
sbt clean compile
// ommitted everything in between
[success] Total time: 95 s, completed 04.06.2016 23:54:53
And this is the point - if you want to be productive, you have to use _may_ libraries. Unfortunately, Scala's way is not to carry about compatibility, making it extremely hard to grow up large projects with a lot of dependencies. For example, we heavily use Apache Spark which targets Scala 2.10, but this makes impossible to plug in many libraries that only support Scala 2.11. The same applies to library dependencies: just yesterday we had to introduce a number of exclude rules into SBT file to reconcile Spark and HBase, because SBT was failing with cryptic messages.
> I mean slower than what?
Go, Java, Clojure, C#, Python (PyPy), Haskell? In my experience, only C++ and Scala make compilation so slow that users bother about it.
SBT is fine if you're not doing anything complicated. But the moment we tried looking into building a plugin, even just to add a step in the asset compilation pipeline, it's a land of mystery. Instead of wasting time trying to decipher SBT plugins, we just used npm for our front-end assets.
I also found working with Java libraries to be overly frustrating. We generally ended up having to write Scala wrappers around Java libraries in order to give ourselves an idiomatic interface.
I will grant you that compilation time is usually exaggerated. Incremental compilation means you normally only wait a few seconds at a time. Now I do a lot of work in Haskell and find compilation significantly faster.
You may not want to use libraries, but that's not always feasible. I definitely don't want to write my own DB library, or FTP library, or logging or... and, if you have to do that, you'll find that the Scala options aren't easy to reason about. For example, I had to look inside the slick codebase recently and I found it...obtuse. OTOH, I've looked into a number of Java and C DB libraries and found them way more straightforward.
This is my biggest frustration about Scala: that the community appears to stress "neat" and "cool tricks" over pragmatism. This IMO, makes it a bad choice as a language for a startup.
When it takes several dense paragraphs to explain simple problems, it means that problem is not solved easily. For some programmers easiness is very important -- this seems like something Scala supporters refuse to understand.
> I appreciate the concision and the strong collections API
Everybody who said that, didn't worked with it. The Collections API is the Weak Point of Scala. You ever tried to make your own? Fine work with CanBuildFrom Blabla stuff. It's nearly impossible to understand it.
And IntelliJ made a real progress in the last years. I mean sometimes IntelliJ fucked up pretty badly. SAM support is one example it messed scalac's "-experimental". But as already said even Java didn't worked correctly all the time. When you work with a lot of generics you stumble across some types that just won't work in IntelliJ but compile.
And types that won't work in Scala, I've seen them, but mostly that was a problem of the Library.
And another thing. If you don't upgrade IntelliJ you will mostly won't get bug fixes for the Scala Plugin. I've started with IntelliJ 12 and from that point I always needed to upgrade the whole IDE when there was a new version since they didn't backport most bugfixes.
And when you use a lot of Macro's the implementation of them definitly has some non introspectable code. And as said if you did them wrong their usage will be red, too.
However I have to agree with many of the common criticisms. My last team had many WTF moments with the language. We also suffered from poor type inspection and IDE support, failed attempts at writing an SBT plugin, and slow compilation. Having access to Maven repos is great, but using Java libraries is often a frustrating experience where you end up having to write a whole wrapper library around the Java lib just to work with something idiomatic.
The language is a great stepping stone to bring hardcore Java devs into the world of FP, but honestly, if you don't need the JVM, I would rather be using Haskell.
The language has well-documented warts. I don't like all parts of the Play framework that we use. It's slow to compile and sometimes the error messages are weird. If you miss a close-curly brace you'll sometimes just get a completely impenetrable type error, things like that.
But honestly, all that I can live with. Case classes, sealed traits, function types, generics, monadic error handling... So much application logic can be expressed at the type level where it gets mechanically checked. The only errors that get past the compiler are in JSON parsing and database interaction. Everything else just works after it compiles. I've refactored and redone abstractions that touch all other components of the code, and I'll do it and deploy on a friday afternoon and go home without worrying. I trust the code, even after changing it. I have integrations tests that spin up the backend with a real database and send it HTTP requests, but nothing like complete test coverage. I'd like to add that out of principle, but it doesn't feel necessary.
Scala has warts. But it works well and it has static types without the verbosity of Java. I can't imagine going back to an untyped language plus tests again.
If you are a one language trick, you might have a bad time, but if you have been exposed to multiple languages (like a typical CS education) it shouldn't be too hard to be productive and even fairly idiomatic code in a couple months.
Overwhelming pros:
* Type system is awesome, massive time saver once a project is underway
* Operator overloading very well done. Infix, postfix, prefix operators can allow for very readable, concise DSLs. Apache Spark being a great example but also many DB libs.
* Excellent frameworks like Playframework, Apache Spark to build on top of
* Being stacked on the JVM - you have access to tons of production worthy libs for just about anything
* ScalaJS.. really awesome if still young project to unify web development in a much more reliable way than weak JS and $backend.
Overwhelming cons:
* Compile time is quite bad
* Operator overloading can cause some really bizarre syntaxes like SBT. SBT is probably better than the mainstream Java build systems, but it's often a head scratcher.
* Being stacked on the JVM - you soon desire native Scala libs for everything
In 2016 I would also heavily consider Java 8 for the same team usage vs when I started using Scala in 2011. I think Java 8 really slowed the adoption of Scala down which may be worrisome for long term market share.
Out of curiosity, would Clojure be up for consideration? Why? Why not?
What I don't like is that tooling is either missing, terrible at battery life on Mac or incredibly complex to learn. Intellij sucks battery because it locks the high performance graphics card on. Counterclockwise is nice, but it's got a background task that spins every minute or so. That drains the battery too. Atom drains the battery because it's Chrome-based. Emacs is...Emacs. Could be nice, but holy crap the learning curve. Finally, there doesn't appear to be a great DI library like Spring. Component works, but its naming and interaction flows are confusing.
I do like Clojure. I've got a few project for Clojure around ArangoDB.
After all of this, I'm switching back to Eclipse (Spring STS), Java 1.8 and Spring. For all the language issues, I can knock the project out in a few months. I can probably train my wife quickly in it too.
The good thing is that "runtime" doesn't have to be production - you can experiment with live data in a REPL and flush out most of the bugs, which of course doesn't diminish the need for good tests.
I wouldn't recommend it for teams consisting only of junior developers and/or large codebases, where it can be hard to maintain good hygiene in the long run.
I didn't even realize Scala had it, until a few months ago when somebody was doing a demo that involved some SBT. It was quite frightening. (Did we learn NOTHING from the C++ debacle???)
Come to think of it, I've come to dislike "operators" in general (as opposed to function calls). Memorize these operator precedence rules? No thanks, just do the inside parens first, now you're trained.
C might be a good step up from assembler (though it, and its spawn need to go away as app dev languages), but I think Lisp got this aspect (no special, jacked up, operator syntax) right, in a "less is more" kind of way.
1. Methods can use symbols
2. Single-arg methods can be called infix
Scala doesn't have operator precedence rules. All operators (infix methods really) are left associative unless they end in a colon in which case they are right associative. Examples
2 + 1
Is
2.+(1)
2 :: 1 :: Nil
Is
Nil.::(1).::(2)
The colon thing is the sort of thing that on paper seems hacky but in practice Just Works and is seldom used. But when you do use it it really helps a lot.
That's not true. The operator precedence is determined by the first character of the operator:
(all letters)
|
^
&
< >
= !
:
+ -
* / %
(all other special characters)
Curse the fool who invented "Ant", rather than figuring out how to parse a Makefile and provide a few rules or pre-set-variables to support DOS/Windows, Java compilation and such (e.g. - in a Java program to interpret a Makefile and avoid running external processes for steps that were defined as doable in the JVM). Now we have a generation raised on XML who thinks that extending a build requires creation of "Ant tasks", "Maven plugins" or similar rigor-morale.
This is an area ripe for "disruption". No idea if there's any money in it though.
I think that for "programming in the large" that Scala is not going to be a winner here. I'm talking about a single code base with hundreds of engineers with a big eye towards maintainability. Because lets face it, for a 1 person small project, it doesn't matter what language you use as long as it provides single developer productivity. But once you talk about large team projects, things get very difficult. Scala only compounds this imo.
We use it exclusively on the server side at Verizon Labs, and it's quite beloved by all. We lean hard on the functional side of things and make heavy use of what some people call 'advanced features', but because of our micro services infrastructure even the teams new to Scala can be productive in a less functional style, and refactor later to be more idiomatic.
I do think large teams need to have enough language experts to help from making some bad design decisions, but I have plenty of productive coworkers who are new to the language and learning as they go.
I really don't agree about the 'maintainability fears' or 'write only' language criticisms. 'Bad' Scala is still so much better than bad java, and good scala is phenomenal to work with. Because of the type system, I have a high level of confidence when working with unfamiliar code and making minor changes.
Smart folks with no exposure to functional programming are picking up the 'advanced' features very quickly with the right training and guidance. I would not recommend it to a team who has on one with any experience in the language, but if a team can hire those who have experience with the language, even if it's just a fraction of the overall team, I'd be confident they could succeed.
There are real downsides:
* slow compile times
* IDEs can struggle with advanced libraries (Scalaz)
* some language features are 'sharp' (implicit conversions)
* finding developers who know scala
We're overcoming the first one by the fact that we're already a microservices shop.
The IDE issue doesn't affect me, I'm on Vim, many coworkers on Sublime/Ecmacs.
If you go crazy with implicit conversions it can cause issues, but again we try to mitigate this with some training and having enough advanced users of the language/code reviews to show everyone how to relegate using conversions for syntactic sugar or typeclass conversions.
There's a good side to it though. I took my current job because I wanted to use Scala more and my manager admitted (after the fact, sneaky bastard :) that hiring Java+Scala developers helped him filter/attract curious and talented people.
If you want to write Scala to actually get things done, you will find that it's a miserable language to learn and that you will need to invest significant amounts of time, energy, and sanity to learn it. Furthermore, there are no shortages of Scala apologists who are so utterly enamored with the cleverness of the language that they will not concede how much of a pain in the ass it is to learn. They will claim it is easy, and then they will have no empathy for your plight. It is actually quite shameful.
The attitude in Scala-land is that you are the problem, and not Scala. When you do encounter a roadblock to a seemingly simple problem, which will be quite often, you're lucky if you find a 20 paragraph explanation on StackOverflow. Otherwise, you can hopefully find a lengthy blog post about something closely related. In either case, the attitude is that it is _not_ Scala's fault that solving problem XYZ requires you to understand several chapters of advanced concepts -- it's _your_ fault for not already knowing it.
But it's difficult to master and therefore difficult to sell to others without a scala background. You're also competing with the finance industry and salaries for decent practitioners easily top 200k.
So my verdict is good for an individual contributor, bad if you need to convince others to use it.
Also, scalajs is awesome and just works, with the exception of some reflective calls. Otherwise normal scala compiles to good JavaScript, and interop is as good as coffeescript (i.e. perfect).
My initial impressions of Scala:
1. There is an unavoidable learning curve when onboarding. My personal experience with Clojure lowered the curve a noticeable amount. While I was tackling learning a new syntax, my team mates were struggling with syntax + functional concepts. People with exposure to functional concepts will have an easier time.
2. Our experience was made more difficult in that there were no in-house Scala experts/mentors to whom we could ask questions and get answers. Bootstrapping Scala knowledge takes some time and I admit that while I'm productive writing Scala, I know I'm not at the level where I can analyse design trade-offs, extol best practices, or use my language knowledge to pre-empt and avoid problems like I can in Java. We had a good experience with a consultant + videos + books + online articles, but it isn't the same as having a mentor colleague.
3. I write a LOT less boilerplate. Both my Java work and Scala work involve consuming and producing JSON data. In Java-land I used Jackson to annotate classes and used builders to build that as the data classes themselves were made to be immutable. It was a lot of repetitive code and mind numbing to write. Now you could argue that the code should be generated automatically, and you're not wrong. But it feels like something is broken in the language if this is the best solution.
In Scala, I find myself using case classes and spray-json and typically write two lines of code for a JSON-serializable class definition. (1 for the case class, and one for the implicit JsonFormat definition. Occasionally I have to break out RootJsonFormat and write more code, but it's short and concise and non-boilerplaty. I like writing Scala much more than Java and I'm because I like it I'm more productive.
4. Some of the libraries feel like a force multiplier. Specifically Spark, and to a lesser extent Akka let me do things I wouldn't consider trying in Java. Sure, they can be used outside of Scala, but the way they are presented in Scala is beautiful and makes the programmer feel powerful because when using them, the are.
We've created several in house DSLs and used them to great effect.
Compiler is slow, though it has gotten better over the years.
Very much looking forward to DOT (next gen of compiler and language, union types will be awesome).
Use it on four production websites, about to transition a Ruby on Rails site to it soon.
Disclaimer: I spent some time December 2014 over a Christmas break whacking away at a small Play app. I've not been back since due to a number of "hygiene" factors.
* I loved having type-safe templates.
* I found that object and class distinctions were weird and the multiple namespaces (iirc) didn't make a great deal of sense.
* The language felt very grab bag: things didn't seem to "fit" logically together well.
* Scala still has null.
* The tooling is slow. The tooling is fat. The tooling didn't run on my (few years old) laptop with 4Gigs of ram. More on this below.
* High-class documentation about the language is hard to find. The Java (and scala) ecosystem is filled with really ignorant people and really low-level docs.
* And implicits were stabbing me in the face because they are unusual and technical descriptions of what they are were hard to find.
* The compiler error messages are hard to scrute. This is definitely less of a problem than above. (I've used C++ templates, so after that, it's hard to complain too much).
Regarding the bloat.
I have Haskell, OCaml, Lisp, Java, Clojure, Python, Ruby, Rust, and ... others on my machine. Of these, exactly one has ever had a problem for my home machines: Rust, years ago, when self-compiling would alloc more than 3 gigs of ram - and my 3 gig machine would reap the process. Only Scala has presented the memory and speed issues I encountered. Given that its ecosystem is kind of designed to use an IDE, this is terrible.
My takeaway from building this (small) web app well over a year ago is that Scala probably works well in a team with the funding to buy hefty machines and the ability to maintain an operative knowledge of the Scala Lore. If you need the JVM, and you need advanced type systems, it's the only choice. Otherwise Java 8 is probably adequate.
Eating my own dogfood - I'm working on another well-typed smaller web app at home, and my ordered preference list from top to bottom looks something like this: [OCaml, Java 8 (9?), Rust, Haskell, Scala].
A great contender to Scala is starting to rise though, in the form of Clojure and ClojureScript. Clojure is starting to become pretty great for backend work and once you start using ClojureScript and topple it off with a framework such as Om or Reagent for building React based reactive UIs you will never look back.
I think the biggest potential competitor to Scala may be Swift, now that it is open source. If it becomes as multi-platform as Scala is, and good back-end libraries become available, then Apple's support may be very hard to compete against. There were even rumours that Google was considering using it for Android.
It is nice to see that Scala will soon be available for mobile development too. In a short time, there will be a nice overlap of platforms supported and language features, which will provide for interesting competition.
Now, I have been able to use Scala along with Akka on some smaller projects at work (usually where I'm working solo), not that I bothered asking for permission or anything :) At home, I've started using Clojure on a side project and I generally find it to be a more pleasant language to use than Scala (but Scala is still vastly superior to Java, of course). But it leans more toward FP than Scala does, so I know if Scala is a no-go at work, Clojure isn't even worth asking about :)
So, yea, if I'm ever in a position to select languages/technology stacks for a new project (hasn't happened yet) I'd definitely lean toward Clojure for a true 'greenfield' project. If I needed to integrate with an existing Java codebase, Scala might be a better choice.
I'd say Scala, used pragmatically (e.g. avoiding advanced stuff like Shapeless or Scalaz), is currently the best mainstream programming language, meaning it offers the best compromise between language power, maturity of implementations and library eco-system.
My main criticism has always been the slow compiler which was eyewateringly terrible in 2011. With SBT's incremental compilation and faster machines the situation has become bearable.
I liked the language for its ambition. I think the notion that OO and functional approaches are complementary worked very well for me. There were a lot of interesting ideas in the language and the ecosystem.
However, there were a number of downsides. I came across a couple of bugs that were well known and apparently people were just expected to work around them. As others have said, compile times were terrible, and the tooling wasn't great. And at a one day Scala unconference one of the prominent Scala community members was a flaming asshole to me, which definitely didn't help.
What really put me off, though, was what Bruce Eckels described: "I’ve come to view Scala as a landscape of cliffs – you can start feeling pretty comfortable with the language and think that you have a reasonable grasp of it, then suddenly fall off a cliff that makes you realize that no, you still don’t get it." http://bruceeckel.github.io/2015/08/29/what-i-do/
This was backed up for me by a talk by Paul Phillips, a major contributor to the Scala compiler: https://www.youtube.com/watch?v=4jh94gowim0
So in the end, I'm likely to look elsewhere. The next language I'm trying out is Kotlin:
It's from the IntelliJ IDEA people, who I trust a great deal, and I just got done working through their nice in-browser IDE tutorial:
So far it seems to have all of the things I like about Scala plus a number of improvements. E.g., fast compile times were a clear goal, and I didn't come across anything that struck me as language astronautics.
I am currently giving Kotlin a shot and so far I do like it. I downloaded and installed IntelliJ, then the kotlin-dropwizard github repo and imported it into IntelliJ with no problems. It runs well and the code is clean and legible.
I would think the answer to the original question would be that if you have to ask then probably it's not the right direction for you. This is especially the case if you are looking for something for your next project. You have a big learning curve before you even get started.
Start at the end and work your way backwards. What's the fastest way you can get from point A to showing off your project? Even better. What's the fastest way to get from point A to making a dollar?
Pick up something that's already on your shelf, throw up a sketch of something finished on your wall and then get to THAT as fast as possible. No project should start with "learn Scala" unless you're being paid to learn it.
I'd rather stick a functional language built from ground up without the baggage. Or even straight Java. You can do pretty nice functional stuff with Java 8 streams, but it's not trying to be something it's not.
Which ones?
The only reason I'm not using it is that my professional Java-related tasks tend to be small enough that pulling in an extra dependency is an overhead.
Mind you, I haven't looked at Clojure or Kotlin, both of which offer the same benefits I find attractive in Scala.
Pros:
* The type system
* Being able to mix functional and OO design coherently
* Less verbose than Java
* Utilise existing Java libraries
* Huge amount of syntax to learn
* Fragile and slow package managers
* sbt uses Ivy and so downloads package files in a single thread. In Australia this is horrible
* too slow for scripting due to JVM boot timeOne problem I have is sbt - our projects are all defined in Maven, and for many library / dependency problems, Google leads me to sbt solutions which are hard to understand. Guess I will have to learn another build system.
I would like to use Scala again in a larger project, but there are so many existing Java frameworks out there that can solve the problems we have. So it is hard to define a business use case that requires experienced Java developers on the team to switch to Scala and use libraries and frameworks that are not as thoroughly tested and supported compared to the already known ones.
Tooling: The Scala plugin for IntelliJ is getting better, but it had some bugs like suggesting to remove required imports which broke classes. I like the easy navigation between Java and Scala libs - sure, you get "null problems" from using Java, but you can use almost all your existing Java code (Just don't try to use commons.StringUtils.join instead of Scala .mkString ;) ).
My favorite thing about Scala is how well it supports DSLs , and it being the base language for a probabilistic programming language called Figaro.
The biggest problem with the language, is that it is huge. There are a lot of language features, like implicits, that run the risk of being abused. Additionally many Scala developers don't write code like salad of Elders but rather like Java developers trying to learn functional programming.
The first project used Spark, so Scala was a natural choice. One big-ish issue at the time was that our team didn't have anyone with experience in the language, so the code base ended up being very Wild West-like, with different styles mixed and heavy usage of implicits. However, none of my teammates had much difficulty becoming productive in Scala (not being an expert, but just being able to produce sane code, quickly), even though they came from mostly non-functional backgrounds (Java, C#). I did struggle with the type system initially - needed some time before it all "clicked", e.g. when to use 'flatMap' instead of 'map', things like that.
The second project uses Akka, which is by itself pretty neat. This time however, I'm the only engineer with experience in Scala and writing production code in general. I feel like the type system provides a lot of protection against silly mistakes that newbies sometimes make (especially with the poor test coverage, such as it is), and the code is much more concise that Java would've been, so yay smaller code reviews! The rest of the team is familiar with traditional C-like syntax, so it's easy for them to get started. I try not to push too hard for the functional features initially, instead showing people how their imperative code could be rewritten in a more clear and concise way.
One major problem is the IDE support. It's 2016, yet IntelliJ is still slow as hell with Scala, and I've abandoned Eclipse long ago.
I do notice that the language seems to promote in some people a certain tendency to write esoteric code and use some very cryptic features just for the sake of it, but I haven't been bitten by this in the libraries we use, and try to be pragmatic in our own codebase.
Lastly, the value of the infrastructure - the JVM, the rich set of libraries for Scala and Java, Scala itself with stuff like Scala.js and Native - should not be underestimated by anyone who considers the language for production use.
Oh, and I do have some comments about Clojure as well, if anyone's interested.
Would I use it? Maybe. I don't know it well enough to do anything substantial yet, and lately I've been more focused on learning R, Octave and brushing up on Python. But there is a new Scala specialization that just showed up on Coursera that has me intrigued...