Hands-On Scala Programming
handsonscala.com
handsonscala.com
In one 400 page book, Hands-on Scala takes you from from FizzBuzz, to Sudoku solving, to Websites and API servers, all the way to implementing programming languages, actor-based concurrent architectures, and massively-parallel web crawling with futures. Every chapter has multiple multiple self-contained executable examples, and multiple exercises with full executable solutions.
It’s a paid book, but the first 5/20 chapters (~100 pages) are free to read online or download in PDF/Epub/Mobi formats, as is the github repo of 125 executable examples.
Whether you are a Scala newbie or an experienced Scala programmer, there’s something for everyone in this book. The chapters on Actors and Futures in particular are the best explanations for the concepts you will find in any language, and would be useful even for people not using Scala.
Feel free to check it out!
This book is an excellent way to dive in to Scala for application development, even though at this point it is more of a refresher than an introduction for me personally.
I started using Scala at a new job in 2014 with no prior Java/JVM experience. I bought Functional Programming in Scala, followed by Programming in Scala. FPiS was completely the wrong choice for getting up to speed in my new role, though it was academically interesting. Programming in Scala was closer to what I needed although it didn't deal with larger ecosystem issues. Hands-On Scala Programming feels like the book I should have started reading the first day I started working in Scala development.
6 years later, I no longer have the new-user perspective on Scala and the JVM ecosystem. Does Hands-On Scala adequately prepare someone for the JVM/Java quirks that leak through into the Scala environment? I can't tell now. I've internalized too much of that knowledge.
Assumed Java knowledge was a rough point for me in the first year or so using Scala. If something goes wrong when you're fetching Maven dependencies in an SBT based project, and you have never used Maven-as-Maven because you never used Java before, you may encounter some baffling error messages and lack the context to understand where you go next to fix it. I also encountered a lot of "Like Java's Foo, but with X" explanations of Scala features that didn't help because I never used Java. In the case of the problem with Maven dependency failures and baffling error messages from SBT, I had to resort to running SBT under strace before I found the root cause.
All guests languages traditionally bring their own build tools, package management, wrapper libraries for more idiomatic code instead of making direct use of FFI bindings, require extra IDE support.
And in the end it is all leaky, because as you mention the underlying platform pops up when one least expects it.
So I do spend the effort to learn what those languages bring into the table, "7 languages in 7 weeks" kind of stuff, but for production code I rather rely on the boring bare bones platform languages.
And the lovely typesetting is a huge plus when reading this on my laptop. Looks great even when viewing in two-page mode.
Only complaint I can think of is that I'd have liked more exercises at the end of each chapter. Ideally with less Sudoku as I don't want to have to dive into the rules of a game in order to practice the collections programming. But this is really a minor nitpick, 5/5 book in general.
Updating a single chapter and re-rendering the PDF takes less than 1 second, and re-rendering the entire book in one format takes about 30s running parallel on my 4 core macbook.
Are there other languages with a well developed actor model that is also performant with numerical computing? That seems like a deal breaker with respect to a language like Elixir.
I'll be honest, I'm personally not a fan of Scala (I prefer Clojure for my JVM goodness), but Scala does give a lot of niceties IF AND ONLY IF you're willing to learn how to program the more pure functional side of things.
A lot of Scala converts fall into the trap of "writing Java in Scala", and if you're doing that I'd argue it's not really worth it...at that point, you can really just write Java; however, if you're willing to learn functional principles (including some basic category theory), it has an incredibly powerful Hindley Milner system under the hood.
Spark is a really useful tool, and the Java bindings work well enough, but it clearly is a "Scala-first" framework. For that alone, I'd argue getting proficient with Scala is probably worth it.
My biggest complaint with Scala is that, because it's very much not opinionated, you end up getting paradigm-clashes with larger teams of people. This will happen to some extent with most languages, but I find it especially egregious with Scala. If you decide to use Scala on a team of multiple people, I cannot overstate the need to codified standards of what is allowed and not allowed.
I disagree.
Context: Scala has been my language of choice for the past 9 years, and I'm the author of Scala.js. I don't know anything about category theory, I don't really care about referential transparency, and I've used a monad as a monad exactly 0 times in my career.
If you're really writing Java in Scala, then yes, perhaps Kotlin is a better choice for you. But there is a whole world of mix of OO, imperative and FP that you can do with Scala, and as soon as you start writing a bit more functionally (immutable collections, not category theory), you get a huge value from Scala. IMO no other language gives you that.
I have crystallised the essence of what I consider Scala to be in my talk Functional Object-Oriented Imperative Scala [1], for those who are interested.
It's still a nicer language and has some useful libraries that wouldn't be possible in Java or even Kotlin. When I got started in Scala I wrote basic spring-hibernate webapps, I didn't even use the Scala collections. But the language still made for more concise and readable code than the equivalent Java would have been. What do you lose?
Scala does not use Hindley-Milner, which is why it has much poorer type inference compared to Haskell or OCaml. It uses subtyping to encode sum types and under many circumstances will infer "Any" for a type.
Those two extremes hate on each other a lot, and this also spills out into annoyance at Scala not being ever more perfect for one of those extremes at the expense of the other. Then there are career enterprise Java programmers who are fine with Java and make it Scala's fault.
Scala community is not a monoculture, it's a refuge for different people with different ideas, and the vast majority of its users are quite happy with it and with each other.
But that also means that different Scala teams work differently, everyone's experience with Scala is greatly affected by the preferences of the developers they're working with. Everyone mostly agrees what a React.js application should look like. Not the same for Scala. You're going to have a tough time working with a Hascalator codebase if that's not how you think.
With preferences-compatibility warning in mind, Scala is still amazing for complex projects thanks to its rich type system and language features as long as everyone's on the same page for how they want to use Scala. And yes, it's overkill for your microservice that has seven classes and a plain JSON HTTP API. Use node or go or whatever for that kind of thing.
Scala is doubly amazing for complex full-stack projects thanks to Scala.js. These days I basically choose my stack from either full stack JS or full stack Scala depending on project complexity.
It's true but not unexpected that Scala 3 is taking a while. Good things take time. And once again, the amount of whining on the scala internet even about things like Dotty not being officially called Scala 3 (until that inevitably happened) was insane.
If you want to learn a safe-bet language strictly for career purposes, I'm not sure about Scala. The ecosystem is mature, the jobs pay well, and the users are very qualified on average in my experience, but it's a more niche job market than react/redux/node.js and all the other hip stuff. If you want to learn Scala for any other reason though, go ahead, it can be fun and productive. Odersky's coursera course is a great introduction.
In particular, none of the projects in this book take more than ~2 pages of text (~90 lines). You implement a programming language in <90 lines, a real-time network file synchronizer in <90 lines, a websocket chat website in <90 lines, a massively-parallel web crawler in <90 lines. These are 90 line complete applications, not "90 lines changed in a sea of boilerplate" or some other nonsense like that!
The goal of all this is to make a case that yes, you can use Scala for small projects! But you'll have to read the book to see for yourself if you find it convincing :)
Type safe pythonic simplicity is underrated.
Honestly that course completely put me off the language. The questions didn't match what he just told you and it was very difficult to figure out what to do. I gave up half way through and bought a Spark book to try to learn it, I might buy this one too.
If somehow you are convinced to use Akka because of high throughput streaming architecture, please just take a step back and make sure you understand what you are getting into!
For simplicity, it uses a tiny Actor library instead of a big framework like Akka, but all the concepts and techniques should be transferrable to any Actor-based application
Also, consider that while message passing as an architecture can lend itself to well-structured applications, it's not a panacea against issues like deadlock & livelock.
It also sounds like you may need some kind of distributed fabric for that kind of system and again, while Akka provides an elegant toolset, you still have to contend with network resiliency issues.
TBH, I have found biggest challenge in constructing these architectures is coordinating their completion or shutdown. You can up with thread leaks that will only show up through extensive testing or experience.
To me, scala used to own the niche of "if you must use a jvm language but don't want to use java". But now kotlin is the first language that springs to mind. What would you say scala is best in class for now?
Also with Scala you get a best in class JS compile target, a great scripting platform in Ammonite, and Graal native image for binaries with fast startup.
There is a huge difference now. Maybe 50% of that difference could be made up by what Java has in the planning stage.
Even if it was 100%, I'm writing code right now.
It all adds up to making me far more productive and more sure of my code.
- import aliases
- type members
- generic variance that doesn't make you want to gouge out your eyeballs
- generics over primitive types
- fully unified types
- an ecosystem that avoids nulls and exceptions
- pattern matching
- case classes
- everything is an expression
- immutable collections out of the box that most every library uses
- higher kinded types
- macros
Sure, maybe Java will get import aliases 20 years after they were due but I'm pretty sure half of that list will _never_ happen. Combined they are a huge deal for productivity, expressiveness, and safety. Java can't die due to the huge existing investments and continuing to improve the language is the right thing to do, but its really well past time for folks to consider more modern languages for new projects.
Apparently there are no plans to distinguish between JVM and Android on multiplatform Kotlin, as per their latest blog post.
Scala.js amazing. It just works. I've picked it up after multiple years of non-use and it everything worked perfectly with no surprises. I don't cover it in this book, but might write about it in another book in future!
Scala has always attracted an outsized amount of hate and FUD. That said, Scala 3 has taken a long time to arrive, costing the language a lot of momentum, and makes some fairly radical changes.
I'm a huge fan of Scala and I really want it to succeed. Haskell's laziness is a dealbreaker IMO (it makes it impossible to reason about performance), and no other language offers the combination of:
1. A type system powerful enough to replace all your uses of reflection/AOP/metaclassing/etc.. No more magic frameworks to learn; write real-world business applications using nothing but plain old functions and values that follow the normal rules of the language (e.g. your web routes can be literally a function from request to response; they don't look quite as nice as a dedicated routing config file would, but they're normal code that automated refactoring etc. will work normally on). Concretely, what makes this possible is the combination of higher-kinded types (letting you represent any custom effect, e.g. a database transaction, as an F[_] type, and then use the language's for/yield syntax to compose those effects in a way that makes them visible but not overly verbose/intrusive), and records (letting you "walk the object graph" in a type-safe way). Scala can do all this without having to resort to general-purpose macros (there is one macro in shapeless that the whole rest of the ecosystem just reuses), so you can be confident that your IDE/debuggers/etc. will all understand your code.
2. A proper first-class IDE, and a good declarative build tool integrated fully with that IDE (maven - for some reason the Scala community pushes the incomprehensible SBT, but ignore that).
3. Completely seamless compile-to-javascript experience. You write normal code, push the button, and there it is running in your browser. When you want to use a library you import typescript type definitions with https://scalablytyped.org/ and it all just works. Combined with the previous point that means Scala is actually the best language to use for frontend development right now. Unfortunately the set of people who know about Scala and the set of people who do frontend development don't really overlap.
I'm not wedded to Scala for the sake of Scala. But no other language comes close.
All that said, it definitely feels like the next year or two could be a turning point (in either direction) for the popularity of the language. Scala 3 may revitalise scala or destroy it. So it's undeniably a risk. But I'd say that even if the language becomes less popular, learning it will make you a better programmer and give you an appreciation for what's possible.
The meme is only relevant for memory allocations. In a lazy language you don’t always know what memory the runtime is holding on to as you haven’t consumed the full result of a function.
In practice, you only encounter this problem in very rare situations. I have personally never had to care of bad performance in Haskell in any of my projects.
In some projects I had to deal with high memory consumption problems. I could reason about why with ease: I was deciding very large JSON structures in hundreds of threads. Any other language would have had the same high memory profile.
def h(a) = {
val b = f(a)
g(b)
}
Sure. There's nothing to stop you using lazy values in a strict language if you want to; it's just that you get the option of using strict values as well. Whereas in Haskell you have no way of building strict constructions other than some ad-hoc annotations.
They implement the language server protocol and add ide-like functionality for Haskell to any editor that support the protocol like vscode.
I totally agree that the records system is annoying.
Printing out the result of g(f(a)) is 2 times faster than printing just f(a). In a non lazy language f(a) and g(f(a)) would take roughly the same amount of time.
In this example the expensive calculations take much longer than printing the output to the terminal. (no nitpicking please)
The performance can only be at most f + g. It may take less time in a lazy language, how is that difficult to reason about, and why would you care if it runs faster?
Imagine you've got a performance problem because f + g used to take x time but now takes 4x. You start trying to break down the problem, and you see that f takes 2x time and g takes 2x time. You check out the old tag where f + g takes x time and you discover that f takes 2x time and g takes 2x time. Where do you even go from there?
If anything, it can make things run faster, but if something is already slow, you just start by looking at the slowest function. You don’t even have to guess, the Haskell profiler will tell you what function that is, even with laziness involved!
Because the slowdown didn't happen in either function. f still takes the same time it used to. g still takes the same time it used to. But somehow the composition has gotten significantly slower. Both functions may be slow when run in isolation, but it's impossible to tell whether one or both is "supposed to" be slow. Profiling and looking at the slowest function can easily lead you on a wild goose chase here.
Could you formulate an example where the composition would be asymptotically slower only for lazy languages?
As I said before, laziness is actually an optimization strategy. If anything, it can only make the composition faster than it would be in a strict language (although rarely so)
No; we both know no such thing exists. The same code will always be asymptotically faster in a lazy language.
The problem is that lazy languages (or at least, the one lazy language in widespread use) encourage a style of code that would be asymptotically slower if written in a strict language. In a strict language, a function is slow or fast, and a slow function is always a problem - which means performance is easy to reason about. In a lazy language, a slow function is not necessarily a problem, depending on how it's used, and so you can't understand performance without understanding the whole program.
An unreliable optimisation can be worse than no optimisation at all. Worst practices should be hard. In theory an extremely vigilant development team could always write code that would be fast if executed strictly, and therefore guarantee no performance problems, but this would require an enormous manual effort since the language doesn't help you with it at all.
If anything, in a lazy language it may take less time, but never more than the sum of both.
How is it that you cannot reason about the performance of each separate function?
How would you solve an iterator in java producing infinite values?
If anything, the performance of a lazy language would be better than a strict one in the presence of an infinite sequence, as it gives you the opportunity to stop sooner and not enter in an infinite loop.
Example in haskell:
main = do
let a = [1..] -- infinite list
print (a !! 0) -- only prints the first element and exits
def f():
a = []
b = 0
for True:
a.append(b++)
return a
def g():
print f()[0] # Takes infinite timeIn practice an infinite loop in Python is often an immediate error (stack overflow) at the point where it's declared. In cases like you showed it's a little fiddlier, but any execution of that code will immediately show that there's an infinite loop (e.g. any unit test of it will time out) and a stack dump taken while the program is "stuck" will show exactly where the infinite loop is (which is therefore where the error is, because an infinite loop is always an error in these languages). Which is still not an ideal situation, mind; I'm very much in favour of an Idris-style language where function termination is actually checked by the type system and there's a type-level distinction between data and codata.
> How would you solve an iterator in java producing infinite values?
I'd avoid ever passing around a bare iterator, ideally with a lint rule.
The point isn't that it's impossible to have lazy values in other languages. It's that it's practical and idiomatic to avoid them, or at least limit them to cases where you absolutely need them.
The effects were only visible in production, as it was the only environment where the error condition happened. It caused thousands and thousands of messages to be queued in vain.
It took me, and another person 3 days to review the code line by line to understand where the problem was.
As you said, any execution of the code would show that there is a problem. Just that un this case no one new where it was executing from.
I think you should drop the argument that performance with a lazy language is difficult to reason in terms of time. It is well studied that asymptotically, strict and non-strict languages are the same.
It is a different story, and a valid argument for Memory consumption, though. It seems to me that you read the “impossible to reason about” meme and formed your own opinion of why that would be so. But this is not the actual reason.
If you write the same code in a strict language or a lazy language, the asymptotic performance will be the same, sure. But that ignores the fact that people don't. Lazy languages use different idioms (there would be little point in laziness as a language feature if people didn't write code that relied on it), which is what results in code whose performance is impossible to reason about.
Not because the language is strict, it makes infinite loops or infinite data structures more obvious to debug or understand.
In Haskell it would have been the same exact infinite loop.
There's a lot I don't like about Go, and plenty of things to be cautious about in message-queue-based systems; laziness is by no means the only thing I worry about. "decrementing a counter" suggests the kind of imperative control flow I would be very cautious about; I favour iterating/recursing over data structures, which is inherently safe against infinite loops (but only in strict languages with immutable datastructures!). I can't remember the last time I decremented a counter that was being used for a loop termination condition in the code itself; I've occasionally written functions like "run this block n times", but the most trivial test of such functions tells you whether they're infinite loops or not. Of course this is only really practical in a language powerful enough to write recursion-schemes style constructions.
If it is worth learning, the really depending on what you want from this language. It was and it would be the most novel language in the mainstream industry - it has almost all the features developers can dream of, and most of the features are not arbitrarily piled together like C++, instead, there are synergies between features.
With that being said, it's still a hard language to learn. And programmers from different backgrounds wouldn't interpret it in the same way. In real-world codebases, if the teams are not very good at training and aligning, it would be too diverse.
All in all, it's not only a get-things-done technology like Go, but it's pretty educating and mind-blowing.
It is true that you can use the JVM ecosystem, but you'll see that your code doesn't look like scala at all. Then you look for scala ported version of java libraries (some essential but boring libraries like xls read/write) that isn't likely to be maintained actively
For better or worse it's super easy to write application code in Scala without going to a very abstract level, particularly in an ecosystem like Play. And that does make library code feel less familiar because it's using features that you just don't necessarily use very much in your application code.
Scala seems to have many black magics under the hood and this is one of biggest drawback of using scala
With Scala 3 almost in sight, will this book update to Scala 3?
How do you do that? I've only used javap on the command line for this.
Any plans to offering a printed version using one of the print-on-demand services(Leanpub etc)?
I'd gladly pay the full $75 for such an offering.
I plan on using the book for a practical Scala course that I will be teaching in the fall.(Programming in Scala is fine but not as practical).
I just bought a PDF version. It looks nice at the first glance.
Do you plan to keep updating the book's content and code in the future? I'm really looking forward to your views about Scala 3.
I don't expect the code to need to change too much to support Scala 3: Scala 3 is likely going to be backwards compatible to a large degree, and the code you'll be writing in Hands-on Scala is pretty simple Scala that doesn't touch the more advanced features that might be affected by the upgrade
what do you think of Scala adoption and prospects these days
A lot of the initial hype around things like Akka and Pure Functional Programming has died down since the frothy excitement of 2015-2016, but honestly I think a lot of that hype was oversold and many of these things never quite lived up to their original excitement.
Right now, the level of hype has fallen back to where the real value is: a solid language allowing flexible styles to suite a wide range of use cases, as concise and convenient as any scripting language, with excellent performance, tooling, and static checking to prevent defects.
With this book, I'm obviously hoping to drive adoption and prospects up, and I'm doing so by trying to sell the value prop above rather than the value props that were (over-)hyped in the 2015-2016 era