Are there other languages with a well developed actor model that is also performant with numerical computing? That seems like a deal breaker with respect to a language like Elixir.
I'll be honest, I'm personally not a fan of Scala (I prefer Clojure for my JVM goodness), but Scala does give a lot of niceties IF AND ONLY IF you're willing to learn how to program the more pure functional side of things.
A lot of Scala converts fall into the trap of "writing Java in Scala", and if you're doing that I'd argue it's not really worth it...at that point, you can really just write Java; however, if you're willing to learn functional principles (including some basic category theory), it has an incredibly powerful Hindley Milner system under the hood.
Spark is a really useful tool, and the Java bindings work well enough, but it clearly is a "Scala-first" framework. For that alone, I'd argue getting proficient with Scala is probably worth it.
My biggest complaint with Scala is that, because it's very much not opinionated, you end up getting paradigm-clashes with larger teams of people. This will happen to some extent with most languages, but I find it especially egregious with Scala. If you decide to use Scala on a team of multiple people, I cannot overstate the need to codified standards of what is allowed and not allowed.
I disagree.
Context: Scala has been my language of choice for the past 9 years, and I'm the author of Scala.js. I don't know anything about category theory, I don't really care about referential transparency, and I've used a monad as a monad exactly 0 times in my career.
If you're really writing Java in Scala, then yes, perhaps Kotlin is a better choice for you. But there is a whole world of mix of OO, imperative and FP that you can do with Scala, and as soon as you start writing a bit more functionally (immutable collections, not category theory), you get a huge value from Scala. IMO no other language gives you that.
I have crystallised the essence of what I consider Scala to be in my talk Functional Object-Oriented Imperative Scala [1], for those who are interested.
Scala does not use Hindley-Milner, which is why it has much poorer type inference compared to Haskell or OCaml. It uses subtyping to encode sum types and under many circumstances will infer "Any" for a type.
It's still a nicer language and has some useful libraries that wouldn't be possible in Java or even Kotlin. When I got started in Scala I wrote basic spring-hibernate webapps, I didn't even use the Scala collections. But the language still made for more concise and readable code than the equivalent Java would have been. What do you lose?
Those two extremes hate on each other a lot, and this also spills out into annoyance at Scala not being ever more perfect for one of those extremes at the expense of the other. Then there are career enterprise Java programmers who are fine with Java and make it Scala's fault.
Scala community is not a monoculture, it's a refuge for different people with different ideas, and the vast majority of its users are quite happy with it and with each other.
But that also means that different Scala teams work differently, everyone's experience with Scala is greatly affected by the preferences of the developers they're working with. Everyone mostly agrees what a React.js application should look like. Not the same for Scala. You're going to have a tough time working with a Hascalator codebase if that's not how you think.
With preferences-compatibility warning in mind, Scala is still amazing for complex projects thanks to its rich type system and language features as long as everyone's on the same page for how they want to use Scala. And yes, it's overkill for your microservice that has seven classes and a plain JSON HTTP API. Use node or go or whatever for that kind of thing.
Scala is doubly amazing for complex full-stack projects thanks to Scala.js. These days I basically choose my stack from either full stack JS or full stack Scala depending on project complexity.
It's true but not unexpected that Scala 3 is taking a while. Good things take time. And once again, the amount of whining on the scala internet even about things like Dotty not being officially called Scala 3 (until that inevitably happened) was insane.
If you want to learn a safe-bet language strictly for career purposes, I'm not sure about Scala. The ecosystem is mature, the jobs pay well, and the users are very qualified on average in my experience, but it's a more niche job market than react/redux/node.js and all the other hip stuff. If you want to learn Scala for any other reason though, go ahead, it can be fun and productive. Odersky's coursera course is a great introduction.
In particular, none of the projects in this book take more than ~2 pages of text (~90 lines). You implement a programming language in <90 lines, a real-time network file synchronizer in <90 lines, a websocket chat website in <90 lines, a massively-parallel web crawler in <90 lines. These are 90 line complete applications, not "90 lines changed in a sea of boilerplate" or some other nonsense like that!
The goal of all this is to make a case that yes, you can use Scala for small projects! But you'll have to read the book to see for yourself if you find it convincing :)
Type safe pythonic simplicity is underrated.
Honestly that course completely put me off the language. The questions didn't match what he just told you and it was very difficult to figure out what to do. I gave up half way through and bought a Spark book to try to learn it, I might buy this one too.
If somehow you are convinced to use Akka because of high throughput streaming architecture, please just take a step back and make sure you understand what you are getting into!
Also, consider that while message passing as an architecture can lend itself to well-structured applications, it's not a panacea against issues like deadlock & livelock.
It also sounds like you may need some kind of distributed fabric for that kind of system and again, while Akka provides an elegant toolset, you still have to contend with network resiliency issues.
TBH, I have found biggest challenge in constructing these architectures is coordinating their completion or shutdown. You can up with thread leaks that will only show up through extensive testing or experience.
For simplicity, it uses a tiny Actor library instead of a big framework like Akka, but all the concepts and techniques should be transferrable to any Actor-based application
To me, scala used to own the niche of "if you must use a jvm language but don't want to use java". But now kotlin is the first language that springs to mind. What would you say scala is best in class for now?
Also with Scala you get a best in class JS compile target, a great scripting platform in Ammonite, and Graal native image for binaries with fast startup.
There is a huge difference now. Maybe 50% of that difference could be made up by what Java has in the planning stage.
Even if it was 100%, I'm writing code right now.
It all adds up to making me far more productive and more sure of my code.
- import aliases
- type members
- generic variance that doesn't make you want to gouge out your eyeballs
- generics over primitive types
- fully unified types
- an ecosystem that avoids nulls and exceptions
- pattern matching
- case classes
- everything is an expression
- immutable collections out of the box that most every library uses
- higher kinded types
- macros
Sure, maybe Java will get import aliases 20 years after they were due but I'm pretty sure half of that list will _never_ happen. Combined they are a huge deal for productivity, expressiveness, and safety. Java can't die due to the huge existing investments and continuing to improve the language is the right thing to do, but its really well past time for folks to consider more modern languages for new projects.
Apparently there are no plans to distinguish between JVM and Android on multiplatform Kotlin, as per their latest blog post.
Scala.js amazing. It just works. I've picked it up after multiple years of non-use and it everything worked perfectly with no surprises. I don't cover it in this book, but might write about it in another book in future!
Scala has always attracted an outsized amount of hate and FUD. That said, Scala 3 has taken a long time to arrive, costing the language a lot of momentum, and makes some fairly radical changes.
I'm a huge fan of Scala and I really want it to succeed. Haskell's laziness is a dealbreaker IMO (it makes it impossible to reason about performance), and no other language offers the combination of:
1. A type system powerful enough to replace all your uses of reflection/AOP/metaclassing/etc.. No more magic frameworks to learn; write real-world business applications using nothing but plain old functions and values that follow the normal rules of the language (e.g. your web routes can be literally a function from request to response; they don't look quite as nice as a dedicated routing config file would, but they're normal code that automated refactoring etc. will work normally on). Concretely, what makes this possible is the combination of higher-kinded types (letting you represent any custom effect, e.g. a database transaction, as an F[_] type, and then use the language's for/yield syntax to compose those effects in a way that makes them visible but not overly verbose/intrusive), and records (letting you "walk the object graph" in a type-safe way). Scala can do all this without having to resort to general-purpose macros (there is one macro in shapeless that the whole rest of the ecosystem just reuses), so you can be confident that your IDE/debuggers/etc. will all understand your code.
2. A proper first-class IDE, and a good declarative build tool integrated fully with that IDE (maven - for some reason the Scala community pushes the incomprehensible SBT, but ignore that).
3. Completely seamless compile-to-javascript experience. You write normal code, push the button, and there it is running in your browser. When you want to use a library you import typescript type definitions with https://scalablytyped.org/ and it all just works. Combined with the previous point that means Scala is actually the best language to use for frontend development right now. Unfortunately the set of people who know about Scala and the set of people who do frontend development don't really overlap.
I'm not wedded to Scala for the sake of Scala. But no other language comes close.
All that said, it definitely feels like the next year or two could be a turning point (in either direction) for the popularity of the language. Scala 3 may revitalise scala or destroy it. So it's undeniably a risk. But I'd say that even if the language becomes less popular, learning it will make you a better programmer and give you an appreciation for what's possible.
The meme is only relevant for memory allocations. In a lazy language you don’t always know what memory the runtime is holding on to as you haven’t consumed the full result of a function.
In practice, you only encounter this problem in very rare situations. I have personally never had to care of bad performance in Haskell in any of my projects.
In some projects I had to deal with high memory consumption problems. I could reason about why with ease: I was deciding very large JSON structures in hundreds of threads. Any other language would have had the same high memory profile.
def h(a) = {
val b = f(a)
g(b)
}
Printing out the result of g(f(a)) is 2 times faster than printing just f(a). In a non lazy language f(a) and g(f(a)) would take roughly the same amount of time.
In this example the expensive calculations take much longer than printing the output to the terminal. (no nitpicking please)
The performance can only be at most f + g. It may take less time in a lazy language, how is that difficult to reason about, and why would you care if it runs faster?
Imagine you've got a performance problem because f + g used to take x time but now takes 4x. You start trying to break down the problem, and you see that f takes 2x time and g takes 2x time. You check out the old tag where f + g takes x time and you discover that f takes 2x time and g takes 2x time. Where do you even go from there?
If anything, it can make things run faster, but if something is already slow, you just start by looking at the slowest function. You don’t even have to guess, the Haskell profiler will tell you what function that is, even with laziness involved!
Because the slowdown didn't happen in either function. f still takes the same time it used to. g still takes the same time it used to. But somehow the composition has gotten significantly slower. Both functions may be slow when run in isolation, but it's impossible to tell whether one or both is "supposed to" be slow. Profiling and looking at the slowest function can easily lead you on a wild goose chase here.
Could you formulate an example where the composition would be asymptotically slower only for lazy languages?
As I said before, laziness is actually an optimization strategy. If anything, it can only make the composition faster than it would be in a strict language (although rarely so)
No; we both know no such thing exists. The same code will always be asymptotically faster in a lazy language.
The problem is that lazy languages (or at least, the one lazy language in widespread use) encourage a style of code that would be asymptotically slower if written in a strict language. In a strict language, a function is slow or fast, and a slow function is always a problem - which means performance is easy to reason about. In a lazy language, a slow function is not necessarily a problem, depending on how it's used, and so you can't understand performance without understanding the whole program.
An unreliable optimisation can be worse than no optimisation at all. Worst practices should be hard. In theory an extremely vigilant development team could always write code that would be fast if executed strictly, and therefore guarantee no performance problems, but this would require an enormous manual effort since the language doesn't help you with it at all.
Sure. There's nothing to stop you using lazy values in a strict language if you want to; it's just that you get the option of using strict values as well. Whereas in Haskell you have no way of building strict constructions other than some ad-hoc annotations.
They implement the language server protocol and add ide-like functionality for Haskell to any editor that support the protocol like vscode.
I totally agree that the records system is annoying.
If anything, in a lazy language it may take less time, but never more than the sum of both.
How is it that you cannot reason about the performance of each separate function?
How would you solve an iterator in java producing infinite values?
If anything, the performance of a lazy language would be better than a strict one in the presence of an infinite sequence, as it gives you the opportunity to stop sooner and not enter in an infinite loop.
Example in haskell:
main = do
let a = [1..] -- infinite list
print (a !! 0) -- only prints the first element and exits
def f():
a = []
b = 0
for True:
a.append(b++)
return a
def g():
print f()[0] # Takes infinite timeIn practice an infinite loop in Python is often an immediate error (stack overflow) at the point where it's declared. In cases like you showed it's a little fiddlier, but any execution of that code will immediately show that there's an infinite loop (e.g. any unit test of it will time out) and a stack dump taken while the program is "stuck" will show exactly where the infinite loop is (which is therefore where the error is, because an infinite loop is always an error in these languages). Which is still not an ideal situation, mind; I'm very much in favour of an Idris-style language where function termination is actually checked by the type system and there's a type-level distinction between data and codata.
> How would you solve an iterator in java producing infinite values?
I'd avoid ever passing around a bare iterator, ideally with a lint rule.
The point isn't that it's impossible to have lazy values in other languages. It's that it's practical and idiomatic to avoid them, or at least limit them to cases where you absolutely need them.
The effects were only visible in production, as it was the only environment where the error condition happened. It caused thousands and thousands of messages to be queued in vain.
It took me, and another person 3 days to review the code line by line to understand where the problem was.
As you said, any execution of the code would show that there is a problem. Just that un this case no one new where it was executing from.
I think you should drop the argument that performance with a lazy language is difficult to reason in terms of time. It is well studied that asymptotically, strict and non-strict languages are the same.
It is a different story, and a valid argument for Memory consumption, though. It seems to me that you read the “impossible to reason about” meme and formed your own opinion of why that would be so. But this is not the actual reason.
If you write the same code in a strict language or a lazy language, the asymptotic performance will be the same, sure. But that ignores the fact that people don't. Lazy languages use different idioms (there would be little point in laziness as a language feature if people didn't write code that relied on it), which is what results in code whose performance is impossible to reason about.
Not because the language is strict, it makes infinite loops or infinite data structures more obvious to debug or understand.
In Haskell it would have been the same exact infinite loop.
There's a lot I don't like about Go, and plenty of things to be cautious about in message-queue-based systems; laziness is by no means the only thing I worry about. "decrementing a counter" suggests the kind of imperative control flow I would be very cautious about; I favour iterating/recursing over data structures, which is inherently safe against infinite loops (but only in strict languages with immutable datastructures!). I can't remember the last time I decremented a counter that was being used for a loop termination condition in the code itself; I've occasionally written functions like "run this block n times", but the most trivial test of such functions tells you whether they're infinite loops or not. Of course this is only really practical in a language powerful enough to write recursion-schemes style constructions.
If it is worth learning, the really depending on what you want from this language. It was and it would be the most novel language in the mainstream industry - it has almost all the features developers can dream of, and most of the features are not arbitrarily piled together like C++, instead, there are synergies between features.
With that being said, it's still a hard language to learn. And programmers from different backgrounds wouldn't interpret it in the same way. In real-world codebases, if the teams are not very good at training and aligning, it would be too diverse.
All in all, it's not only a get-things-done technology like Go, but it's pretty educating and mind-blowing.