What's Next for Scala
lihaoyi.com
lihaoyi.com
Dotty's union types are a thing of beauty, its implicits are really second to none, and _so_ damn useful. Their decision to embrace whitespace sensitivity, while simplifying the syntax for predominant use-cases (again, union types!) betokens flexibility, and a desire to swing for the rafters.
The only thing that saddens me somewhat is that Scala Native does feel like a bit of a second class citizen. The `stackalloc` feels a little sad (coming from C++ -- it should be inverted!). I wonder if it's possible for Typesafe to leverage Scala Native in a manner akin to Python with Numba/Cython. With whitespace indentation - native code inlined and FFI'd automatically - right there with the rest of the nice JVM stuff.
I can't resist citing my email from Feb '07 asking (politely) why we needed a 'new' keyword. Dotty apparently removes the need for that keyword. Should have listened earlier, team Scala ;)
http://www.scala-archive.org/quot-new-quot-keyword-td1989540...
That talk is almost a year old now :) a lot of progress has been made towards launching Scala 3. The latest Dotty release was March 18 https://dotty.epfl.ch/blog/index.html
Scala 3 combined with Scala.js and GraalVM native compilation will be a killer stack :). You can write code that's as high level as python and typesafe as Haskell. Use the same code on the backend and in-browser with Scala.js. Compile your code to statically linked native executables with instant startup time, great for CLI tools.
In terms of developer experience, Scala 3's compiler is much faster than Scala 2 because the data structure representing incremental compilation is not rebuilt for each compiler phase. The faster compiler can be used with Microsoft's Language Server for a delightful IDE experience, especially in VS Code which is the current recommended editor for Scala 3.
Scala 3 still in beta, lacks clear roadmap, allows choosing between two syntaxes (python-style vs braces).
Dotty Community Build lacking Lightbend frameworks (no Akka, no Play, no sbt)
With the recent layoffs at Lightbend of Scala 2.x core contributors, a full endorsement of Scala 3 would certainly help.
For me, one of the joys of Scala is that you can use different paradigms and mix them up as convenient. Haskell Scala is still Scala and so is Java Scala!
* True intersections and union types -- use them sometimes, not something you can't live without though.
* Trait parameters -- slightly easier life for OOP, ok I guess.
* Function types and implicit function types -- right, will make your code even harder to read (subjective).
* Generic tuples -- again, I can't remember we ever had problems with tuples in Scala.
* Opaque types -- another small OOP improvement.
* Type lambdas -- I've never seen them used.
* Erased parameters -- probably useful if it improves performance even at the cost of readability right?
* Enums -- finally something practical, good.
And that's all about major changes in Scala 3. Ah, yes, we get a new compiler. Are you excited?
These changes look like a quarterly release than a major version shift. Now, imagine a project manager who's estimating the migration downtime for Scala 2 for the sake of the above.
Support features of JDK 11+ https://github.com/scala/scala-dev/issues/139
Support Java Module System https://github.com/scala/scala-dev/issues/529
In comparison language features seem to attract much more attention:
Removing existential types https://contributors.scala-lang.org/t/proposal-to-remove-exi...
Significant indentation syntax https://github.com/lampepfl/dotty/pull/7083
Eventually they won't keep up with the underlying platform.
Java is a pretty safe bet, as it has an outstanding backward compatibility track record.
Edit: oh. And this is on top of a rewritten, faster, compiler, a unified base paradigm making it easier to learn the language, and better error messages for compile errors. I think dismissing this as as minor release is disingenuous to how much effort goes in such a large scale rewrite. Granted, most of it is under the hood but it will pay off dividend.
Opaque types are not a "small OOP improvement", they finally bring zero-cost newtypes to Scala, superseding value classes. Now you can finally ensure that object Ids, people's names, meters, feet, and degrees Celsius all have different types - without any extra boxing at runtime.
Your list is also missing typeclass derivation, which is going to be a fantastic quality of life improvement. Yes, even for people who don't care about what a typeclass is.
This is a point I want to stress the most, I think. Features like type lambdas, or libraries like Shapeless - they can possibly be extremely beneficial to you even if you don't use them directly in your code. Ask your nearby Haskell programmers about the virtues of `deriving (Eq, Ord)` and they'll talk your ear off. And that's before they even get to things like `deriving (FromJSON, ToJSON)`. Don't have a Haskell programmer nearby? It's ok, ask a Rust programmer about Serde instead. Typeclass derivation is just such a massive, massive feature to have in your language's repertoire.
I dont think this looks good on a language, I prefer a language that comes with a small, concise set of features that are broadly understood and applied. But different folks different strokes.
These two languages are impossible to write code in because of their complexity.
It's completely mystifying to me that even today, Martin still uses the "spec size" argument in his keynotes to attempt to demonstrate that Scala is a simple language, and it demonstrates a complete ignorance of what language design actually is, or a troubling disingenuity meant to deceive.
Wow.
Is it the language designer's place to say what can or cannot be done with it or the person/team who chooses what they do with it?
Personally, I would rather work with programming languages which allow me/the team to choose. YMMV.
I don't know about that.
To me, the criteria he uses to allow incorporation of features in both Scala 2 and Scala 3 is "Will this feature allow me and my students to apply papers for conferences and fund my PhD students?".
Language design seems to have very little to do with that. As long as you can derive multiple papers to multiple conferences from a proposed feature, it's in.
It also reminds me of the famous Kernighan quote: "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." [2] Scala might be optimal for some group of people, but definitely not for me, and I think not for a mainstream developer audience either.
It sounds like you've worked with a group of people who want to treat Scala like it's Haskell, but that's a minority community that IMO isn't really embracing the strengths of the language. I've worked on projects that go all-in on libs like Cats and Scalaz, and I agree that they're mostly unnecessary complexity that also obscure the modelling of the business domain. You're going to massively confuse newcomers with all the Applicative/Effect/Monad/Monoid/Functor/etc. talk, for next to no benefit. Actors are a bit of a different story - excellent for the 1% of times where you really need them (lots of mutable state and lots of concurrency), but 99% of the time you don't need them. Some devs are enamoured with the mathematical purity of pure functional programming, without properly considering how hard it is to understand for most other devs. If you let these types take over your company, that’s a cultural problem, not really a language problem.
Odersky describes why he created the language here: https://www.signifytechnology.com/blog/2018/01/why-scala
> ... programmers are increasingly facing new challenges for high-level domain modeling, rapid development, and, more recently, parallelism and concurrency. It seemed to me that a unification of the traditional object-oriented model with functional programming concepts was feasible and highly desirable to address those challenges.
I completely agree, and have yet to find a better language than Scala for the above demands, assuming you really embrace the OOP/functional mix. I've been writing lots of Scala at my day job for the past ~5 years, and it's my favourite language. I've worked on backends in Scala, Java, Go, Python, PHP and Node, and prefer Scala to all of them.
I also agree with Li Haoyi - it's becoming a really solid, stable, reliable language, with improvements focused on the build tools, compiler, and the odd language wart, without significant changes to syntax/style, which is great. It does take awhile to learn, and you do have to be careful about what style you program in, but I think if you just embrace the language's OOP/functional fix, and for "core style" mostly stick with the standard library (Future/Try/Option/etc.) vs. going crazy with Cats/Scalaz/Akka/etc. (unless you REALLY need Akka specifically), it's an outstanding language.
It's hard not to fall in love with Scala in the first months you start using it but downright impossible to like the language after that.
Kotlin has won my heart. So far, anyway.
Going full FP in Scala with Scalaz and "tagless final" (I still have no clear idea what's that) and whatever else that's hip, leads to madness. Sure you can write maximally polymorphic code. Great. But you end up doing the extractions/refactors that the type system and the compiler allows, not those that would help the code the most. Because those are too slow (too much boxing and GC churn), or the compiler has a bug (or the library) so you end up putting a TODO comment in the code with a link to a github issue.
Or it's a feature that's not supported by Scala, so you end up with a FP spaghetti. :(
1) First, Scala is designed and developed by "ivory" tower academics primarily for research/PhD thesis purposes. Nothing wrong with that. But that also means language features are added not to solve a practical problem but mostly to show that "it can be done"/"prove a point". When you are awake at 3 AM early in the morning because a release made in the evening went awry and you are trying to grep though logs/code to see what caused the issue I can assure you that debates about "pure" versus "impure", Monads, Kleisli arrow will be further from your mind. So the problem that Scala seems to suffer from is that of "image".
2) Second, Scala is neither a fully functional (FP) nor object oriented (OOP). Support for mixed/multi-paradigm sounds nice on paper but let me ask this question: do you know of any hard core FP programmer who is excited about inheritance or abstract classes or other OOP features? Vice-versa the same thing can be said about many OOP programmers lukewarm response towards many functional features of Scala. I believe that support for mixed/multiple paradigms has hindered large scale adoption of Scala. The language seems to be caught at cross-roads not knowing which way to advance next.
3) Third is lack of niche. True, I absolutely love "Spark" and primarily use it to extract/manipulate data from HDFS/Hive storage but I am gradually gravitating towards "Python" libraries due to their ease of use. What about better Java? Kotlin check. What about sidecar systems used by DevOps? Go check. What about a better designed system language? Rust check. The days of pure general purpose programming languages are truly over and Scala will be in great peril if it fails to expand into newer niches. Without broadening its programmer base it will finally run out of people that can design newer language features in light of ever advancing progresses in processor, OS and memory architectures.
Just my two cents.
Scala is a heavy weight languange. By heavyweight I mean,
1. Even intellij struggles to figure out the meaning of implicits in the given context. And then there are macros.
2. Way too many theoretical concepts, monads, monoids , isomorphisms ect that make you feel stupid. There is always a nagging feeling that you don't 'get it' when you program in scala. Here is an example of scala conference in 2020:
https://scaladays.org/2020/berlin/schedule
Features talks like these.
> a. Composition with Functional Effects
> b. Scala 3 and Abstract Algebra
> c. Describable, typesafe computations for the less monadically inclined
> d. Understanding Scala's Type System
> e. Roll your own Effect System
3. Every code base is written in its own flavor of scala.
I was spending absurd amount of time "learning scala", now i look back at that and cringe.
Scala is a fusion of object-oriented and functional programming. There are at least three major styles of Scala programs: OOP, functional, and reactive (actors). Scala indeed places the burden on the development team of choosing which patterns are right for them.
The theoretical concepts that you disliked are "available" in other languages too, you just don't hear about them much because functional programmers prefer languages with strong type systems. That's not to excuse the learning curve they create! I still don't understand them too well. Some of the people who do understand them will unnecessarily push them onto their projects or teams. Scala doesn't force these on you, you can ignore them. For example I write a game engine in Scala that is mostly imperative-style and has limited use of the theoretical constructs you mention.
Python's lack of a strong type system is a serious deficiency for large projects. Same with JavaScript. The costs it creates are large and often undetectable because they are the zeitgeist and death by a thousand cuts.
Or... have one way of doing everything (python, go, Rust, even Java) and simplify. That simplicity means 3rd party libraries will all largely have the same basic constructs for how things work, which makes it far easier to learn, which makes it easier to contribute.
Simplicity comes at a cost, compare Elm to Purescript/Haskell for example. Elm lacks type classes, so it’s simpler and more explicit, but that comes at the cost of verbosity and duplication. This was an explicit decision to make the language more friendly for beginners, and that’s a reasonable trade off to make, but it’s not universally better.
Yea but one cannot write everything from scratch. I had to work on project that use cats, scalz and stuff. I felt like i was splitting an atom when i was actually writing a crud app :) .
Obsession with those theoretical concepts with dubious and unproven benefits is deeply interwoven in community. Cost of learning cats vs benefit is just absurd.
I would only recommend scala to programming language nerds that want to sound smart.
I did a bit of Python, then did a bit of Scala, and now do both at work. Oh what a pain Python is! And what a joy Scala is!
> Way too many theoretical concepts, monads, monoids , isomorphisms ect that make you feel stupid.
Concepts are theoretical in their nature, aren't they? Perhaps these concepts are useful and you just haven't yet found out why?
Wrt feeling stupid, there are two possible reactions: "gee, I'm not so smart" and "whee a learning opportunity" - I'll prefer the latter anytime.
> There is always a nagging feeling that you don't 'get it' when you program in scala
Yes I have that too - so much more to learn!
Or maybe they are not that useful? If they were really that useful we would see those concepts everywhere.
monads : stuff which kind of encapsulate other things, but you can still nicely manipulate both the thing and the stuff. Let's say, lists.
monoids : set of things where you kind of can add one to another. Let's say, numbers
isomorphisms : functions where there is exactly as many outputs possible than there are possible inputs. Let's say, adding 1 to a number. It's kind of useful knowing whether or not a function is an isomorphism (hello, hash collisions !).
Those concepts aren't specific to any programming language in fact, and are in fact everywhere. You just don't need to explicit them to use them, expliciting them is an option (even in Scala) which allows you to do meta manipulations on them (which Scala doesn't require, and doesn't even really suggest since none are part of the standard lib).
I would expect Clojure programmers (while still functional) don't use these abstractions either.
On the other hand, typed languages, like Haskell, Scala, OCaml, F#, do employ these abstract concepts. Their type system makes these worthwhile endeavour with great payoff.
Also, I just clicked on the most recent conference on clojure.org : https://clojured.de/schedule/ , and there is a talk mentioning monads while promising to not talk about monads :-D So it kind of reinforce my point that basic theoretical concepts are silently everywhere ^^ (and fully not necessary for almost everything).
But I don't think conferences are representative of the "on the ground production experience" that you have, or at least can have.
I personnaly wouldn't submit a paper to a conference with a talk "Here's how I made a whole backend stack using boring scala, and it just works" because as you say, I feel it's just full of theoretical speakers and that wouldn't interest anyone. Meanwhile, I also did code a whole backend in boring scala, and it in fact just works, and it feels good :-)
Personally I liked it. I spent a while doing java web dev with the play framework, and also spent a while doing scala web dev with the play framework, which allowed me to compare directly. I have to say scala is a far nicer OOP language than java. The collection classes are better designed, the way futures and options work is so much more elegant than the java counterparts, and the overall syntax is much more efficient. IMHO if you just use scala as a nicer java, it absolutely trounces it. I couldn’t really say what OOP feature it lacks compared to java. I never tried kotlin, so I cannot tell how that compares.
I also spent a while doing FP style programming with scala in spark and akka. I cannot compare it against other FP languages because I never used them but I can say it was similarly very enjoyable and it was easy to write elegant and understandable code.
I would agree mixing both approaches in a single codebase is a path fraught with peril. If scala has a flaw then it is having too many features. Like C++ you need to be careful which features you use in which combination, or you can end up writing an intractable codebase.
After a while the project became just a dick measuring context of who could do the best trick shots. Declaring a lot of implicit parameters, implicit conversions automatically applied, crazy type inference that makes compiling 10x slower, you name it. Just to make the code unreadable enough that other developers need days to understand a three lines of code change.
And after all of that the more they do it, the more they are seen as the best Scala developers on the team.
To wit, 15 years ago I was on a team where it was “How J2EE can you get?” Ughh. Along the way they(we) basically forgot to solve the business problems.
And that sion was the end of that.
Being an undergrad doing my dissertation into programming language design, it seems that FP evolved from an area of research that was very much tied to the theory of computation and never really danced with the practical side of software engineering. Of course there are a few languages that have tried to marry the two (OCaml, Rust to an extent) it seems to very much a mathematicians game. Of course I love the math but it gets annoying to explain that no programmer should have to have a working understanding of category theory to use a language effectively.
The language is too flexible with too many ways to do the same thing, increasing cognitive load and making things difficult to understand. Maybe you can limit that in your own projects, but what if you have to look at a library codebase that's structured completely differently. Trying to understand that can take a lot of time.
Also there is the problem with hiring. Maybe your current team can work well with Scala, but what if a key member leaves or a service gets transferred. Hiring for Scala turned out to be impossible, so you get all these people that need to learn Scala first. Normally that may be ok, but Scala has a pretty steep learning curve so it takes a lot longer to onboard people.
That said, "First, Scala is designed and developed by "ivory" tower academics primarily for research/PhD thesis purposes." I totally agree.
Error messages when using libraries are horrendous with dozens of lines of type errors (take a look at Elm how it's done right).
Compile times are horrendous. There are whole consulting companies [1] which do nothing than help companies with compile time of their Scala projects.
The article talks about 2x speedups over the years. Which sounds nice, but if you're 10x slower than others (Go, Rust, ...), then with a 2x speed up you're still 5x slower. Which is fine far small projects but kills you with large ones.
Would come back if the compiler is rewritten in Go (which will not happen because Scala is primarily a research project).
I now use TypeScript with Wallaby.js and Quokka developing test first. It's like night and day. I do miss Scala, because I really love the language, but I would not go to the pain of compiling and executing unit tests again in Scala.
[1] https://triplequote.com/ "Compile Scala Faster"
2.) When founders ask me about what language to choose, I tell them: "Choose one where you easily find developers, then you can't go wrong".
As an example, we know that PHP is a "fractal of bad design"[1], but I can't deny that Facebook as a business is doing just fine.
[1] https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/
2) Plenty of FP developers enjoy having access to the JVM and its huge ecosystem. I can trivially use and wrap most Java libraries, even the ones that overuse outdated OO patterns.
3) There are still enough companies and teams out there that don't just want a better Java. We'd see a decline in job openings if they were massively going back to Spring with Java or Kotlin, but that's not the case
Calling Scala an ivory tower with no interest in ‘practical’ programming is ridiculous. Scala and Java are remarkably similar in this respect (note Odersky was heavily involved with Java before Scala), albeit one is sponsored by a uni and the other a corporation. JEPs are just as likely to have actual PhD theses associated with them, and those that don’t often could have. Look at Project Valhalla or Loom or the GraalVM, these are novel programming language research projects with the end goal of bringing them to a widely used industry language in a backwards compatible and ergonomic way.
As for pure FP in production, just last week we had a brief bit of message loss in a system because some layer in an sdk swallowed a socket timeout exception and re-threw something totally unrelated and undocumented. This was hell to debug for other reasons related to imperative code, that simply wouldn’t have happened in a ZIO project.
That’s not to say you can’t make reliable software in Java, my team spends a lot of time doing just that. It just often feels like the whole ecosystem is fighting against you. Things are getting much better in Java land, but there are parts that can never catch up with Scala because of past design choices.
I agree with your other 2 points, multi-paradigm is both a blessing and a curse and probably becomes more of the latter in a large org.
Also, unrelated, but imo better Java is Java or maybe Java in 3 years, Kotlin didn’t go far enough.
My advice:
- Options, immutability, pattern matching, and list comprehensions are all awesome and better than the default Javaesque way
- Scala concurrency abstractions all better than Java, but concurrency is still hard. Proceed with caution
-- Except parallel collections. Use those liberally
- No Scalaz or any other FP-crusader stuff ever
- Macros and implicits should be rejected by default in code review. Amazing justifications required
- Enforce coding rules with linters. This applies for any language, but especially important for dynamic and non-straightjacket languages
So what happens when you are dealing with multiple option types, like you two optional ints you need to add together? Or you have an optional field in an optional object? This happens all the time in code that heavily uses Option types.
So do you match on every one, and nest match expressions inside match expressions inside match expressions? That can get very, very messy and even confusing.
Although yes there are other more complex instances in which nested matches might come up. Generally I would think to handle them by creating a function that contains the next layer of matching instead of trying to come up with an uber function. Adds a bit more verbosity perhaps but I find it is fairly easy to follow and show intent.
What I should have asked was: if you're using monads such as Option, then how do you use them effectively without using integral tools such as flatMap?
And you've answered: you can use for comprehensions. I primarily have used Haskell and OCaml, so I sometimes forget about Scala's very nice for comprehensions, which I agree is a great solution. But note that for comprehensions are just syntactic sugar for `flatMap`.
I agree that if you do use Option types in Scala, for comprehensions are a very elegant way to interact with them.
But I'll end with a question. If you're already using option types with for comprehensions, why not also throw in other useful monads like bifunctor IO, which is a great way to deal with async code? I understand the dislike for the tendency in the FP Scala community to use custom sigils, but I do think most people who are using options and for comprehensions would also be comfortable and happy using other useful monads with for comprehensions.
You seem to vastly underestimate the gap there is between using a for comprehension and understanding what a bifunctor is.
Nothing wrong with that: once you gain a certain advanced knowledge, it's often very hard to put yourself in the shoes of someone who hasn't gained that knowledge, and even harder to explain it to them in terms that they will understand.
Which is why we have so many unhelpful monad tutorials.
> No Scalaz or any other FP-crusader stuff ever
That stuff which some people might consider FP-crusader vanity is something which can make concurrency manageable, maybe even joy.
I encourage you to open your mind, at least for a bit, and have a look at ZIO or Monix.
> Enforce coding rules with linters
Absolutely! Scalafmt and Scalafix for the win!
Implicit parameters are nothing more than a terminal argument list which the compiler attempts to provide from "the implicit scope." If you would rather it not, then simply provide them yourself at the point of invocation.
To wit:
object HitchhikersGuide
{
implicit val answer : Int = 42;
implicit val question : String = "the universe";
def foo ()
(implicit a : Int, q : String)
: Unit =
System.out.println (s"${q} = ${a}");
// This
foo ();
// Is the same as this
foo () (42, "the universe");
}
HTH
0 - https://docs.oracle.com/javase/7/docs/technotes/tools/window...
I wrote an example using this new (optional syntax) https://github.com/olofwalker/ting, if anyone wants to have a look.
To me, you have to be intrinsically motivated to become productive in Scala because of that early friction. Which is fine if you are, but if you're looking at a team of programmers and thinking "how do I get them all over that line" when about half of them have no intrinsic motivation to do that, it's a real disincentive to use the language.
Lightbend secures 25MM funding round, proceeds to layoff Scala contributors https://news.ycombinator.com/item?id=22842166
Also, when I code in Scala, I frequently google and end up learning something new about the language. I know it sounds like distraction/wasting time, but there's a certain satisfaction when I found the "Scalaic" way of doing things, like, "There's gotta be a one-liner way of doing this."
Of course, my complain is still compile time (yes it's significantly faster, but I'm impatient).
Btw, the actual thing to be inspired with in Scala is immutability and Option/Some/None instead of null. These are the things that make you code stable and reliable.
But: 1) the ecosystem makes it seems like I need to be a genius enough to learn a new DSL a bit too often, and everything is so...complex, which leads to...
2) It is hard to pitch to an organization because of the learning curve if you assume any project maintainer must learn scala to expert level. I failed (in one attempt on a scala mailing list) to pitch to the scala community the idea of having language levels: a compiler switch or the like that would guarantee that a given project will be maintainable by java programmers with only N (1-2 I hope, for the first level) hours of scala training or study, maybe another level for adding certain features (or listing them in a config file), and full-on expert. So teams can be formed with realistic long-term maintainability in mind. Edit: This may be practical with add-ons I didn't try yet.
3) It was unclear what would be the best reasonable, maintainable way to use sqlite. (Edit: in a multiplatform way to include at least BSD, linux, windows, mac. Maybe call a native code library for each? Sounds like lots of work and maintenance to do it well.)
4) Startup times, future viability of scala native (edit: or equivalent), and whether it helps or adds enough more complexity given all the above, to just learn Rust and (hopefully) simplify life (in some ways).
5) And you have to learn java first (for those who didn't know it already), and know its libraries and idioms to some extent. Will that still likely be true?
Comments welcome. (I enjoyed the article.) Maybe I'm just the type that wants to hit the sweet spot between what C, Java, and Scala do (edit: and Scheme), in a way that is hopefully/eventually at a complexity level that is inviting to other devs, but with plenty of learning headroom for my future growth.
One of the coolest things about Scala it that it already contains lots of Java best practices. Case classes, singleton and companion objects, traits, powerful generics... almost every Scala feature is a builtin answer for a Java pattern. Hell, even implicits in all its various forms are elegant solutions to common Java problems. People getting creative with these features is an entirely different issue. It's possible, and desirable, to use Scala to craft simple, elegant, and understandable code. Fancy but unmaintainable code should not ever be accepted: it's the job of the developer to teach the reader what the code does, and while this is true in any language, is more important when coding Scala, because the multiparadigm approach means there are multiple ways to solve things.
What you have to learn, at least in a basic form, is how the JVM works.
And the maintainability issue seems similar to perl vs. python in that way: it can be done, but I've found it takes much more work and long-term attention & presence, to enforce things with code reviews than with a project-level config that everyone knows why it is there. Just once when you are not looking and now you have a new (large or small) social and technical headache.
(Edit: in other words, while code reviews are essential I think for many reasons, any time there is a way to enforce something good, before the review, it seems like a win.)
It's one of the few books that is structured in the same way a Scala developer thinks. Lots of Scala books follow the same order: keywords, usual syntax, features also in Java, features exclusive to Scala. I think this is damaging for any person looking to learn Scala, because it does nothing to lay out the way the features interact together. for instance, the Oreilly book shows pattern matching and implicits before class hierarchies, and that maps well with how Scala devs work, because the former features are encountered daily, while the latter tends to be uncommon.
As for maintainability: you're right. Stricter languages are more maintainable. There's also the issue of powerful languages attracting certain kinds of people. Over the years, I've settled on a simple policy: reject everything that is not documented. This goes for both internal code, and external dependencies.
Example connecting: https://github.com/tkawachi/sqlite-scalikejdbc-test/blob/mas...
Edit: ps (maybe a note to myself as much as anything in case I go look up this stuff): And does sqlite run in the same process as the app? Can the app process control the db location?
I know Android uses it a lot, so I opened up an Android project at my company and it looks like the android SDK comes with a bunch of SQLite abstractions, and it's not clear what the underlying driver is. But the SQLite official website ships an Android AAR so maybe that is it? (https://www.sqlite.org/2020/sqlite-android-3310100.aar)
This seems to be the main general-purpose JVM lib: https://github.com/xerial/sqlite-jdbc and it seems to be keeping in lockstep with the official SQLite versions (within a couple months).
Edit: Also can you elaborate on the "in a multiplatform way"? Maybe I'm having a brain-fart or memory-hole but shouldn't you not have to worry about that at all?
Edit2: Looks like there is a nice blurb in the docs on that github page:
Since sqlite-jdbc-3.6.19, the natively compiled SQLite engines will be used for the following operating systems:
Windows (Windows, x86 architecture, x86_64)
Mac OS X x86_64 (Support for SnowLeopard (i386) has been deprecated)
Linux x86, x86_64, arm (v5, v6, v7 and for android), ppc64
In the other OSs not listed above, the pure-java SQLite is used. (Applies to versions before 3.7.15)
If you want to use the native library for your OS, [build the source from scratch.Edit: yes, I think it does not currently run on OpenBSD, etc. One could contribute that library, but in that case, I'm more inclined to learn Rust and not deal with the JVM at all. Then I also get convenient access to things like pledge() and unveil(), and other OS-specific system calls, when I want. :) I will be grateful, however, for further correction & enlightenment... :)
(Edit2: I have been reading "the book" in Rust, but probably should write a test to confirm that it will make calls to sqlite and pledge() as directly as I think, using the tools I found.)
For example, implicit conversions are now disabled by default. It's possible in Scala to pass a Foo to a function requiring a Bar due to a chain of automatic conversions from Foo -> Baz -> BazMutable -> Bar. This is fun and magical but difficult to understand and debug. The community and language itself now suggests against this pattern.
Another example is that it's now (more or less) recognized within the community that it's not preferable to dogmatically adhere to a particular style of programming, such as pure functional or reactive/actors. It's better to use the right tool for the right problem.
Whole classes of confusing code are now prevented or mitigated by language changes in Scala 3. For example, Scala 2's implicits are famously hard to learn and follow within a codebase. One reason for this is that in Scala 2 the keyword "implicit" was overloaded into a few common patterns, including the aforementioned implicit conversions, typeclass instances, and dependency injection. Scala 3 introduces new constructs and keywords to split out these patterns into separate language features, making them easier to learn and more understandable within a codebase.
I don't have experience with Scala 3 or Scala 2.13, but I don't believe this is true. Implicit conversions cannot be applied more than once.
Oh God I'm having flashbacks to using Lift in... 2012? There were about four levels of implicit conversions between my model and the final JSON, trying to understand that was hard.
That said, David Pollak the founder of Liftweb was amazingly helpful on on their mailing list.
The code written by experts were at a whole another level. Couldn't make much sense of them. Maybe I could understand one line at a time, but the design patterns appeared hard to understand after my ~12 months Scala experience and over a decade of programming.
That said, the expert code looked amazingly beautiful :)
And since we spend way more time looking at and interpreting other people's code than writing our own, personally I think Java is better for large projects. I wish it wasn't the case but it is.
GraalVM native compilation works really well. Many projects will compile to native out of the box. Many more with a bit of reflection config that's autogenerated. The native executables are slim, statically linked so they "just work", and startup instantly as one would expect.
Scala is now a great choice for CLI tools.
What's the size of hello_world.exe?
Not that slim.
$ \cat hellogo.go
package main
import "fmt"
func main() {
fmt.Println("hello")
}
$ go build hellogo.go
$ \cat hellonim.nim
echo "hello"
$ nim c hellonim.nim
...
$ ll
total 2.2M
-rwxr-xr-x 1 kbd staff 2.1M Apr 11 20:55 hellogo*
-rw-r--r-- 1 kbd staff 66 Apr 11 20:42 hellogo.go
-rwxr-xr-x 1 kbd staff 87K Apr 11 20:55 hellonim*
-rw-r--r-- 1 kbd staff 13 Apr 11 20:29 hellonim.nim
Last I looked (a couple of months ago) the executable produced linked dynamically but there was a flag to link statically.
I'm hoping in the future the native-image friction can be lowered, but as of now it's not trivial like other languages.
If I remember correctly you have to redistribute some .dll (VC something) file alongside your .exe on Windows since fresh Windows installations don't have the .dll. Did that change?
AFAIK Typesafe/Lightbend has made no commitment to Scala 3.
Too bad the roadmap for Scala 2.14 has been scrapped.
I am very excited about Scala 3 :)
Scala 3 isn't a python2/python3 situation. It will take years for large codebases to be upgraded to Scala 3. However, the tools to migrate code from Scala 2 to Scala 3 will be reliable and highly automated, due to Scala's strong type system. Most popular Scala libraries and frameworks will be available on Scala 3 right from the get-go. I expect that most Scala programmers and companies will want to write Scala 3 code asap.
Within the Scala community, the only truly divisive aspect of Scala 3 afaik is the introduction of optional python-style and Haskell-style syntax with significant whitespace. I was stridently against significant whitespace and now prefer it after having tried coding Scala 3 for awhile.
I stopped using scala around 5 years back, want to get back to it. Would you recommend jumping straight to dotty? I’ve been doing a mostly python and a bit of Haskell in the meantime.
Which for production level code bases means never. I can't even imagine what we are going to do with our 7 year old Scala code base.
Six years at Lightbend, by Eugene Yokota (Scala sbt, scalaxb, scopt) https://news.ycombinator.com/item?id=22836399
Likewise, on top of the real problems created by ill-conceived adoption, the irrational backlash against new paradigms is real as well. I'm old enough to remember the fuss over OO programming. Many experienced, successful coders were scared that they were about to become personally obsolete. Some OO proponents insinuated — and panicked veteran coders believed — that a large number of procedural programmers would have to retire or switch careers because they were too old or not intellectually nimble enough to adapt to object-oriented programming. Major companies such as Sun responded to these fears with a marketing blitz to prove their their OO technology was simple and easy to learn and would herald a new age of programmer productivity. Programmer culture was more driven by magazines and books in those days and therefore much more easily influenced by industry marketing. The "simplicity" of Java was an invention of Sun marketing, a successful self-fulfilling prophecy as Java became so familiar that similarity to it became millions of programmers' benchmark of simplicity.
There has been no total war marketing blitz for functional or actor-based programming from a company with the same stature that Sun had in the 1990s. The emotional reaction, the resentment at feeling a step behind the future, the anxiety about whether you will ever be as comfortable and confident in the new paradigm, operate virtually in a vacuum, or worse, in the echo chamber of the internet. Whatever latent fear of functional programming a person feels, the prospect of using Scala will bring it to the fore. The only countervailing force is the (usually lone) individual pushing adoption, and they are unlikely to be up to the challenge. There is a tension between programmers who are more oriented towards business problems, or who just want to get their job done and go home, and the programmers who get excited about different and potentially better ways to express programs. They both think of themselves as humble and results-oriented and have a hard time accepting each other as being equally humble and results-oriented in their own way. They both experience the other as snobbish, condescending, and threatening to their self-image as good engineers.
Sadly, the root of the problem is that Scala is expressive and powerful enough to be a natural vehicle for these new paradigms. An engineering organization should not fear expressive power, because they should have a culture that is incompatible with a single engineer (or a small cabal) committing their peers to a learning curve without consultation, debate over the cost and payoff, and buy-in from the team. Unfortunately, there seems to be an assumption that beyond choosing a language, you have no ability to consciously choose a style. I.e., Scala-as-a-better-Java (the least imaginative approach, but not a bad one) is an impossibility. This is grounded in the assumption that any aspect of a language that can be abused, will be abused, and that engineers cannot make collective decisions about how they will work together, which are odd assumptions to make about the behavior of highly paid professionals. As someone who wrote C++ professionally for many years, I can testify that it is possible for engineers to conform to coding standards that restrict them to a strict subset of the expressive power of a language. However, it sometimes requires management authority to bring people in line -- and perhaps this is a deal-breaker in an industry where teams of engineers are supposed to be entirely self-managing.
That being said, even though Haoyi said he doesn’t think Java has “caught up”, the fact is it has closed the gap significantly on the language side, and will always remain ahead in other ways (performance as one example). OpenJDK 14 is an outstanding general purpose development platform, and the days are long gone of anyone actually _needing_ an alternate JVM based language, whereas 10 years ago (or maybe even 5) there was a desperate need.
This is also being taken over by Python. Databricks Koalas is a pandas interface on top of Spark - https://koalas.readthedocs.io/en/latest/
PySpark was already there (and very much in production). But Koalas (in beta) completely changes the go-to production aspects of spark.
I've also had the experience of using it on undisciplined teams and that sucked. One overly "clever" programmer can turn off a sufficient number of other programmers and give it a bad rep. I don't think a language being expressive is sufficient cause to blame the language for that, though.
https://trends.google.com/trends/explore?cat=5&date=all&geo=...
Early 2017, perhaps! Are Go and Rust seen as competitors to Scala?
Another tip, by default Google Trends just show data from the US, which is not very interesting, select Wordwide to get a better picture: https://trends.google.com/trends/explore?cat=5&date=all&q=%2...
Competitors are hard to say as there would different categories. Some workplaces would only choose a JVM language, then Scala's competitors are Java itself, Clojure and alike.
Go and Rust have mainly been described as "system" languages but they take vastly different approaches from each other.
If you be a bit more specific on what you're compared them on, it'll be easier to know if they are competitors or not.
If you viewed Scala as a competitor to Java, then Go is a competitor to Scala and not Rust; because Go is a competitor to Java in my opinion.
If Go 2.0 ever happens, it will be a competitor to Java 5.
Go is an imperfect match for Java The language, largely because of the whole Generics thing. But if you’re talking about how teams use Java and Go, they’re a good match now.
Go was designed for large companies like Google to have a huge number of engineers and produce consistent, quality code, all while constantly on boarding people and moving them around. From an organizational perspective there are a ton of Java shops that use Java for similar reasons, plus the ability to cheaply and quickly acquire contractors who know Java.
If you are talking about doing REST APIs, talking to Postgres in Java 1.0 servlets style code, yeah Go is a replacement for Java.
Except that is like 1% of everything that is currently being done in Java.
As a Java guy, i would agree that Go is not actually competitive with Java. But then, a Go guy would probably say the opposite.
After getting generics, Go still needs to offer JFR/JMX/VisualVM like monitoring and dynamic code loading capabilities (Go plugins are very limited), JEE/Spring like Web tooling, Liferay/AEM/Magnolia like CMSs, an OS of its own and real time GC, card chips, M2M hardware gateways, printer enterprise configuration appps, and plenty of other use cases that many on HN seem unaware of its existence in enterprise scenarios.
So where is the EA for generics like the value types EA I can download today from OpenJDK repository?
It still appears quite far from being done in 2021, though.
What does this mean? I thought Clojure has a dedicated, if small, user base.
As a Clojure/Script developer, I could understand that Clojure looks dead from the outside if you're a JS developer, you're used to new language features being shoehorned into the language every year.
But for someone inside the Clojure ecosystem who used to be a JS developer, it's a breath of fresh air when I can take a 4 year old library and include it, everything works and no bugs are to be found. If there is a bug, I can patch it from the outside, if I really have to.
Clojure in general lends itself very well to be extended from the outside (like what clojure.spec is currently doing [and many others]) so the core language doesn't really have to change anymore, we're just building libraries on top of the language which extends the language.
I'm a JS developer and I admire the way Clojure has seemed to skip both the hype and disillusionment phases and jump straight to the productivity plateau.
In the end, we will never know the "real" numbers, and developers like me who use Clojure and find themselves most productive in it, could not care less either.
Because of it's dynamic nature, there is no plateau in Clojure (lisps in general), the responsibility just shifted to outside the core language that Rich Hickey et al maintains.
Oh, no, as a developer who dealt with web front-ends for a long time, I can attest - it is totally justified.
Modern Javascript, in just a matter of a few years from being a small, simple language with a few quirks has blown into this colossal monstrosity. ECMAScript specification today is almost as big as Java's. There are tons of things that were added ( and still being added ) without careful thinking and upfront design. And that not to mention things that break all the time in the ecosystem of any JS framework. Have you ever tried updating dependencies in a nodejs project that is just a few months old? It rarely is simple and straightforward. Things break left and right. It feels there's no stability at all, compared to Clojure(script) projects. You can pretty much pick any Clojure project (that is a few years old) and update its dependencies, and most of the time, you can expect things not to break.
People got excited about React hooks, and Clojurescript people were like: "folks, we've had a better way of dealing with stateful components for a few years now."
JS devs got excited about destructuring (which looks like a weak attempt to borrow it from Clojure), without even slightest hint of how much more straightforward and cooler destructuring actually looks and works in Clojure.
They say they love JSX, which, compared to Clojure's Hiccup, is just a total and absolute crap.
JS devs are eager to use nulish coalescing and optional chaining and write unreadable crap like: `input ?? obj?.key ? 'yes' : 'no'``. And they dare to say that Lisps are not readable? ¯\_(ツ)_/¯
Honestly, after years of dealing with it, my take on it is this: Javascript is a morally outdated programming language. And Typescript is somewhat not the best attempt to fix its issues.
It doesn't serve anybody's interests: for all of us, nearly everything we are passionate about in software will be declared "dead" even one day, with this logic. I usually interpret someone saying "X is dead" as saying more about the person expressing it than the language or software they are talking about.
It never ceases to amaze me. Almost every single time, whenever there's a post or a comment on HN that mentions Clojure or Clojurescript, there's always someone from outside of the Clojure community attacking the language, saying things that are simply untrue. Like people are afraid of it becoming too successful. It's not some zero-sum game - the success of one language ecosystem doesn't mean that your other, the more favorite tool would be immediately forgotten.
Somehow I don't see Clojure people attacking other languages; they instead learn and borrow from them.
- Python people would be like: "you can't do machine learning stuff...", and what Clojure people say? Instead of fighting Python, they're like: "We can¹, and also We have nothing against Python, we'll figure out a way to talk to it from Clojure.²"
- Golang people would be: "CSP be is cool," Clojure people: "Awesome, we'll borrow it just the way how you did it. Also, we like Golang, we're going to write a Clojure interpreter and linter in it.³"
- R people: "Clojure is data-driven? Pfff... you know nothing about statistics ...." Clojure people: - "Oh yeah R is cool, btw. we've just figured out a way to interop from Clojure⁴."
- Elixir people: "BEAM is way better than your JVM crap...", Clojure people: "Not really, but okay, we see your point, we'll build Clojure on BEAM⁵"
- JS/TS people say things like "Modern Javascript is so nice, who needs Clojure now?" They get excited about stuff like React Hooks, and Clojure people: "Hmmm... actually we had a better way of dealing with stateful UI components for a few years now. We've been trying to tell you, but you never listened..."
They've been saying, "Clojure is dead" for years now. But despite all the odds, Clojure is very much alive and thriving. It's the most popular language within languages with a strong FP emphasis. Just look around - it has more books, more jobs, more podcasts, more conferences, and meetups than of Haskell, OCaml, Purescript, Elm, Elixir, F#.
I suggest people from other PL communities learn from Clojure people. It is a small but diverse and very friendly community of pragmatists - while others keep shoveling the same shit from one corner to another, trying to re-invent things over and over again, Clojurists keep borrowing good ideas and filtering out bad ones. And calmly and quietly keep building cool stuff and getting paid for it⁶.
² https://github.com/clj-python/libpython-clj
³ https://github.com/candid82/joker
⁴ https://github.com/scicloj/clojisr