Towards Scala 3
scala-lang.org
scala-lang.org
Yes, some people who previously used Scala, now use Kotlin. And some people who would've used Scala if Kotlin didn't exist, use Kotlin. Same probably goes for Rust.
But there is a big enough market for people that like the intricate and expressive type system that Scala gives you, in combination with the JVM ecosystem. People that think Kotlin is nice, but not expressive enough, for example. People who don't want to deal with Rust's memory management and/or don't have a use for that. People who think Go's simplicity is sometimes more of a burden.
Why are people so intent to bash sombody else's language choices?! Go, Rust, Kotlin, Scala are all great languages in different ways. They all cater to different needs, sometimes radically different (Go vs Scala), sometimes subtly different (Kotlin vs Scala). I think there is a market for all of them, and more. And the introduction of a new language (Kotlin for example) does not necessarily spell doom for another (Scala).
Let's all enjoy our own tastes and needs, and respect those of others.
I've been around scala long enough to see the rise and fall of multiple expeditions into the bowels of the OSGI eclipse cave of horrors (Sean McDirmid / Miles Sabin etc). Before switching horses and settling in Intellij for several years. So I understand where you are coming from.
I've since jumped ship again to vscode (along with typescript), and in my humble experience / opinion, vscode & it's language server protocol (LSP), alleviates quite a bit of this. It abstracts away a lot of common operations from a language to the languages compiler, while providing a consistent front-end. I think if scala eagerly adopts supporting this it would be a good thing. Having a language server protocol applies back-pressure in a certain sense on language features, if you are going to add them, then the tooling (and LSP) needs to support them.
In addition, an argument for "my language is better than X" can usually be easily countered with an argument for the opposite, creating a net result that doesn't help your goals. Debates are sometimes interesting and informative, but they're not effective marketing.
That's just, like, your opinion, man.
(By which I mean: I disagree, and we both know there's no clear evidence one way or another)
> To increase the appeal of a language, the best things you can do is create great tooling, lots of useful libraries, and a friendly community.
I do what I can, but a bad language can do those just as easily as a good language - tooling and libraries are much more a function of big-corp backing than they are of good language design. It's remarkable how much Scala has managed to achieve in those areas without having a big name behind it, but there's no competing with the amount of programmer-hours the likes of e.g. Google can pour in. The only way Scala can hope to win is on actual language design merit (and maybe winning popularity on that is impossible, but given how well Scala has managed to do so far, I remain hopeful).
> In addition, an argument for "my language is better than X" can usually be easily countered with an argument for the opposite, creating a net result that doesn't help your goals. Debates are sometimes interesting and informative, but they're not effective marketing.
A genuinely better language should have at least a slightly better chance of winning a debate over which language is better. If we don't believe that then we have no hope of ever learning truths, and may as well pick languages to use at random (or I guess go with whatever Google picked).
I think that the lack of evidence in favor of a strong effect is normally counted in favor of the null hypothesis. It may not feel fair, but the burden of proof is solely on those who claim a strong effect exists, not on those who don't.
> but a bad language can do those just as easily as a good language
Again, the burden of proof is solely on those who claim there is such a thing as a good language with a strong effect. If you can't display an effect, or claim that others can also induce it by other means is a win for the null hypothesis. If you claim there is such a thing as a "bad" language, and there is no evidence this is true, doing so will harm your cause.
> A genuinely better language should have at least a slightly better chance of winning a debate over which language is better.
I just don't think that a debate is the right way to establish an empirical claim. It's ok to engage in a debate - even a heated one - but the lack of evidence should at least encourage humility. My point is just that an imagined "win" at a debate only harms your mission. If it convinces anyone, which is highly doubtful, it's probably not the people you want, anyway. An approach that says, "I think this is cool. I like it and it helps me and may help you, too" is so much more effective at marketing than "my way is best" especially if the evidence is not on your side.
Aren't JVM languages interoperable - because they use the same classfile/byte code format?
What does it matter if the jar is written in java or scala, as long as it is possible to use the external interface from any of these languages?
JVM languages can usually use other JVM libraries, but they often aren't idiomatic to that language. Scala can use JVM libraries, but it often feels wrong or cumbersome, e.g. Java is full of mutable builder classes, which I've never once encountered in "native" Scala.
The other way around, in Scala you have to be careful if you want your library to be usable from other JVM languages. There's certain Scala features you just cannot use.
This is a problem that Kotlin actively markets itself with, they promise 100% Java compatibility all the time, going as far as encouriging to mix Kotlin and Java code in a single project.
In my experience mixing scala and java in a single codebase is inelegant, with plenty of conversion code at the boundaries.
Up to a point - libraries from another paradigm tend not to be idiomatic, but I could live with that. I'm more worried about tooling availability - I talked about IDE support in particular, and it's also things like profilers and monitoring/instrumentation.
It looks like its just tied to the jetbrains ide
Partially it's simple tribalism. But part of it is also the rational awareness of the opportunity cost of investing in a language other than the author's preferred one. The more people using language X that I don't like, the fewer people using my preferred language Y. That means fewer libraries I can use, docs I can read, bugs that get fixed, etc.
Language ecosystems aren't entirely zero-sum, but they aren't totally orthogonal either.
In any case, if people want other people to invest in "their language", they should focus on making that language and its ecosystem compelling to use, not bash other languages...
That said, in terms of language features and type system, Kotlin is arguably closer to Go than it is to Scala. Yes, Kotlin competes with Scala on the JVM, but it's a very different language. Scala is much closer to a language such as OCaml than it is to Kotlin.
However others who approach from position of authority like compiler hackers, language authors themselves, or very senior developers etc. Ideally criticism from them should be more valid but more often than not I have seen they keep making bad faith arguments and justify their hate by precise technical arguments so they can't be challenged by non-technical arguments. It makes vary of their arguments even on topics other than favorite programing language.
Thats somewhat by happenstance though isn't it; you take go, build a good enough scientific computing library, and tease out good enough performance, and get enough coworkers on it, and you'll probably have Go advertising itself as a scientific computing language (when talking to the relevant people).
And then you'll probably have go language devs implementing features better targetting the scientific computing community
And as the new people feed in, and implementing their own needs and libraries, suddenly go becomes good at ML...
And eventually we reach an 80s C-like status, advertised for nearly everything, because of its heavyweight ecosystem.
In terms of ecosystem, its really not constant for what they're good at. What might be constant is the flavour of programming the language prefers; you're probably not getting rid of goroutines as a significant language feature no matter how big Go gets.
Here's my view on languages: They have strengths and weaknesses. They cater to different needs, as you say. If you have the need for what Rust, say, does, and Rust solves some real problems for you and makes your job a lot easier, then it's kind of natural that you think Rust is wonderful. In fact, what you found is that Rust is wonderful for that problem, not that it's wonderful in general. But it's real easy to think that your situation is more universal than it is, and therefore that Rust (in this example) is this wonderful language that makes all of programming so much better.
Once you've fallen into that flawed perspective, then it becomes easy to criticize other languages. Why would you ever want to use Scala? It doesn't have Rust's advantages. But people miss that, if you have a Scala problem rather than a Rust problem, and you pick Rust anyway, it's not going to go well...
Whereas with Microsoft's .NET CLR there's a standard calling convention supported by all the languages. So it's easy to mix and match C#, F#, VB.NET, etc within a single application or reuse libraries. So developers have more freedom to pick the best language for each problem domain.
(It looks like Hacker News filters out the Unicode sharp symbol. Why?)
;)
* One can now use implicit function types to basically build your own table language syntax that is type-safe. [1]
* Multiversal Equality: you get to decide whether it makes any sense to compare an Apple and an Orange using "==" or "!=", as opposed to Java's forced requirement of allowing you to compare anything with anything. [2]
* Null safety checks! Quoting from [3], "Adding a null value to every type has been called a "Billion Dollar Mistake" by its inventor, Tony Hoare. With the introduction of union types, we can now do better. A type like String will not carry the null value. To express that a value can be null, one will use the union type String | Null instead."
* First-class enums, finally. [4]
* Erased parameters: you can declare variables specifically for type-safety that _don't exist_ during run-time, improving efficiency. [5]
[1] http://dotty.epfl.ch/docs/reference/implicit-function-types....
[2] http://dotty.epfl.ch/docs/reference/multiversal-equality.htm...
[3] http://dotty.epfl.ch/docs/reference/overview.html
Unfortunately it's being introduced in a fail-unsafe way that as far as I can see makes it virtually useless. If I see "x == y" in code, I have no way to be confident that this isn't an old-fashioned universal comparison without going into the details of x and y, so I'm no better off than I was without this feature.
EDIT just realized you are m50d on reddit and probably saw the exact same comment I got that from.
I'm impressed with what they're doing (and love the language), and it's hard work, and their speed is faster than, say, Java's dead-pace evolution in the 2000s.
But, poking around at TypeScript, I've been blown away with the MS/TS speed of development. ~2-3 month release cycles, with non-trivial changes to the language (mapped types, conditional types, etc.), that are themselves unique/novel type system features, not like Java finally getting around to copying the obvious/best-practice approach to case/data classes.
Granted, I'm sure the MS/TypeScript budget is huge comparatively...
Seems like that's the "best" (realistic) way for dev tooling to evolve lately: come from, or attach yourself to, a ~top-5 tech company that makes their money somewhere else and can bankroll developer tools/languages (e.g. MS with TS, FB with React/Flow, Google with a myriad of things e.g. Dart & AdWords, Go).
Bringing it back to Scala, seems like they were close to this (flirted with adoption/sponsorship from a few startups like FourSquare, etc.) but, thinking about it now, "software development in the large" is a huge concern for those companies (e.g. anyone with enough extra money to actually pay for language/tooling improvements), and that's never really been Scala's strong suit (compile times, painless compiler upgrades).
When you want the whole system to actually deeply rely on the static type system, it needs to be much more robust, and that takes a lot of effort.
I'm not trying to minimize what the TypeScript folks are doing — it's a really nice, well-designed language. But the complexity required to evolve an optionally-typed language is closer to that of a dynamically-typed one than a statically-typed one.
But I hadn't appreciated the difference in difficulty between an unsound type system like TS and Scala's type system...
I guess I've taken the formalism that underlies Scala's type system (e.g. the academic research behind dotty) as a given/for granted, but you're likely right, that TS can move faster because it doesn't need/have that level of rigor.
Thanks!
Did you mean "behind"?
The trend is pretty clear: dynamically typed languages are disappearing and all the popular modern languages are statically typed with type inference.
Most dynamically typed languages are busy adding static safety through gradual or opt-in typing, but you don't see the other way around.
Javascript will probably be the last major dynamically typed language we ever had to use.
It does seems that all the dynamic languages are learning what the systems people knew in the 70s: for the Enterprise, correctness counts.
A happy v1 is only 10% of the story... Getting to v225.5, warts and all, is just something else.
Yeah, it is. At the very least, I know that function parameters are bivariant, which isn't sound:
function foo(callback: (obj: Object) => void) {
callback("not a bool");
}
function main() {
foo(function (bool: Boolean) {
if (bool) window.console.log("!");
});
}
Since TS 2.6, if you enable all the optional "strict" mode checks than TypeScript does complain about that code.
I see no disadvantage in those checks, e.g. "strictFunctionTypes", being optional, since you can always make them mandatory for your own project(s).
To test it:
- Go to the TS Playground at https://www.typescriptlang.org/play/
- Enter the above code
- Select "Options" and enable at least "strictFunctionTypes"
I guess it's always envisioned differently like "can we tackle this subsystem in isolation".
My experience says, a rewrite just has different bugs. Combined with delay and cost, its often not worthwhile.
TypeScript broke my code for the past year on every new release.
---
Scala evolves a lot on each new version. There's a big difference between 2.10 and 2.12, the current version.
Dotty is a big change. That it ships in 2020 that's OK. Shipping it any sooner is risky.
A Typescript release usually has 1-3 exciting new features, and a number of smaller bug fixes and improvements.
The time it takes to develop a feature is very rarely the bottleneck.
So it's not like it won't be used until 2020.
I'd like to share a different perspective based on my own work with OOP-centric and FP-centric languages.
If you take OOP to its logical conclusion, OOP is a slippery slope that eventually leads to Gang-Of-Four centric designs.
If you take FP to its logical conclusion, FP is a slippery slope that eventually leads to Monad-Transformer centric designs.
Both are equally valid ways to design complex applications, so it all comes down to individual taste.
Personally I have a taste for mathematical abstractions, so Scala is better suited for my way of thinking.
It's obvious that Scala was specifically designed to be a solid foundation on which one can implement the concepts of a little known branch of mathematics called the category theory, and almost every design choice in the language flows from there.
If we use this as the basis for comparison to other languages, it's very easy to understand that Scala has carved out a very powerful niche for itself and is here to stay.
Nonsense. Scala was designed to make programming easier and safer than Java - XML literals and pattern matching certainly have nothing to do with category theory. Scala's for/yield has surprising behaviour when e.g. mixing lists and sets, precisely because it doesn't have a categorical grounding and was implemented in terms of what working programmers wanted to do with it rather than enforcing that you've formed a valid semigroupoid in the category of endofunctors.
Over time it's emerged that some categorical constructs provide useful ways of thinking about code and solving programming problems, but you're putting the cart before the horse if you think the language was designed for category theory.
Not sure what you're talking about, but for/yield is syntactic sugar for map/flatMap, just like the "do notation" is in Haskell. Of course it has theoretical grounding, because it wouldn't work without flatMap being the monadic bind.
> mixing lists and sets
Again, not entirely sure what you're talking about, but if true, it's entirely unrelated to for/yield.
Scala has other containers that have a .flatMap operation that are not real monads, such as Future and Try.
That can lead to surprising behavior on for/yeld loops.
Disclaimer: I do work in Scala and enjoy the language.
(for {
x <- List(1, 1, 1)
y <- Set(x)
z <- List(y, y)
} yield z).length
I don't understand what you mean by this, and I write production haskell for a living. We don't have teetering towers of transformers, and the best advice I've seen is often "put away the shiny tools and just use functions", https://lukepalmer.wordpress.com/2010/01/24/haskell-antipatt... . Similarly in http://www.parsonsmatt.org/2018/03/22/three_layer_haskell_ca... , which is like one real transformer layer and a way of claiming only the capabilities you need in your impure code. His "invert your mocks" article talks about this, too.
Ed Kmett's comments on Scala make me worry that the "solid foundation" isn't as solid as it could be: https://www.reddit.com/r/haskell/comments/1pjjy5/odersky_the... . How much of this is true in more recent Scala versions?
Some are fixed (Either, inference for many of the type lambda cases), some are being fixed in the next version (container overloads, CanBuildFrom), some are not really issues at all (free theorems are fine, an extra map on a monad is not a problem and sometimes more efficient, specific compiler bugs have been fixed but never said anything about the language in general, Haskell as used in the wild (with orphan instances) doesn't guarantee typeclass coherence either), some are real but exaggerated issues (subtyping and implicits, type inference for tricky recursion), a few are genuine issues that remain and probably always will (having to trampoline your monads, no kind system).
No it is really not. Haskell only really has one category of any import, typically denoted 'Hask'. There is no builtin way to embed or express arbitrary categories in Haskell. It's not even clear what this would mean, given that Haskell only has one conception of arrow (the function type '->'). In fact, Haskell is so far removed from true category theory that we have to use a compiler plugin (CCC by Conal Elliot) to get anything approaching compilation to arbitrary categories.
Haskell is based on the polymorphically typed lambda calculus. Some decades after its design, some people realized they could use some abstract algebra concepts to help structure programs. You can use these concepts in any language with a sufficiently expressive type system. It is not specific to Haskell
The fact that many organizations working on massive datasets have embraced Scala proves there are not many alternatives around, sadly.
can you elaborate more on that?
Haskell is category theory
as a language.
One of Odersky's motives in creating Scala was bringing the power of Haskell into the JVM world, although this is not the only motive (tight integration of OO and FP being another).
Haskell's key innovation over its predecessors (Miranda and ML) was the addition of higher-kinded types, which in turn made ad-hoc polymorphism digestable in a typed world (in the form of type-classes). The combination of HKTs and type-classes enables the monadic abstractions that contemporary Haskell programming is widely known for today.
Note that Scala has HKTs and type classes (via implicits), so Haskell programs can typically transliterated into idiomatic Scala without major complications.
[1] P. Hudak, J. Hughes, S. Peyton Jones, P. Wadler, A History of Haskell: Being Lazy With Class. http://haskell.cs.yale.edu/wp-content/uploads/2011/02/histor...
[2] Interview with Simon Peyton-Jones. http://www.cs.cmu.edu/~popl-interviews/peytonjones.html
[3] E. Moggi, Notions of Computation and Monads. https://core.ac.uk/download/pdf/21173011.pdf
I'm not sure why you would think that. Odersky has long claimed ML as his primary functional programming influence, not Haskell. It does have HKT, so there's that...but that's about it. Typeclasses aren't really a part of the language. They were made possible by implicits. They have since had some language level support (context bounds), but that is more of a nice to have than a primary influence.
Just because Scala can "do" Haskell doesn't mean odersky was inspired by it any more than ML or Java.
Implicits are a generalisation of default arguments (I don't know where they were pioneered, maybe C++). Implicits were first tried in Haskell [1]. Scala refined them in several steps, the last being [2] which represents implicits on the type leve. Mimicking type classes via implicits is a well-established Scala idiom [3].
[1] J. R. Lewis, J. Launchbury, E. Meijer, M. B. Shields, Implicit Parameters: Dynamic Scoping with Static Types. https://galois.com/wp-content/uploads/2014/08/pub_JL_Implici...
[2] M. Odersky, A. Biboudis, F. Liu, O. Blanvillain, Simplicitly: Foundations and Applications of Implicit Function Types. https://infoscience.epfl.ch/record/229878/files/simplicitly_...
[3] B. C. d. S. Oliveira, A. Moors, M. Odersky, Type Classes as Objects and Implicits. http://ropas.snu.ac.kr/~bruno/papers/TypeClasses.pdf
My general assumption is that if Scala was intended to be category theory, implemented, something along the lines of scalaz or cats would have been built into the language -- the philosophy of the language does not include a JS-esque philosophy of small stdlib, big library ecosystem.
He mentions module systems fairly early on.
If scala team were to disavow sbt, that'd be the single best thing they could possibly do for the ecosystem.
I used to write a lot of scala, and working with sbt was enough to eventually get under my skin.
I really like some of the OOP aspects of scala, the left-to-right style of thinking matches how my brain works. Scala having a lot to offer can distract new learners, especially with just how overwhelming all the different features can seem for someone not coming from both sides. I had ocaml and ruby under my belt when I found scala, so it was an easier curve, but lots of guys struggle with it. I don't mind showing them, and of course I learn a lot from guys with different experiences and different backgrounds, but that's sometimes a complain I hear, also.
It's a language that doesn't compare very well. What is scala like? We don't know, there's only one language like it, it's a departure from many different paradigms.
I'm back to writing ocaml, all said and done. I enjoyed my time with scala, and interacting with my guys on the other side of the office still doing scala is always enjoyable. Smart guys! I'm not academic enough to grasp some of the new DOT stuff, but they are excited about it.
Anyways, I gave up hope a while back on scala getting rid of sbt. There are other build tools that exist, cbt, mill, maybe more, but without the blessing from lightbend or whatever they call themselves this week, it's not feasible, or responsible to deliver a solution to a client with an unofficial build setup. Of course, after five years of doing sbt I decided it's not ok to deliver sbt either.
Someday! Maybe?!
Seconded. Between http://www.lihaoyi.com/mill/ and https://bazel.build/, there are plenty of alternatives.
The only remotely viable alternative to SBT I've found is Pants. (I guess Maven and Gradle might qualify too, but it's been so long since I've used them with Scala that I can't be sure.)
There is plenty of scala in fintech, I can tell you that. But there's plenty of ocaml, f#, matlab, excel macros, r, kdb, cobol, fortran, c++, a vast array of different technologies.
Most places that choose scala are choosing it over erlang or Java or c#, they USUALLY only choose scala over ocaml if they want to have some old ocaml code talk with new infrastructure that they want built in scala, ime. Because of how performance works and that multi core isn't working yet, ocaml is limited in application if they're willing to find or already have scala programmers.
ScalaJS for example mandates the use of SBT.
And by a very, very long margin. It has arcane syntax, is slow, is incredibly complex if you want to extend functionality, it has multiple configuration files, the ridiculous Ivy global lock and inability to fetch Ivy artefacts in parallel etc. And that's just the start.
This seems a bit hyperbolic and a little vague. So, I can't really address it directly. Interestingly, I find sbt much easier to extend than most build tools because sbt tasks generally don't rely on side effects to communicate. Obviously YMMV.
> it has multiple configuration files
Doesn't every build tool have this? Or do you mean multiple config file formats? If that is the case, newer version of sbt have moved a single file format.
> the ridiculous Ivy global lock and inability to fetch Ivy artefacts in parallel etc
I can understand this concern. Ivy is kind of annoying. Fortunately, a new artifact resolver called Coursier[1] has been written and it solves this problem. You can use it now and it is currently in the process of being integrated as part of the default SBT install[2].
[1] https://github.com/coursier/coursier [2] https://github.com/sbt/sbt/issues/2997
The thing that makes sbt difficult in practice for me is twofold. I don't want to deliver hackedy shit to a client that I then have to defend if their quality control comes back complaining about it. Being able to use regular scala in sbt definitions is convenient but at the cost of making things expressable that should wisely be done another way.
Which brings me to the second point which is getting a sbt definition from a client and then they don't want to change it, or they don't have quality control so it's just awful, or its so bad that someone has to rewrite it for the solution. The latter being the easiest to deal with, but that also costs time.
For those looking, what things do you feel have most improved the sbt experience?
I'm not sure I totally understand this. What do you mean by "macros going wrong"? It doesn't seem like there are that many implicits used. The mostly to add methods to strings to make expressing things more concise. Do you think implicits shouldn't be used at all?
> The thing that makes sbt difficult in practice for me is twofold. I don't want to deliver hackedy shit to a client that I then have to defend if their quality control comes back complaining about it. Being able to use regular scala in sbt definitions is convenient but at the cost of making things expressable that should wisely be done another way.
> Which brings me to the second point which is getting a sbt definition from a client and then they don't want to change it, or they don't have quality control so it's just awful, or its so bad that someone has to rewrite it for the solution. The latter being the easiest to deal with, but that also costs time.
This seems contradictory. It seems like in the first paragraph is at odds with the second. In the first paragraph, it sounds like you are saying that you want to be able to write messy builds and not have anyone complain. In the second paragraph, it sounds like you are saying you don't want deal with other people's messy builds. Am I understanding correctly? If so, do you want messy builds or not? How is this related to the build tool? What build tool prevents people from writing messy builds? Are you thinking of something like maven that is very rigid?
> For those looking, what things do you feel have most improved the sbt experience?
Some things that may (since I don't know when you last looked) have improved:
* SBT 0.13 greatly improved the .sbt build file syntax (no more line breaks between settings, multi project in one file)
* SBT 0.13 also cleaned up the symbol soup. Now to define tasks/settings you only need to know 3 symbols that are fairly self explanatory ':=', '+=', and '++='. If you want to depend on another task/setting output you just have do `taskName.value` in your task/setting definition.
* A new artifact resolver has been written[1] that fixes all of the issues with ivy. It can be used today and is currently in the process of being integrated in the default SBT install.
* SBT 1 introduced a build server concept with language server protocol support that is starting to improve IDE/editor integration
* This isn't strictly SBT but IntelliJ now natively understands .sbt files so you can autocompletion and code navigation.
This is double if the person you're correcting doesn't think what they're doing is "wrong".
I don't give that privilege to others and I don't expect anybody else to.
I say guys because my developers and analysts and quality managers are all men. The only women I work with are two secretaries, and they're not even in my department.
What's too bad, friend, is that you have deluded yourself into believing you have the right of way over another person, for any reason whatsoever.
Bless your heart!
Please take your SJW-ism somewhere else. This is HN.
1.1 guys People of either sex.
I feel like this isn't getting much attention but it is a huge deal. A big part of the reason that Python 3 went the way it did was users didn't want to upgrade until their libraries did and libraries didn't want to upgrade until their users did. With Scala 3, users can upgrade pretty rapidly, freeing libraries to upgrade without fear of leaving their users behind.
Who's still in Scala boat? Are Google or Fb or other big cos known to give back to open-source growing any Scala codebases now?
In the last ~18 months, the amount of CVs we're getting has been constantly increasing. Part of that is probably that our startups hiring matures, but it's also the kind of CV that is changing.
I'd say that 3 years ago, there was an 80% chance that the applicant was highly self-motivated to learn Scala in their freetime, and tried/did introduce it at his/her current workplace. Today, there is an 80% chance that the applicant either "had to" learn it in their current workplace, or learned Scala when switching jobs. (Don't get me wrong, they're still motivated, and they took the chance when it was there!)
So there is a switch from the Early Adopters to the Early Majority (where the Early Majority now has worked 1 or 2 years with Scala at their current job, and is confident enough to look for a new one).
One driving force was definitely Spark, but there are a lot of Enterprise apps, unrelated to ML (usually with higher traffic requirements). The sort that would have most likely been done with Java or C# 5 years ago. It seems a lot of Enterprises introduce Scala when they try to break up their (Java) monolith into microservices.
So it seems that Scala has been carving out it's place in backend/microservices with scalability requirements, and is eating part of Javas cake there.
I've been using Java and Scala exclusively for the past 10 years at two large NY banks and two fintech start-ups, and I literally do not know anyone who has ever compiled a Kotlin program.
Simply extrapolating from one's own experience is fraught with peril.
So most companies actually are more keen to bet on Scala than F#.
OCaml/Reason world lacks the wealth of Java libraries, which are an import away on Scala.
Regarding Kotlin, so far its killer use case is targeting Android, where one is stuck with an ageing Java subset. Outside of Android, it remains to be seen how it will keep up with the Java improvements coming up every 6 months now.
Let's face it, Kotlin hurts Scala adoption.
I know Java since the early days and only used Spring during one month as validation of an architecture proposal.
Outlook largely negative, falling in rankings across the board.
Scala is a fantastic language, but it’s not one your average Java developer can pick up in a day or two.
In fact, Microsoft has to advocate library writers to actually care about .NET Core and .NET Standard.
https://channel9.msdn.com/Shows/Visual-Studio-Toolbox/NET-St...
I have not listed it, because in spite of spec, it still is a dynamic language.
And what subset would that be? The only place you can't use F# IIRC is UWP application which is likely not the deciding factor in choosing F# or Scala.
It is still playing catch up with VS 2017 tooling for VB.NET and C#.
- Windows form works but not perfect. The editor works if you install the template for it, but not the auto generation of code, like double click on button -> handler generated. You write code programmatically. but is used a lot. I use it in the repl (fsi), to generate chart, custom data visualization.
- WPF the same, works, no editor (but codegen is less needed)
- Xamarin support F# ( https://docs.microsoft.com/en-us/xamarin/cross-platform/plat... ) on Forms, etc
Some good from community+MS, because community tried to adapt techonologies and make it more friendly to use in F#
- Xamarin XAML in Elmish style, really more idiomatic ( https://github.com/fsprojects/Elmish.XamarinForms ) from F# creator itself (Don Syme, who now work on xamarin division too)
- WPF and Xamarin xaml can be used with a type provider too for statically type view at compile type ( http://fsprojects.github.io/FsXaml/ )
The only one not yet supported is UWP, because of of .NET Native.
All that without speaking of the gui stack outside .net framework, like electron+fable or just fable+elmish/react/react.native ( https://github.com/SAFE-Stack/SAFE-Nightwatch )
Never understood the mentality for designing UIs by coding instead of visually.
I care for what comes in the box, and is directly supported by Visual Studio and Blend.
If someone needs to lose their .NET GUI tooling productivity to embrace F#, then better wait while C# keeps getting F# most relevant features.
Even C++ has better UI tooling support on Visual Studio than F#.
I don't believe most organizations considering adopting F# or Scala are considering them for GUI development so I'm not sure why you'd rule out F# because of that
And maybe not across the board, but my clients are very comfortable with the (very few) F# solutions we've done. Ocaml clients generally use it for stability, or correctness, they certainly aren't competing in the same sphere that kotlin, scala, Java really operate in. Prominent in the financial spade especially, but the trend is the same everywhere.
Reason may eventually help ocaml move closer into those spaces, as people are coming to ML with more openness and are more eager to learn and get involved, so that'll be cool to see in the future.
Anyways, don't count F# out either. I'm no expert, only ever used it with other project leads and with smaller clients, but MS is doing impressive work lately, like NET Core and stuff like that. They at least can show that making net more visible and flexible is part of what they're after.
Java won't be killed. Ever. But people may stop writing it eventually.
This statement is wrong. F# can use .NET Core, and it works same as JVM.
In fact with spark 2.3 python UDF, the performance gap has also reduced. https://mindfulmachines.io/blog/2018/4/3/spark-rdds-and-data...
PySpark is often used for data science experimentation, but is not as frequently found in production pipelines due to the serialization/deserialization overhead between Python and the JVM. In recent years this problem is less pronounced due to the introduction of Spark dataframes which obviates the performance differences between PySpark and Scala Spark, but for UDFs, Scala Spark is still faster.
A newer development that may change all this is the introduction (in Spark 2.3) of Apache Arrow, a in-memory column store engine which lets Python UDFs work with the in-memory object without serializing/deserializing. This is very exciting as this lets Python get closer to the performance of JVM languages.
I've played around with it on Spark 2.3 -- the Arrow interface works but still not quite production-ready but I expect it will only get better.
Many folks are making strategic bets on Arrow technology due to the AI/GPU craze (and an in-memory standard enables multiple parties to build GPU-based analytics [1]), so there is tremendous momentum there.
At some point I expect the relative importance of Scala on Spark will decrease with respect to Python. (even though Spark APIs are Scala native)
[1] https://www.nextplatform.com/2017/05/09/goai-keeping-databas...
Scala has a for/yield construct similar to Haskell's "do notation", which is the perfect way to work with async code - it strikes the right balance of avoiding an unreadable callback pyramid of doom, but still your async yield points visible (the difference between <- and = is about as concise as it gets, but still very visible). And having HKT and allowing you to express concepts like monads means that a whole library ecosystem can build up of standard operatons operations (e.g. traverse, cataM) that work on async futures and also on other contextual/effecty types (e.g. audit logging, database transactions). That's the big advantage it has over Rust and most other competitors, and it particularly shows in things like Kafka/Akka that need to work with async a lot, but also in large programming projects generally. (What it shows up as in practice is that you can do everything in "plain old code" - all the things that need reflection/interceptors/macros/metaclasses/agents/... in other languages are just libraries in Scala - and that makes development so much easier and bugs so much rarer)
Haskell has all that - the trouble with Haskell is that "there's no way to get to there from here". I was able to go from writing Java on Friday to writing Scala on Monday - not great Scala, but working Scala, and I was just as productive as I was in Java. I couldn't've done that with Haskell.
Can't speak too much about the others, but I recently looked into using Akka with Java. Akka itself had a good deal of documentation for Java users. However, other related products (look tools for monitoring Akka, etc) were clearly treating Java as a second class citizen. When there was documentation, it would be outdated, and unmaintained. I ran into this over and over again, which weighted heavily on my decision to not use Akka. I'm not quite sure if there is anything about the language that would have made it particularly challenging (doubt it), but it definitely seems like the community built around it is more Scala centric. And that in itself makes it challenging to use with anything else.
I'm much more of an ML-style programmer, but the thing keeping me away from OCaml is the lack of a multicore runtime. Not only is Scala's concurrency mature and performant, implicits (like having an ExecutionContext) make it extremely easy to use. Even if OCaml released it's multicore runtime today, it would take years to get to the maturity of the Scala ecosystem.
Plus, occasionally I run into areas where OOP really blows away functional programming. It's nice to not have to learn a new language for those occasional times.
LinkedIn, Twitter, The Guardian, Morgan Stanley, Barclays, Zalando. Generally speaking, Scala is used a lot by companies involved with Big Data (because they use Spark).
It's not even clear to me that most companies doing data science in Scala have that scale -- they're just using tools and libraries companies at that scale have open sourced. You could call it cargo culting, but I think it's more nuanced than that. I think engineers can be separated into two camps roughly: those who are passionate about the language(s) they use, and those who are simply trying to get the result they need and don't care what language they use to get it. A lot of data science engineers naturally fall into the second bucket, so using Scala because a library they want to use is written in it comes naturally, even if they could get the job done in Python (possibly with a bit more wheel reinvention).
1- https://www.cisco.com/c/en/us/products/hyperconverged-infras...
2- https://jobs.cisco.com/jobs/SearchJobs/hyperflex?3_12_3=187
Scala at Morgan Stanley: https://www.reddit.com/r/scala/comments/480ral/scala_at_morg... https://www.linkedin.com/jobs/morgan-stanley-scala-jobs/?cou...
Scala at Barclays: https://underscore.io/jobs/2017-11-02-barclays/
Scala at Zalando: https://jobs.zalando.com/tech/blog/why-we-do-scala
Scala open source projects from Twitter: https://github.com/twitter?utf8=%E2%9C%93&q=&type=&language=...
As for Scala usage. I'm a Scala consultant, so I'm biased, but I'm seeing a lot of adoption. Most finance companies and most media companies are using Scala.
Martin Odersky seems like a really good language designer. I took a look at the Scala 3 languages features, and a more radical departure from historic Scala probably would have helped Scala.
(Did you have specific breaking changes in mind? The most important changes I'd want to make to the language - having a syntax for guaranteed-safe equality comparison and guaranteed-safe pattern matching - could be done in a backwards compatible way, and the only other thing I can think that I'd like would be Idris-style totality checking which could also be backwards compatible.)
Wait what? In the last post you were complaining it was too close to existing scala, now you're saying it's too big a step? Again, what is the change you're actually advocating?
I liked the idea of Scala and took Odersky's course. But I built a few things in it and it was never not frustrating. What Bruce Eckels said resonates with me: "I’ve come to view Scala as a landscape of cliffs – you can start feeling pretty comfortable with the language and think that you have a reasonable grasp of it, then suddenly fall off a cliff that makes you realize that no, you still don’t get it."
Then I watched this Paul Phillips talk, which convinced me that the problems were deep, not superficial: https://www.youtube.com/watch?v=4jh94gowim0
My hope with Dotty, etc, was that they had learned a little humility and were pruning back enough to make it a decent developer experience. But I could well believe that it was impossible to do enough and still end up with something that could be fairly called Scala.
Maybe. But high-risk. See Perl 6.
https://www.linkedin.com/jobs/amazon-scala-jobs/?country=gb
https://www.indeed.co.uk/cmp/Amazon.com/jobs?q=scala&l=Londo...
https://www.amazon.jobs/en/search?base_query=scala&loc_query...
Its popularity also far surpasses F# and OCaml in jobs available, books, libraries or other metrics that count.
Not a "big" company by global standards, but not small either, with around 120 engineers.
What if you want functional programming without having to go to MS stack and stay on JVM. You only have have two real options clojure and scala.
All that said, how have others dealt with this issue? Did you just not work directly with Java libraries? Did you use the OO side as a Better Java?
I prefer an intermediate solution when I can get away with it: Localize the mutable Java objects as much as possible, and just make sure that I don't leave a team/teammate that has little Scala experience all alone for a while. This often leads to styles that might be frowned upon by both camps of programming, but in my experience, dropping to imperative code when FP solutions are harder to optimize, while making sure that mutability is well contained and doesn't cross interfaces is Scala's happy place. Depending on the work to be done, each codebase can be pretty FP heavy, or be mostly imperative with an FP facade.
The real trick with Scala is really library design though: It's very easy to make a new library that has dozens of new concepts and is hard to learn and use, all while exposing things like Shapeless HLists to the outside world, while libraries that are easier to consume and don't crush compilation times through type magic are often tougher on the author, leading to more code generation and macros. Most library authors know so much Scala that they don't realize that just using their library well incurs in quite the mental cost to new developers.
I've done all of these approaches in various projects - using Scala just as a better Java, with Java-style code, using Java libraries, then writing code that was in a more immutable style that used wrappers around Java libraries, then writing very pure (i.e. no unmanaged effects) code that either used scala libraries, or used Java libraries only in an interpreter for a monad. All of them work. Mixing mutable objects and pure functional style is never going to go well, and that's as true in Python-only projects (say) as it is in Scala/Java hybrid projects; the language interop doesn't leave you any worse off than you were without it, if that makes sense.
It's useful because the JVM ecosystem is much larger than the Scala ecosystem. There's almost always a library for your purpose. If it isn't as idiomatic Scala as you'd want you can often easy create a small wrapper.
> Suddenly I have to worry about the Jr Dev doing silly things like introducing mutable Java objects into functional code.
code reviews and education
Catching bad practices and anti-patterns in code review sounds fine on paper, but I'm sure we've all been in the situation where there simply isn't enough time in the day to do intensive code reviews while still maintaining the pace needed for delivery.
Implicit conversions (i.e. where a type is silently converted to another type) is dangerous and can lead to very surprising problems.
There's ongoing performance work to make both Scala 2 and Scala 3 faster than they are now. A lot of what people perceive as slow compile times are actually other factors (such as inefficient macros or slow build tools).
I'm not sure what you mean here... but Scala 2 and Dotty have the same incremental compiler, and their independent bridges feature overall the same quality (which is high and heavily battle tested, for those interested).
The quality of the used names extraction affects accuracy, but not compile speed. I must say, Dotty and Scala are extracting about the same amount of dependencies, with some corner cases being covered better in Dotty (for example, changes in constant values).
I don't see a way Dotty can become a faster incremental compiler that doesn't involve adding the incrementality in the compiler.
Edit: typo
Granted, it's probably more fun to start from scratch :-), just seems more realistic for long-term adoption/maintainability to leverage existing tools.
The model of Mill, Bazel, Buck, Pants are fundamentally different in very important ways which allow e.g. using a shared build cache across all the developers in a team (or even all the developers in a whole company if one takes the monorepo approach for absolutely everything). This is not something that can be retrofitted into, say, SBT or Gradle.
https://docs.gradle.org/current/userguide/build_cache.html
It is a newer feature, so that might be why you're unaware of it.
Also, I get that the models of Bazel/Buck/Pants are fundamentally different, because they are inherently cross-language / "sit on top of other compilers" systems. And are best for mono repos.
But Mill doesn't fit in that camp AFAICT.
You may well be right about Mill, but it's probably too early to tell exactly where it'll land in the design space.
(Btw, I think it's entirely right to be wary of yet-another-build-system, especially when it doesn't seem to do anything that other systems don't already do.)
def asyncFunction(): Future[Blah] = for {
foo <- someAsynchronousOperation
bar <- anotherAsynchronousOperation
} yield bar.bazJust compare Rust's announcement on tooling (https://www.ncameron.org/blog/dev-tools-in-2018/) with the state in Scala, where pretty much every promised improvement has been "around the corner" for the last 5 years. :-/
Not sure much how much can or will change here, given the departures of experienced tooling contributors in 2017.
Kotlin doesn't even get closer to what Scala is, and Rust is for those folks with the mentality that memory management is something worth keeping track of.
Kotlin is nice if all you want is a simpler Scala.
Then there is Java slowly taking the more relevant features from them, while being the platform's language.
Scheduled to be added next, pattern matching.
I didn't realize so much of Apache ecosystem had moved to Scala which is huge.
What about Ocaml (1996, well before scala) and F# (2005, not too long after scala)?
When we do need to use it, it is actually pretty elegant and well-designed. It's just that since we've tasted the power of the more functional abstraction techniques, OOP loses its allure.
I mean I can understand all the reasons not to, however It's just a pita that there is no sane way to do it, besides reimplementing either two api's or come up with your own collections.
Personally, for a lot of my current use cases (APIs) I'm veering towards Go more and more as I find it a lot more productive.
> eliminate inconsistencies and surprising behavior
Oh dear god please.
Back in 2010 - 11, when I last worked with it, the compiler was also dog slow and IDE support was terrible. No doubt this has changed for the better, but it did ruin the language for me. I'll stick to Kotlin and Java, thanks.
Secondly, the implicit concept makes the code difficult to read. Then I would add that the differentiation between function & variable isn't always clear and sometimes you don't know what you call.
Yeah, that's what I was talking about - in Scala operators are just functions, and badly named operators are an issue with some libraries rather than an issue with the language. They're unfortunate, but all one can really do is avoid them - no language is impossible to write bad libraries in. Certainly I wouldn't ever want to go back to a language that uses magically-named methods for operator overloading, like Python and Kotlin do - I find that far more confusing, I can never remember whether a * b is calling a.__times__(b) or a.__star__(b) or something else. I could get behind a language that disallowed symbolic names entirely, but I don't think I've seen one of those since Java.
> Secondly, the implicit concept makes the code difficult to read.
IDE support has gotten a lot better now (green underline to highlight any implicit conversion, expand out implicit parameters) which makes them much easier to work with. If you only use implicits in cases where you otherwise wouldn't do anything in code at all, they enhance readability - but of course if you use them to replace things that you'd otherwise make explicit then they can reduce readability. I don't have a good answer, because I definitely want extension methods, typeclasses with derivation, and the magnet pattern, but I haven't yet seen a language design that makes those things possible while disallowing the bad uses of implicits.
Want to add a value to a (mutable) Seq? Use +=.
Want to add a value and return a new Vector? Use +.
Want to build a new list that is the concatenation of two Lists? Use ++.
sbt is probably where you encountered "weird" operators, but their use has declined over the past few years from what I've seen.
https://github.com/foursquare/fsqio/issues/37
They don't upgrade the open source version because their internal version depends on that open source version and they don't can't upgrade internally to 2.12.
They can't upgrade because of several dependencies of their own, the last one Finagle.
All of those although "cross compiles are no problem" as the default answer from the Scala development team on these issues.
There are Many people who work at many companies doing real development with Scala that goes all the way to production, handling real user traffic.
But I wonder how many companies would make the same decision again after their experience with Scala.
I assume that the "added feature" section of the Dotty documentation and this blog entry have been written by different persons, because it's utterly impossible to reconcile those documents:
More complicated features, more experiments, more NIH, more ad-hoc inventions, larger language footprint with more inconsistencies and surprising behaviors.
I wouldn't want to have any more language features that pushed as hard as implicits do, but one somewhat-experimental feature in a language is ok, just about. I don't think Scala is a thousand-year language, but it strikes the best balance given the current state of the art.
If you don't like them, why don't you add "Use of implicits forbidden" to your coding style guidelines, and check in code-reviews, and add a "grep implicit" check to your CI setup that refuses to commit Scala code with implicits?
Let me paraphrase M. Odersky's PLDI 2017 keynote:
The essence of Scala is implicits
Because Scala libraries that do useful stuff are infested with them. Akka loves the things.
> main complaint
They obscure complexity, pretending that complex stuff is simple. It makes simple stuff trivial but complex stuff harder.
So? Treat Akka as a black box. You can always explicitly pass implicit arguments.
Only someone who has never worked in a team before would suggest this.
But no, "Implicit Function Types" instead. Ok.
I have never worked extensively with Haskell or ML, but what I can say is that Scala's syntax is better than that of most other mainstream languages.
Three of Scala's syntax niceties are:
1) No semicolons
2) No braces required for one-line functions
3) No constructor required: instance arguments passed in class declaration
You can do a big for comprehension, or a if/else, or a big chain of function calls, etc. without having to add brackets.
In practice you often end up not really using braces that often when writing FP style Scala.
Scoping is hard.
You're probably referring to the worst of Scala.
Personally, I don't find expression-centric, brace-less Scala to be "rather ugly".
Which idioms exactly?