The Last Hope for Scala's Infinity War [video]
youtube.com
youtube.com
First, he poisons the well by talking about "integrity" and "intellectual honesty". This is a bad move at the beginning of a speech because it is far too easy to take it such that people with different conclusions are lacking in those qualities. Given his delivery, I suspect that he might actually believe that. It would be better if he just owned that this was his opinion and delivered it respectfully.
Second, the section on academic priorities was weak. He spent a large portion of the talk highlighting the mission statements of various languages, which is marketing language. But if Scala made no other changes other than its marketing language, it would not solve the problems he was highlighting. I thought too much time was spent on that part of the talk. There's another moment later in his rushed conclusion where he made stronger points that could have been fleshed out - how the academic influence leads to theoretical features that later get abandoned when the academic project is completed. (And I quite liked his revised marketing statement for Scala.)
I also didn't like that he brought up Cats vs scalaz as a flaw with the community, given that it became clear later in the talk that he had a clear bias on which was preferable. That seemed needlessly provocative.
I think the final conclusion of abandoning the OO path was interesting and fun to think about! I don't know if there's already an opinionated compiler plugin that discourages the more OO techniques while encouraging FP techniques, but I can see how that would be a lot of fun to play with, and educational in the same way that static analysis tools are. (I don't understand why "partial functions" was crossed out as something to abandon, though.)
Sadly while the language itself has excellent OO support and Java interop, the community support just isn't there (by which I mainly mean: too many people like DeGoes will shout you down rather than trying to help you).
Proposition for discussion (i.e., I don't necessarily believe this, but I don't necessarily not either): The biggest challenge facing languages and their communities that focus on correctness is that the very personality types that are willing to focus on them are also statistically much more likely to be, ah, potentially less suited to the sorts of compromises and engineering discussions that are necessary to make a personal language than other more pragmatic communities. The sort of relatively uncompromising spirit necessary to swim upstream against the current general consensus of very pragmatic programming nearly entirely unconcerned with academic-type correctness is, well... uncompromising.
"Community" is largely a toxic concept anyway.
Oh, we have to have a community. Community this, community that. Nice project, but where is your community? I can't write a line of code without a community!
Tired of "community".
The core language is very much OO. The functional aspects of Scala are either syntax sugar that desugars to OO stuff (e.g. automatic eta expansion, special syntax for defining FunctionN vs new classes, case syntax for partial functions, context bounds) or decisions made by the standard library (e.g. the prevalence of immutable data structures).
Understanding this has been the key for me to understanding where most of the warts of Scala come from. For example some of the weirdness of functions vs methods, weird performance issues, the existence of some of the encoding tricks that libraries like cats and scalaz use, and strange decisions made in the collection libraries all can be fundamentally traced back to this layering.
That being said you can do a lot of FP in Scala because of the combination of ecosystem and syntax sugar, but you have to always keep in mind that you're encoding FP concepts with OO tools (not as equals) otherwise Scala's limitations will come back to bite you.
My opinion -- and I mostly write Scala in my day job! -- is that Scala is in a truly difficult position. Its OOP (and Java) DNA hurts its FP aspects. If it goes full hardcore FP, it will probably lose relevance (because most programmers can't be bothered with it, and those who can will probably hope for Haskell on the JVM). If it stays like it is and tries to be a "better Java", it will keep losing ground to more practical and "easier" languages on the JVM, possibly including losing to Java itself!
PS: DeGoes didn't sound disrespectful to me, and bear in mind I don't know who he is. He seemed to be arguing out of practicality, not shouting people down. I don't if he is angrier in other forums, but his tone in this presentation sounded closer to a wake up call to me.
That seems more of a strawman than a fiction: I've never seen anyone, especially in the Scala community, suggest this.
I've seen lots of people suggest that OOP+FP fusion is what allows Scala to provide an easy gradual on-ramp to FP for people and projects transitioning from Java, which is Scala’s key value proposition.
It seems to me DeGoes is arguing that Scala's "key value proposition" is a losing battle, because other languages are starting to do it much better. It's likely that the "practical programmer" will go with Kotlin or even the next iterations of Java, and the rest... well, you can guess the reaction of the audience to his question ("how many of you would work with Haskell if you could?"), and I must take his assertion at face value that in Scala conferences, the more popular talks are about FP, not about OOP or even FP+OOP. Maybe he's wrong about this, of course.
Again, I'm saying all this as a Scala practitioner. I'm not leaving it for Kotlin either. Then again, I certainly would work with Haskell if I could!
edit: one more thing:
> "I've seen lots of people suggest that OOP+FP fusion is what allows Scala to provide an easy gradual on-ramp to FP for people and projects transitioning from Java"
In my experience, this ramping up to FP happens in one of two ways with Scala:
- Completely ad hoc, misunderstanding every principle and making a Frankenstein's monster out of it. No lessons learned, just a mess that "works".
- Understanding it and doing it with care, and mentioning Haskell a lot (as these people start to read articles and watch tech talks on FP).
This is entirely my personal experience and probably won't generalize to every workplace.
I feel like I'm at the tail end of that on-ramp, and while I'm still as a whole really appreciating scala, I'm starting to wonder what the end-game is.
Now that I understand and am comfortable with all the functional stuff, I'm left wondering why I still have the OOP stuff, and why some of the FP stuff isn't more natural. I guess you still need some OOP for java interop, but the pure-scala ecosystem is pretty significant already.
I guess what I'm saying is that the easy transition from Java is only a benefit for bringing people in, not for a language as a mature ecosystem. And I'm thinking maybe I should get into one of the full-FP languages.
In fact, if anything, what's keeping me from making the switch is not any OOP features, but actually just imperative! Especially for performance-sensitive sections, and some rare-but-necessary data structures, it seems you really do want a small amount of mutable state.
The end game back in the early days of Scala was probably to provide a gradual on-ramp to FP for people and projects transitioning from Java, but it probably changed to a commercial purpose once more people got involved in its development. It's long since become a self-preserving product, and probably even tries to hinder people moving from FP-only Scala on to Haskell and the like.
I only spent a few months on Scala way back so don't know for sure, but I did see Apache Groovy go through a similar business re-purposing. It began as a JVM-based scripting language complementing Java, useful for glue code and testing, but later became a competitor to Java and other statically-compiled JVM languages, and even tried unsuccessfully to target Android.
Take it slow, and don't expect to be using every Scala feature/library/pattern from day 1 - there is a lot of depth to functional idioms, the learning curve of something like Haskell has less to do with syntax than you might think. Write code that does things that are useful to you - even if you're only writing "Java without semicolons", you're getting value out of Scala - and adopt the fancier features one step at a time. I've been working in Scala for 8 years and there are still places I'd fear to tread (why do I care if my profunctor has strength?), but every year I've produced more concise, clearer, less buggy code than the year before, and even in year 1 I was doing as good as I had been in Java or Groovy.
Well, to my mind the FP is a means to practical ends. I'm not attached to Scala qua Scala, I'm attached to HKT and typeclasses because there are practical problems I can't solve without them. As long as Scala has those things and Java or Kotlin don't, Scala will have an advantage - and if Java does adopt those things I'd say mission accomplished. (I mean, Odersky's original goal was to improve Java, right?)
Here's a function that is only partial, because it can throw an exception instead of returning a value when b is zero:
def divide(a: Int, b: Int): Int = a / b
Here's a total function to accomplish the same basic goal:
def divide(a: Int, b: Int): Try[Int] = Try(a / b)
This function always returns a value of the type it's declared to return.
I swear, probably 80% of my problems in professional life are from overloaded terms.
;-)
Some partial functions would be disallowed, assuming you aren't including an actual totality check (which would exclude all partial functions and some total ones).
For me Scala is a very general and a remarkably unopinionated language. It efficiently encodes and integrates multiple programming paradigms in one language, letting you pick and choose what makes sense to you.
Scala is useful to a wider range of programmer preferences and skill sets than any Haskell-on-the-JVM could ever be. It's a feature, not a bug, both for individuals and enterprises. Pure FP is just some people's preference that requires a significant buy-in, not the one true way of doing things.
In general, this talk just seems like a case for one man's preferences rather than a fair summary of Scala's current market situation.
- Tooling needs improvements. He mentioned how the Scala plugin for IntelliJ IDEA will flag working code as incorrect. It will also fail to flag code that won't compile as written. He didn't mention sbt as a pain point, perhaps because it goes without saying.
- Scala 3 (Dotty) has the potential to split the community and libraries for years, much like the Python 2/3 transition. That said, I'm still kind of excited about Dotty because I only have 10s of thousands of lines of code that potentially need porting work, not 10s of millions, and that seems worth it in exchange for stable macros, faster compilation, etc.
- The Scala community is losing people who just want "better Java" to Kotlin, and losing people who want pure FP to Haskell. I came to Scala because of Spark and I keep using it because I have grown to really like it, despite the pain points. I'm not eager to jump ship toward Kotlin, plain old Java, or Haskell. But if the community keeps shrinking, and the ecosystem shrinks with it, I will eventually be forced to jump toward something else.
SBT keeps getting better, and you have a handful of viable alternatives these days.
2. All major Scala releases bring breaking changes, this is incomparable to Python.
3. By any real-life metric (jobs, conferences, meetups, libraries, ...) I don't see evidence that the community is shrinking, quite the opposite.
The Dotty transition looks like it will be significantly more breaking than the changes in the 2.x releases over the past 5 years. I'll be pleasantly surprised if the transition goes smoother than I expect, though.
I am excited by some recent alternatives to sbt, particularly Li Haoyi's Mill.
Also, I hope that this hypothetical tool will get the necessary testing on the community build system, so it'll be actually usable and useful.
I've never looked deep into SBT myself, but the sbt-native-packager plugin use to be an insane mess of craziness the last time I looked at it.
As far as tooling, I got pretty use to running sbt even if I got red lines and felt that my types were correct. Often it was to see what the compiler error would be and then I'd be surprised to see it compile with no warnings.
Scala is complex. The Intellij issues an be frustrating, but I can also understand how difficult it is to resolve the complexities in Scala.
I still <3 Scala though.
Typelevel ecosystem is thriving and has great fp projects including Cats, Monix, Doobie, Circe and Fs2.
Having Oop is a great gateway so we can convince our pointy haired bosses and mainstream colleagues that Scala can be gradually adopted into a project
Reactive and actor based patterns, Lightbend and so on don’t get a mention and yet are used throughout industry and are very valuable tools
ZIO maybe 100x faster than Future in certain benchmarks but can I get a 100x speed up on my real world oop code, my actor code, even my future based code, by switching to ZIO
It's important to note, the world has transitioned to Python3. Django removed the py2 comparability imports a while ago and the Python3 Wall of Superpowers stopped updating a while back when it was over 90% green.
For most devs, all new Python code that's green field is Py3. There is still a lot of py2, yes .. a lot in commercial/closed apps that have embed python in there, along with scientific applications. But I think that will change eventually too.
Perl6 is the opposite end of the spectrum. Massive Perl codebases will always be Perl5. I don't see any enterprise people going to Perl6. If you're like Booking.com and still using Perl for everything, you code is probably so decrepit, old and lacking meaningful testing you should just burn your entire company to the ground so your devs are no longer in miserable agony.
He has a point with Python3: it is a large community. Even with that large community, and even with how small many of the syntax changes were, it took a long time to get to py3! Scala is nowhere near as big.
Scala needs to move carefully with Scala3, so that Scala3 is more like Python3 and doesn't end up like Perl6.
What makes you think that? The perl community pioneered a lot of the current best practices around testing.
A lot of the old perl code bases are still around because there is no compelling reason to move away from perl.
That's a lot of assumptions:
* Booking.com still using Perl for everything
* Code being decrepit
* Code lacking meaningful testing
* Developers in agony
Do you have some inside information to know that all of these are true?
You also don't seem to understand that companies are in it for the money, not to provide nice new development projects to make developers happy. If you are programming for a living, you should know that programs are written to make money.
By the way, torturing the developers to make a better product, is a well known meme among Perl 6 core developers.
> doesn't end up like Perl6.
Perl 6 is ending up quite nicely, thank you.
Any maybe that's something Py3 could have handled with a py2 interop (at module load time you set assumptions that lift your py2 code into py3 with those assumptions). But python is not that big on these hacks, it instead just adds 10 more incomplete libraries into its std and calls it progress!
Based on this talk, it's not likely I'll ever become a Scala developer.
For reference, I learned FP in the nineties while in university; was exposed to monads, and a few haskell dialects (Gopher mainly). Been doing Java for about 20+ years at this point as well. However, despite this background I have trouble wrapping my head around Scala. The documentation seems to require a phd, which I have, just not in esoteric type system kungfoo. I find most Scala code completely unreadable. I admit, it's me. I'm probably too stupid or something. But if I'm too stupid, then most people I work with are definitely not smart enough either. So, that in a nutshell is why I ended up doing Kotlin instead.
Completely agreed. Though the funny thing is, Scala has/had a Documentation Tzar and she is trying to get things to a better place. To supposedly take care of this. I'm very much a learn by example type, so I'd appreciate a lot more examples (e.g. for methods: https://www.scala-lang.org/api/2.12.6/scala/collection/$colo... ). On the other hand, the documentation is open source in the best way: https://github.com/scala/docs.scala-lang/commits/master anyone can shoot a PR if they feel they can/want-to contribute. And there's plenty of overview articles ( https://github.com/scala/docs.scala-lang/tree/master/_overvi... ) covering a wide range of topics.
Sometimes the fault is with the libraries and their documentation. Especially anything that's HC FP (scalaz : Let the types speak for themselves via the Scalaz Scaladocs! and then you're completely at the deep end - for example: https://static.javadoc.io/org.scalaz/scalaz_2.12/7.2.24/scal...):F[(A,B)] ).
What makes Scala code unreadable is - IMHO - that people rely too much on the IDE to help them understand and navigate code, or their own familiarity with that code (or in some cases the code is simply write only, making it more like obfuscated Perl and KDB+/Q than a modern object-functional hybrid language).
And since Scala has so many faculties, features, facilities to mange code, to abstract problems and their solutions, and people write almost none of their design choices down into READMEs or comments, we're left with the code. For some people it speaks for itself. (For some people types like a -> a tell everything, for me that's just a function with unknown "functionality", because it depends on what the fuck the function actually does. Sure, the type signature constrains the expected return value, and it's useful, but that's not the architecture, not the rationale of/for design choices, it's just the safety striping around the sharp edges.)
Somehow Scala is "the PHP" at the other end of the spectrum. Instead of being too little it usually ends up being too much (insert the relevant Paul Philips rants here), while both are perfectly capable of expressing nicely designed programs/systems.
Furthermore, Scala is actually a bit weak when it comes to expressing really high level concepts. So yes, you can do a lot of magic in it, but most of it ends up being of the Black variety, because "for performance" (minimize GC pressure) or because that part is still "experimental" (macros/reflection) or because that part is just ugly and badly designed (collections).
Here is how I read this in case it helps anyone
Def zip[A, B](a: ⇒ F[A], b: ⇒ F[B]): F[(A, B)]
The method zip has two parameters, one a by name reference to something of type F[A] and the other a by name reference to something of type F[B] and returns an F of the pair of input types (A, B)
Looks like this deals with three total types, a container type F that must be the same for input param a, input param b, and the result.
I know nothing about F other than the fact that it contains another type. I know nothing about A and nothing about B which means I can’t be looking them up somewhere or using defaults, so the only way I could possibly return an F[(A, B)] is if I combined the two inputs.
Also, code written in this style is meant to follow the substitution principle, so I’ll think through a couple of substitutions to see how this method could be useful to me.
For: F = List, A = Int, B = String
Def zip[Int, String](a: => List[Int], b: => List[String]): List[(Int, String)]
F = Future, A = User, B = LastLoginDate
Def zip[User, LastLoginDate](a: => Future[User], b: => Future[LastLoginDate]): Future[(User, LastLoginDate)]
F = Option, A = IpAddress, B = Port
Def zip[IpAddress, Port](a: => Option[IpAddress], b: => Option[Port]): Option[(IpAddress, Port)]
Some(IpAddress), Some(Port) => Some(IpAddress, Port)
Some(IpAddress), None => None
None, Some(Port) => None
None, None => None
Reading signatures is one thing (and zip is "easy"), but without examples it doesn't really help in most of the cases. Because even if I know the concept, I don't know when it comes handy. (For example I just looked at Forall ( https://static.javadoc.io/org.scalaz/scalaz_2.12/7.2.24/scal... ... Forall is an endofunctor in an endofunctor category ... ah yes!), and then there are tools that come with labels (DList's doc page: nice for append heavy workloads, eg. logging; yes, great, what's Forall good for? What's Strong? Why Alpha exists? What's =>: ? That's not even documented :( It cannot be searched for https://github.com/scalaz/scalaz/search?utf8=%E2%9C%93&q=%3D... ... It's not on the cheatsheet http://eed3si9n.com/scalaz-cheat-sheet .. .I mean it's on it, but it is only used, not defined. etc.)
That's not a type reading thing. That's the bad documentation, and the learner-unfriendly naming convention.
As a potential newcomer to Scala, many of these points ring true. Hopefully the community takes a good hard look at itself to see what it can improve.
With some time off this past week I started hunting for things I should be learning. After doing some research on Scala's popularity and trending I came to may of the same conclusions John does.
> the community takes a good hard look at itself to see what it can improve
This has already been happening for a while. FYI, the Scala center is entirely dedicated to improving Scala usage and its tooling, which JDG somehow forgets to mention (along with many other positive things happening in the community).
In fact, it turned out in discussions on reddit that JDG is kind of living in his own bubble, not really aware of a lot of things happening outside of it, and thinks he is the only one to see some problems that have in fact been worked on for years. For example, tooling with the Scala center, and the Scala2/Scala3 transition which has been a _central_ consideration in the design and implementation of Dotty since the inception of the project!
It's also Haskell for the JVM. Made me wonder why there are two efforts. Found an explanation: https://github.com/typelead/eta/issues/3
Having been to Lambdaconf, I can attest to it being an amazing event where people can nerd out shamelessly about functional programming without any bullshit or hostility. I highly recommend it. Easy to forget this is what all of open source used to be like, before the sociopaths moved in.
The supreme irony is the comment above seems to come from a guy kicked out of the Drupal community by the same kind of moralizing busybodies, for being too neuro-atypical and abrasive...
As for being kicked out, that's for the better. Yeah, it took me long enough to get over it but really, those days are gone. I should've left on Jan 8, 2013 when it was made clear how much we've grown apart. I didn't, both the community and I suffered for it. Ending it was good, even if took me close to two years of grieving to get past it.
I fundamentally disagree with calling the CWG "moralizing busybodies", see my take on them https://medium.com/@chx/women-of-drupal-ive-failed-you-and-i... here. What I described there goes for any tech conference organizer: when you are given a chance to amplify a voice, make sure it belongs to someone who deserves their voice amplified. It never was an empty consideration but past #metoo it matters more than ever.
Once you have your eyes open... really, I was watching a pair ice skating on the TV while in the dentist chair (otherwise I wouldn't watch such) and it was appalling how much more skin the clothing of the woman showed than the male. Everywhere we sexualize women. Let me remind you of that time when the South Park creators went to the Oscars in female clothing http://www.trbimg.com/img-535ac5e8/turbine/lat-oscarmoment-l... this?
I think people get snippet fever when they see a few lines of really pretty code and they forget how tangled up the scala scene actually is.
I can't believe, after seeing more Rails disasters that I know what to do with, that I'm looking at a software project and thinking "this shit is almost as bad as Rails."
Scala 3 could've been the opportunity to murder sbt to death, abandon implicits entirely, make the breaking changes they need and compose the foundation for 10+ more years of scala hackery. But from what's visible out of dotty currently, it's almost entirely invested in the old sins, with a stronger soundness story nonetheless (congrats DOT team). There's just no way to slow the train now, too much of it is dangling off the bridge and Spiderman won't be able to pull it back.
Good riddance. Thank goodness F# is starting to come together.
abandon implicits entirely,
If you don't like implicits, simply don't use them. You can add a style checker to your build automation that finds and rejects the keyword "implicit" in your code. As to use of implicits in libraries, you are perfectly free to pass explicit arguments instead.
Odersky has a couple of recent youtube videos where he explains implicits in detail.
Could you write a few sentences about this? What happened? What do you think should have happened instead?
> scalaz/cats debacle
Again, for the unfamiliar, what is this debacle?
> the breaking changes they need
What are those?
> murder sbt to death
yes pls! ( https://github.com/lihaoyi/mill is actively maintained and developed and seems to be the sane replacement for sbt )
Scala _really_ needs to die.
They are ugly, unergonomic, old, unmaintained (or worse, pumped full of decades of features, but never really refactored), and some are just simply broken.
Some libraries are of course the exception to these and all due respect toward them.
Rust is great if you want a language with a lot of type/bug safety and want to be able to compile it to run naively on everything from servers to embedded devices.
Scala tries to bring a functional language that can utilize everything that already exists on the JVM, including all the classic Java libraries, while providing a lot of FP idioms, pattern matching, etc.
If you write good Scala code (basically avoid anything and everything written by TypeSafe/LightBend ever .. except for TypeSafe Config .. that one is okay, and it's in Java), you can have some really awesome, clean looking code and programs compared to their Java equivalent boilerplate.
They both have their purposes.
Which feature(s) is(are) there in Scala which trump running straight machine code for maximum speed and efficiency? I write software which runs on the command line and is designed to take command line options and work well with pipes like all the other UNIX®️ tools and the software has to be efficient on resources and lightning fast. Sell me on Scala.