Hacker News Reader

Comment by virtualwhys | Hacker News Reader

FYI, pron's the author of a would be competing project to Scala's Akka (i.e. competing for share of distributed computing market on the JVM), and takes every opportunity to danger-danger-dragons-be-there on Scala related threads/comments.

Saying that, there is always some truth in what pron brings to the table, it's sort of a higher kinded trolling ;-)

As for Scala and PL concepts, sure, plenty on offer if that's of interest, but not at all a requirement in using the language. A great example of a highly successful project based on the non-FP-centric aspects of Scala is Spark. See their high performance computing style guide [1], no monad transformers there...

[1] https://github.com/databricks/scala-style-guide

View on HN

FYI, I had taken "every opportunity to danger-danger-dragons-be-there" Scala — on record — years before I even dreamed of the database company that would one day have an open-source project which would end up competing with Akka (I'm talking 2009/10, while Quasar was first conceived in mid 2013).

It was after leading an attempt to adopt Scala at a large organization, which failed because of what I saw as Scala's bait-and-switch marketing: it presented itself as "a better Java" while it really was (and is) a strict-Haskell-with-OOP-and-macros-and-more. We needed a car and got a helicopter. While some would consider a helicopter superior to a car, I think everyone would agree it imposes a completely different set of tradeoffs. So yeah, for a good while I was telling people, listen, it's a helicopter, so make sure a helicopter is what you want (the Scala community’s response was, what is this FUD? Can’t you see it's got wheels?! You can drive it on the ground if you want; you don’t have to take off! To which I responded, but you still can’t take it to a car garage for maintenance, to which their response was, you’re a stupid biased troll!). I don't remember when was the last time I participated in a Scala thread, but that community doesn’t forget. So if I'm damned if I don't, I might as well do:

Spark is a good example for a project that would have been better off using Kotlin. They wouldn't have missed any Scala features, would have had better build times, better IDE support and better Java interop (of course, Kotlin wasn’t mature enough to even be considered when that project started). In fact, using Kotlin would have saved them writing 95% of that style-guide[1], which reads like instructions on how to drive a helicopter on the ground without accidentally taking off (Perhaps ironically, that style guide drew derision from the Scala community, who said, you're in a helicopter for fuck's sake! Why won't you take off?!)

[1]: No implicits, no multiple argument lists, no Try, no infix notation, no non-math operators, no recursion, no being-fancy-with-monads, no Scala concurrent collections, no Scala collections, no for loops. They don't mention dynamics and structural types, but I think it goes without saying that they're off-limits, too.

Interesting analogy, incredible how Spark drives the Scala helicopter like a car; they seem to have nailed parallel parking ;-)

Seriously though, I doubt Kotlin would ever be the first choice for a project like Spark since Scala excels in the DSL department (really without rival on the JVM, and off, only Haskell comes close). Look through Spark source code [1] and you'll see: operator overloading, (sealed) ADTs, exhaustive pattern matching, etc. functionality that you can't even think the thought in in a language like Kotlin. This has nothing to do with monad transformers and other FPisms, just straight DSL power.

Finally, to put things in perspective in terms of the mindhsare that Scala and its ecosystem are drawing, IBM is investing over 100 million dollars per year in Spark research and development [2]. Sorry, but that's absolutely massive, maybe there are no dragons? Or if there are, the big players don't seem overly bothered.

[1] https://github.com/apache/spark/blob/master/sql/catalyst/src...

[2] https://www-03.ibm.com/press/us/en/pressrelease/47107.wss

Do you actually know Kotlin? It has many features for DSL support, including operator overloading (though not the ability to invent random new operators fortunately), infix notation, a form of exhaustive pattern matching, extension functions, annotation processing, inlineable anonymous extension functions which sounds bizarre but actually is key to defining many DSLs in the language, and so on. It's a different approach to Scala but to say there's no way to do DSLs isn't right.

As an example, check out funKtionale. It's a library that adds various functional features like function currying and partial application to Kotlin, entirely by using its DSL features.

Or our Kotlin actor API: http://blog.paralleluniverse.co/2015/06/04/quasar-kotlin/

1. None of those features (except sealed ADTs, which really aren't necessary) is missing from Kotlin. The set of Scala features Spark is using is pretty much identical to Kotlin.

2. I don't see how IBM's investment has anything to do with Scala, though. Spark is a great product, and deserves an investment even if it were written in BASIC.

3. Even if Spark did rely on Scala features that aren't in Kotlin (it doesn't), Spark is a library/framework/middleware completely unlike enterprise projects. It is orders of magnitude smaller, for one. That is not to say it isn't awesome, but I wouldn't extrapolate from its choice of technologies (including the language) to the enterprise.

4. Clojure makes writing DSLs easier and with far less magic. Groovy DSLs are also as powerful as Scala's and easier to write. Kotlin DSLs are not as powerful, but are more than enough for Spark.

5. I've never, ever, said Scala is unusable. In fact, I've always said (and believed), it is a very powerful language. I have said that that kind of power is not what most companies want or need, and that that kind of unopinionated kitchen-sink, cutting-edge language is probably not suitable for long-time maintenance in large companies. I completely stand by this. My problem with Scala, again, is not that it's a bad helicopter, but that it insists that it's a good choice if what you want is a car (unless you actually try to drive it like a car -- as Spark does -- in which case you invite the ire of the entire community), and, IMO, it isn't.

6. It's OK to not like your language. It's OK to say it. It's OK to explain why. And it's OK for you to disagree. That doesn't make me biased, and certainly not some kind of bogeyman. It also doesn't justify ad hominem attacks, which -- unlike my criticism of Scala -- are completely unfounded.

Good points, as always.

> None of those features (except sealed ADTs, which really aren't necessary) is missing from Kotlin. The set of Scala features Spark is using is pretty much identical to Kotlin.

I didn't mention implicit conversions, but that's one area where Scala has taken the high risk high reward path. If you look a bit further down on the linked Spark page you'll see the conversions defined [1]. I'd say Spark's style guide is more geared toward Scala consumers and not necessarily library authors (i.e. if you know what you're doing have at it; otherwise keep to a strict subset).

Basically, in aggregate Scala's feature set is incredibly powerful; that power (despite the complexity, tooling challenges, and slow builds) is what draws dsl authors to Scala.

When IBM throws down an absolute boatload of cash on a project written in Scala, by proxy Scala benefits. Right off the bat there are de facto 3,500 new Scala developers (since the initial IBM Spark team will of course be learning Scala). Furthermore, it's not like competitors aren't aware of this, Spark automically becomes more of a "big deal" as a result, which cannot help but be a boon for Scala.

As for other JVM languages, agreed, Clojure has its own supply of awesome, though the lack of (built-in) static types are a no-op here. Groovy, meh, seems to be on a downtrend, I suspect non-immutable-by-default dynamic languages (except of course, javascript) will continue to fall out of favor.

Now, Kotlin, it's a nice language. If they can deliver on near Java build times, might just be the Java killer that Scala will never be, or at least not in its current design (where, as you rightly say, there is often more than one way to do the same thing).

Let's see how things play out over the next 3-5 years, JVM alternative language landscape is diverse and evolving quickly, good times now and great times ahead...

[1] https://github.com/apache/spark/blob/master/sql/catalyst/src...

> I didn't mention implicit conversions, but that's one area where Scala has taken the high risk high reward path.

We C++ veterans know that implicit conversions are one of the worst ideas to put in a language. They start simple enough and then they get clever, which is where trouble starts.

> i.e. if you know what you're doing have at it; otherwise keep to a strict subset

There are few things about programming language design we know, because getting empirical data is so hard. One of the few things we do know (from C++ in the nineties, of course) is that this kind of design doesn't work. The average codebase lifespan is about a decade, during which time people move around and team leads change. Everyone has their own discipline, and by year 5 (usually earlier), the codebase is unmaintainable.

Of course, now that C++ has been re-branded and re-purposed as a specialized language for close-to-hardware software, and is used by specialists, it is doing quite well. I certainly wouldn't mind Scala marketing itself -- like C++ and Haskell do -- as a language for specialists (although even specialists would be better off opting for the excellent OCaml implementation on the JVM, or even Frege if Haskell is your cup of tea).

There is absolutely no reason to be as multi-paradigm as Scala is on the most polyglot platform around, which brings me to:

> that power... is what draws dsl authors to Scala.

One of the great things about the JVM is its relatively easy inter-language interop. I would use a meta-language made for DSLs (like Clojure, MPS or Xtext) for my DSL, while I'll write the infrastructure in Java/Kotlin. Learning Java and Clojure, or Kotlin and MPS, is easier than learning Scala alone (well enough for DSLs), and the results are cleaner, produce more maintainable code and better error messages (well, I wouldn't say Clojure excels at error messages, but at least the DSL writer has full control over them, and can make them quite good with an extra bit of effort).

Also, I'd like to see what those DSL codebases look like after five years of heavy usage (not to mention the error messages Scala spits out, which are completely unparsable to anyone who does not understand how the DSL is constructed).

> We C++ veterans know that implicit conversions are one of the worst ideas to put in a language

Too late to yank them now, can be disastrous in the wrong (read: beginner) hands, but they're fantistically useful in the dsl department so...

> The average codebase lifespan is about a decade

maintainability is an issue for the long haul, sure, Scala won't win that battle until it stabilizes, which is at minimum 3 years away when Dotty (Scala 3) comes on the scene.

> I certainly wouldn't mind Scala marketing itself -- like C++ and Haskell do -- as a language for specialists

Typesafe does the marketing, and as a business they'll never sell their technology short, even if what you say is closer to the truth wrt to specialization.

> better off opting for the excellent OCaml implementation on the JVM, or even Frege if Haskell is your cup of tea

no users = no dice; doubt either will grab much mindshare in the near-term. Personally I would love to see SML cherry pick the non-crufty good parts of OCaml (including coming implicit parameters) and have that find a following outside of current tiny academic circle it languishes in.

> Also, I'd like to see what those DSL codebases look like after five years of heavy usage

They probably look a lot different than how they started out as...paraphrasing someone, "you're going to rewrite it [your application] at least 3X" ;-)

Basically the entire Scala ecosystem is a moving target, including the language itself. Enterprise outfits looking for a static dependency graph that will work today and 10 years from now should look at Java as the host language. I doubt Kotlin will be able to deliver the same guarantees, but if it can, watch out Oracle.

> Typesafe does the marketing, and as a business they'll never sell their technology short, even if what you say is closer to the truth wrt to specialization.

I don't think that's selling it short. Accurately describing your product only prevents people who aren't really your target -- like me -- from being disappointed and telling the world about it, while it helps attract precisely those people who would be most excited about it.

Also, I think Typesafe realize now that there is not much money to be made in developing and supporting a programming language (ironically, there is some to be made -- though not too much -- supporting specialist languages; see Ada and Matlab). That's why they want to change their name, right?

> no users = no dice; doubt either will grab much mindshare in the near-term.

That's splitting hairs a bit; Scala doesn't exactly have mass appeal either. We're talking about the difference between tiny and small.

> I doubt Kotlin will be able to deliver the same guarantees, but if it can, watch out Oracle.

I know they're trying to. Also, I know Oracle actively encourages alternative JVM languages. Their product is the Java platform, which currently -- to be honest -- is not too far from being the same as Java the language (all alternative JVM languages combined don't amount to 10%, or possibly even 5% of the JVM ecosystem). But they wouldn't mind if that changes. Their research team working on Graal, HotSpot's next-gen JIT, coming to OpenJDK in Java 9, spends as much time on making sure it runs Ruby, R, C, Python, and JavaScript well as it does on making sure it runs Java well. I know Oracle's Java team isn't all too happy about it, but I also know that that's what Oracle wants.

To add to my previous comment (now below), Scala DSLs are without rival only in the complexity of their composition. Clojure DSLs are based on macros. Groovy's are on builders and method-missing. Haskell's are on symbols, monad syntax (do) and lazy evaluation. Kotlin's are built with typesafe builders and inline functions. Scala's are built with symbols, right-associative operators, macros, implicits, monad syntax (for) and possibly even dynamics.

Even this most powerful yet most dangerously-clever capability offered by some programming language is supported by like five different Scala features with ten different ways of achieving the same goal. It's absolutely perfect if you want to explore different ways of programming. It's not so great if you need to maintain your DSL for the next 10-15 years in a large organization.

Besides, given that the JVM has two of most advanced meta-programming tools anywhere, Xtext[1] and JetBrains' own MPS[2], I wouldn't call any JVM language's embedded DSLs "without rival". All this is about to get even better with Truffle/Graal in Java 9. Saw an interesting talk about easily constructing very efficient languages with Truffle and object algebras at ECOOP[3]. So, especially on the JVM, going for an embedded DSL, let alone one built with very complicated techniques (rather than just Lisp macros) is probably not the best way to use DSLs. It has no advantages and lots of disadvantages.

[1]: https://eclipse.org/Xtext/

[2]: https://www.jetbrains.com/mps/

[3]: https://twitter.com/biboudis/status/618402989720662016

I very quickly skimmed through that scala style guide and starred it on Github, but having used both Kotlin and Scala, I am of the opinion that Kotlin (once it reaches v1.0) would remove the need for ~50% of the content of this style guide.

What is a pron? That's like a shrimp?

pron is the author of the GGGP post to the one you replied to.

Reply on news.ycombinator.com