Effective Scala | Hacker News Reader

Effective Scala

twitter.github.io

129 pointsauggierose71 comments

Scala is becoming my programming language of choice, especially since I tried out Scala.js which really works well.

I am currently porting a Clojure library of mine to Scala, and it is about 10% less code, and obviously more robust because of static type checking.

Scala.js: https://github.com/lampepfl/scala-js

that runtime is larger than firefox update. why not make a proper user agent?

I'm not sure how that's relevant. A full Firefox install on Windows currently weighs in at 47MB. xul.dll (e.g. the UI) is 20MB of that.

As a word of warning, Scala.js is very new project (started I think only 7 months ago) and thus it's very immature.

I am using Scala.js for a week now, and I am surprised by how mature it already is. Optimized Scala.js is less than a megabyte, and therefore actually usable. Obviously there are caveats, for example Character.isLetter is not implemented; but the entire collection library of Scala is available.

robust is debatable, I suppose you mean type safe.

I mean type safe, therefore much faster code out of the box, easier refactoring (not trivial refactoring, but changing interfaces of libraries and the like), better documentation generator, better build tool, up-to-date tools for google app engine, great Eclipse support, ....

In short: more robust

You actually mean statically typed. Scala is only limitedly type-safe, and is certainly not type-theoretic type-safe at all:

    $ cat NotTypesafe.scala                                                                    
    object NotTypesafe {
      def main(args: Array[String]) {
        println("5".asInstanceOf[Int])
      }
    }
    $ scalac NotTypesafe.scala
    $ scala NotTypesafe
    java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer

http://en.wikipedia.org/wiki/Type_safety

Well, I've got a PhD in computer science, I just use the words as I please. But yes, you are right ;-)

I bet this comment read differently in your head.

Can you compare and contrast Scala with Clojure? I've dabbled in both... I in theory prefer Scala, but I found it just has sooo many moving parts I can't keep it in my head.

I can show you two code samples of mine, one in Clojure, the other one doing something very similar in Scala:

  (defn completion-items [grammar doc items-bin-index completed-bin-index items nonterminal value]
  (loop [items items
         completed '()]
    (if-let [[^Item item previous-value] (first items)]
      (let [alpha (expected-symbol grammar item)
            dot (.dot item)]
        (if (and alpha (= nonterminal alpha))
          (let [rule-nonterminal (.nonterminal item)
                ruleindex (.ruleindex item)
                origin (.origin item)
                value' (gen-nextvalue grammar doc origin items-bin-index completed-bin-index
                                      rule-nonterminal ruleindex dot
                                      previous-value value)]
            (if (nil? value')
              (recur (next items) completed)
              (let [item' (->Item rule-nonterminal ruleindex (+ dot 1) origin)]
                (recur (next items) (cons-item-value completed item' value')))))
          (recur (next items) completed)))
      (apply hash-map completed))))

  --------------------------------------------------------

  def complete_items(completed_binindex : Int, bin : ItemBin, nonterminal : Nonterminal, value : Value) : MoreItems = {
    var completed_items : List[(Item, IntermediateValue)] = List()
    for ((item, v) <- bin.items) {
      val s = expectedSymbol(item)
      if (s == Some (nonterminal)) {
        grammar.nextValue(document, item.origin, bin.binindex, completed_binindex, nonterminal, 
          item.ruleindex, item.dot, v, value) match 
        {
          case None => 
          case Some(nextvalue) =>
            completed_items = (item.inc -> nextvalue) :: 
              completed_items
        }
      }
    }
    completed_items    
  }

That's quite non-idiomatic Clojure...

1) Unnecessary type hint

2) No use of destructuring or field lookup by keyword

3) Alteration of record objects with positional constructors instead of update-in

4) Explicit loop with previous item instead of reduce over (partition 2 1 items)

5) Way too many parameters

6) Apply hash-map instead of into

7) (+ ... 1) instead of inc

8) Weird gen-nextvalue auxiliary function instead of a lazy sequence

I could keep going...

I don't suppose I could talk you into re-writing it into idiomatic clojure? Since someone else already rewrote it in idiomatic scala.

Just guessing from context, but probably something like this:

    (defn completion-items [items options]
      (->> (for [[item v] items]
             (when-let [nonterminal (expected-symbol item options)]
               (when-let [next-value (gen-next-value v options)]
                 [(update-in item [:dot] inc) next-value])))
           (remove nil?)
           (into {})))

The thing is: In order to have performance, you need type hinting (a lot of it). I DO need performance, as the algorithm above is already part of something O(n^3). There also clojure.core.type does not help. And that is the problem with a language feature which is not a compiler feature as well.

By the way, gen-nextvalue is a protocol function.

> In order to have performance, you need type hinting (a lot of it).

No, not really. That has only been my experience when interoperating with Java libraries. Protocol dispatch has inline call caches, which are almost as fast as normal java interface dispatch.

> that is the problem with a language feature which is not a compiler feature as well

You could also say "that is the problem with type system features which are not virtual machine features as well". There is lots of erased information in the Scala type system that could be used for optimizations, but the VM can't make much use of that.

Scala's type system does 3 things:

1) Enable type-related optimizations

2) Ensure internal consistency (ie prevent errors)

3) Enhance expressivity through logic solver search

Clojure's type hints provide just enough type system to trigger host optimizations, where as core.typed is for finding and preventing bugs. I consider the implicitness of #3 to be an anti-feature.

By the way:

idiomatic == I am doing what I am told to do without understanding why it sometimes makes sense, and sometimes doesn't, and sometimes just doesn't matter.

Here's a more functional version of complete_items, if you're interested.

  def complete_items(...) = bin.items.flatMap((item, v) => 
    expectedSymbol(item).map(nonterminal => 
      grammar.nextValue(document, item.origin, bin.binindex, completed_binindex, nonterminal, item.ruleindex, item.dot, v, value)
        .map(nextvalue => (item.inc -> nextvalue)
    )
  )

What can I say, I like my loops :-)

http://arxiv.org/abs/1007.3023

Loops obviously have their place, no question, but you're going to find that assigning a List to a var, and then appending to it / re-assigning it is going to generate a ton of garbage, and in general will just be slow all around. There's a reason that prepending/appending to Lists is "strange" in Scala. This is why map and flatMap are so great, because they take care of a lot of that heavy lifting for you (eg the matcher on None/Some -- this is what map is made for!).

There is no performance difference here, map etc. are producing the same garbage. Scala isn't Haskell. Also note that there is a HUGE performance difference between appending and prepeding to a list. :-)

If you need to append, use a Vector instead.

> Also note that there is a HUGE performance difference between appending and prepeding to a list. :-)

If you're going to do a lot of appending, might I suggest a ListBuffer or a Vector?

I don't think that's right.

TraversableLike, and FilterMonadic will both size up the resulting return value based upon the length of what's passed to it. You'll get a List of N items, and the call to flatMap will also traverse the list to remove/flatten the Options. Items will then get dropped in the List. You're looking at one or two collectable objects every time you run through that method, as opposed to bin.items.length collectable objects in your original code.

Look at it this way-- what happens if bin.items is a List() of 10M objects? completed_items can be replaced up to 10M times. The .flatMap creates one collection of 10M length, removes the None objects, and returns the resulting List.

If you like loops, here it is with a for-comprehension:

  def complete_items(completed_binindex: Int, bin: ItemBin, nonterminal: Nonterminal, value: Value): MoreItems = {
    for {
      (item, v) <- bin.items
      _ <- expectedSymbol(item)
      nextValue <- grammar.nextValue(document, item.origin, bin.binindex, completed_binindex,
        nonterminal,item.ruleindex, item.dot, v, value)
    } yield {
      item.inc -> nextvalue
    }

The usual argument for Clojure is that it allows for short and succinct code; but actually, Scala code is about 10% shorter: its type system is good enough so that you can do almost anything with it in a similar elegant and short fashion as in Clojure.

Programming in Clojure I miss the following features from Scala most:

a) Pattern matching

b) Having machine-checked documentation (aka type system), traits.

c) A proper documentation generation tool. The Clojure one doesn't do protocols properly, although protocols are the only real way in Clojure to have something akin' to interfaces and Scala traits.

For a and b you should check our core.match [1] and core.typed [2]. One of the best parts of using lisps is that things that are features of other languages can be implemented as libraries. Not to say Scala probably isn't great too, but I haven't ever really used it.

[1] https://github.com/clojure/core.match

[2] https://github.com/clojure/core.typed

IIRC Clojure itself is a library.

Not trying to sell you anything, just trying to help you find some things you've been missing:

a) https://github.com/clojure/core.match

b1) https://github.com/clojure/core.typed

b2) I agree that Clojure's facilities for mixins is under-utilized and that it lacks a proper facility for delegation (ie implicit conversions in Scala), but see (doc extend) for how awesome "traits as data" can be. Here's an example: https://github.com/stuartsierra/clojure.walk2/blob/2250e04c7...

c1) I hate generated documentation, but I understand why it's necessary for Scala. When I did some scala programming, I was so pleased with ScalaDoc, since it really helps you navigate that complex scala.collections hierarchy. Feels like having a pretty decent substitute to a good IDE. In Clojure, I don't feel the need nearly as much... Also, I quite like reflective documentation with the doc macro.

c2) Already covered interfaces/traits in b2

just a suggestion if the author is here - maybe be more consistent when showing bad vs good - sometimes you show the good then the bad and sometimes vice versa. Try to stick to one or the other - or alternatively change the background color of the examples to red and green to clearly show bad examples.

Effective Effective Documents.

The description about comments is somewhat wrong. The Scaladoc standard is not

    /** My comment
     *  Next Line
     */

which is pretty ugly, but it is

    /** My comment
      * Next Line
      */

which looks much better. :-)

The question i have been struggling to answer for few months now: Scala or Go?

If you would consider writing your program in C, then you should probably also consider Go. Otherwise, go with Scala ;-)

I think Scala.js is going to be huge. Just alone for that I would choose Scala over Go.

Like the C vs. C++/Java bridge - certainly more succinct answer than my reasoning... :)

But I certainly do not believe that Scala.js has a reasonable chance of having an impact alike GWT or such; maybe it can get close to (ruby) Opal or consorts, though. The main selling point of Scala still is Akka, and, at least in the web-dev world, the Play framework, I guess.

I like your explanation, actually, although you seem to contradict it with this comment here.

I am not using Akka, I am not using Play, I only use Scala because it is currently the language I can express my thoughts in the most elegant way, together with its industrial strength ecco system AND BECAUSE IT COMPILES TO JAVASCRIPT, TOO!!!

GWT never convinced me, Java is less productive than Javascript, so what is the point (except for legacy Java programmers ...).

Fwiw, Scala has several good web frameworks besides Play -

Scalatra - Scala's Sinatra

Blue Eyes - appears defunct but good example of pure asynchronous framework

Lift - strong built-in security, Netty integration wip.

Probably a couple more I'm forgetting.

Spray

I had been asking myself the same question over the last few months, maybe even longer. The final selling point for me was lines of code, caused by the feature-richness of Scala. While implementing a few generic data collections and algorithms I need for my project, I realized that in Go this resulted in much more code than in Scala, or I would have had to sacrifice type safety by declaring everything an interface{} (sort of Go's version of a "void pointer" for C guys). I now have solved this by using Go as my favorite "scripting-like" language because it is so easy to bootstrap something there, and it is nice to quickly build efficient binaries and even wrap a C library. But to manage huge code bases and projects, for me the choice in the end became rather obvious. Particularly, with respect to all the magnificent capabilities starting from generics (~ C++ Templates), to higher kinds, the coming macros, etc. that help to create succinct DSLs and the general benefit "FP && OO", over "FP || OO", in the end made this decision obvious to me. And I have not even touched the issue of the gigantic collection of Java libraries and frameworks you automatically get by choosing Scala...

Well, if you already use the JVM, its Scala of course.

But ask yourself this: do you work in a team or alone? One of the first sentences is:

"While highly effective, Scala is also a large language, and our experiences have taught us to practice great care in its application."

And thats the biggest issue - Scala is a bit like the JVMs C++, you need capable programmers in order to not fuck it up, because it contains every paradigm ever invented.

Go on the other hand is much simpler, and I like it for that - there is great value in simplicity.

I've been working with Scala for the last 2 years, and I also worked with C++. People claiming that Scala resembles in any way with C++ haven't worked with neither of them.

Also, when speaking of simplicity, I like languages with conceptual elegance. Scheme is simple too and that's a rather interesting example. You see, Scheme is homoiconic and has macros and continuations, a combination so powerful that you can easily build on top and efficiently use any pattern or paradigm under the sun. Also, being a Lisp, most builtins can be reimplemented in Scheme.

Now that's simple. What you're describing is actually easyness which is a very different notion. The main difference is that easiness is relative and depends on someone's own biases or limitations. The problem of course is that easy can overnight turn into hard.

Ever tried doing FP in Go? Ever thought about implementing your own data-structures? Try it.

"Haven't worked with neither" is a double negative. Also, 'resembles with' is redundant--'resembles' is sufficient.

I have written both Scala and C++ at my day jobs and I would say that Scala certainly does generally remind me of C++ and specifically in its proliferation of language features. I also know of several others with similar experience and opinion on this matter.

English is not my native language. I actually appreciate corrections.

I figured as much. I just threw them out there in case anyone could benefit (including you). :-)

Also, I completely agree with the rest of your original post (the parts RE: Scheme, Go, & simplicity).

I've worked with C++ going back to the mid 90s, and program Scala full-time in addition to managing a team of programmers. Such a comparison has never occurred to me. Scala is arguably a simple language in its core design. It enables quite feature-rich libraries, and you could argue those make the language seem complex in practice. But to me, it just shows the language is powerful.

I really do like Scala, however I also find it to be similar to C++ in the sense that lots of the Scala code I've read seems to be written by people who trade simplicity of implementation for being able to use every single feature of the language all at once. In other words, people unable to resist the urge to be too clever.

Sometimes the result is Scala source files that almost appear to be written in different languages, or lots of little libraries each with their own wacky DSLs that require a major investment of time to comprehend just to accomplish something that should be trivial.

These sorts of abuses you really don't see in Clojure even though it's just as possible in that language. Of course this is all based just on my experience, YMMV.

I think DSLs, heavy use of operators, and Unicode characters as identifiers isn't abuse. It's unfamiliar. In Haskell, for instance, syntax is taken much more seriously than in other languages, which tend to just copy previous languages in the name of familiarity. And it takes a while to become use to the basic syntax of Haskell, let alone the various libraries that extend on its ideas. But the syntax choices are not arbitrary.

But even Clojure is syntactically heavy compared to Racket or CL. But with good reason.

A powerful and expressive language takes effort, possibly even years, to become expert in. I'm not sure that making trivial things trivial is part of the plan. There are more appropriate languages for that.

I really appreciate Clojure's syntactic heaviness when compared to other LISPs. Just by using braces for associative data structures, and brackets for array's makes scanning through Clojure code much nicer because of the added visual cues.

You don't need proliferation of features to achieve expressiveness though (see Scheme or lambda calculus).

Most of the cruft in Scala is due to its support of O-O and its attempt to modernize it. Similarly, C++ is a superset of C rather than a complete break from it (and often, in the field, you find people using it as if it were 'C with classes').

Also, the O-O side of Scheme is certainly more powerful than F#'s O-O side; however, overall F# is at least as powerful of a language (and yet it's also much cleaner/internally consistent).

[Also, it takes only one counter-example to disprove 'all those who say that Scala resembles C++ have never worked with either' (which was my original intent here)].

"you need capable programmers in order to not fuck it up, because it contains every paradigm ever invented."

My team has just the opposite experience. Scala offers a lot of structure and discipline that you can take advantage of without being an expert, and it is a short time to ramp up to idiomatic usage relative to other languages. In fact, this is one of its benefits. By following some fairly simple guidelines, you can avoid making a mess. This has not been my observation with imperative programmers picking up a new imperative language.

Because of the easy-to-achieve discipline, Scala is ideal for teams in my opinion.

Pick Scala if:

  -You think a fusion of OO and functional programming is the way of the future
  -Runtime tooling and advanced instrumentation support matter to you
  -You like having first-class IDE support
  -You want to work for the likes of Twitter/Foursquare/LinkedIn/Amazon
  -Having tons of mature Java libraries available matters to you.
  -You don't mind investing time to learn the language

Pick Go if:

  -Memory footprint matter to you (and you don't want to pick C)
  -Startup time matters to you (and you don't want to pick C)
  -You don't want to spend much time learning the language
  -You like C-style error handling and aren't expecting something as expressive as a python or a ruby.
  -You don't mind limited IDE support
  -You want every post of yours to be voted up on HN regardless of content

I would add a few more:

Pick Scala if:

    -You find value in strong static assurances.
    -You find value in generics.
    -You want to learn functional programming, not 'functional' programming (and you should).
    -You value succinctness.

Pick Go if:

    -Compile times matter (of course it does).
    -You find value in easier deployments (at the cost of potentially more deployments).
    -You don't want to learn D ;)

Ha...love the last line. If I were in the market for something that solves the problems that Go attempts to solve, I would learn D. I just wish DMD was fully open source...it is awesome when you go to try a new language and all you have to do is "sudo apt-get ...".

I believe that very small sounding issue has played a huge role in D being as uncommonly used as it is. It seems like a trivial thing, but the ability to just pkg_add languageX is incredibly important for adoption.

I don't think this is a small issue at all. I love reading through an interesting tutorial, and being able to apt-get install the language or libraries I need to work along with it. But if you make me jump through a bunch of hoops just to work through a tutorial, forgetaboutit!

Pick Go if: your only other option is C

Pick Scala if: all other things

> -Memory footprint matter to you (and you don't want to pick C)

Is there any clear evidence of better memory management in Go comparing to JVM?

http://benchmarksgame.alioth.debian.org/u32/benchmark.php?te...

For memory intensive tests the score is 2-3 there.

For people with zero knowledge of either, I think the following analogy is somewhat (but obviously not completely) apt:

Scala is to Ruby what Go is to Python.

Interesting, but might want to elaborate a bit more. It's not completely clear what you mean. For example, another that comes to mind regarding prototyping vs production:

Python:C++::Go:Scala

Ruby and Scala are both feature-rich languages that give programmers every chance to abstract away any common logic that they can find anywhere, while remaining elegant at core. Python and Go are both minimalistic languages that provide basic abstraction facilities, but are designed to keep the programming language and source code simple and readable, even it comes at the expense of expressive power. Scala differentiates from Ruby with static typing, faster performance and better support for concurrency. Go differentiates from Python with static typing, faster performance and better support for concurrency.

I really wish they'd work on compilation speed. It probably takes me 20 minutes to do a clean compile of the Play Framework.

What if they can't make it much faster?

Like Go, they make the tradeoff not to have C++ generics for faster and smaller compiles.

They are working on that specific sbt /play combo, tho this doesn't look like huge numbers

https://gist.github.com/jroper/6374383

If you dig thru the internals list, there are a number of fronts they're working on to produce fewer short lived intermediate objects, parallelize builds etc

https://groups.google.com/forum/?_escaped_fragment_=forum/sc...

This is one of the reasons I've been playing with go instead of scala for future projects. Both are great, but go compiles so much faster it's like night and day.

Can someone knowledgeable in both comment on Scala vs. Rust in terms of the languages themselves (as opposed to the runtimes)?

i take it this is modeled after the "effective java" book? sounds interesting as a future reference to read.

Effective Java and Effective C++ have a good format for listing best practices, and nowadays more and more languages seem to adopt their own "Effective" book or web page... I like how useful they are to see real-world problems before you decide picking up a language.

Reply on news.ycombinator.com