Warts of Scala | Hacker News Reader

Warts of Scala

lihaoyi.com

64 pointsmanojlds14 comments

Hmm I would disagree that you its a problem that Scala requires explicit partial method application.

> For example, the Ammonite script-runner (which is a medium-sized command-line tool) spends up to half a second just classloading

That's not just because of the JVM; idomadic Scala generates volumes of byte code, due to anon functions and traits.

I understand that traits are something like typeclasses or interfaces: are single-method traits kept around as traits, or is the function itself passed around in the optimized output?

I'm actually suddenly curious about how other functional language compilers look compared to GHC.

Abstract classes can be efficiently encoded, since the JVM understands that. The Scala collections collapse common sets of traits into abstract classes.

Traits are copied, always I think. Only traits with all final methods could even possibly be not duplicated.

Ammonite ends up loading about 2000 classes to run a cached script (i.e. no compiler required, no parser required, ...)

    $ cat foo.sc
    println(12345)

    $ java -jar -verbose:class $(which amm) foo.sc | wc -l
        2015

Hello world already loads about 400 classes:

    $ cat foo.java
    public class foo{
      public static void main(String[] args){
        System.out.println(12345);
      }
    }
    $ javac foo.java
    $ java foo
    12345
    $ java -verbose:class foo | head -n 5
    [Opened /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/rt.jar]
    [Loaded java.lang.Object from /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/rt.jar]
    [Loaded java.io.Serializable from /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/rt.jar]
    [Loaded java.lang.Comparable from /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/rt.jar]
    [Loaded java.lang.CharSequence from /Library/Java/JavaVirtualMachines/jdk1.8.0_112.jdk/Contents/Home/jre/lib/rt.jar]
    $ java -verbose:class foo | wc -l
         420

Now, 2000 classes isn't nothing, but it's not a particularly large number. Any medium sized application using a few third party libraries easily reaches that number.

Very interesting article. In my experience writing Scala for the past year or so this article does indeed hit on many warts. But, there are many more warts not mentioned. The biggest two are the tooling (sbt for example) and the nearly incomprehensible standard library.

> ... methods should be called with as many sets of parentheses as they are defined with (excluding implicits), and any method call missing parens should be eta-expanded into the appropriate function value.

> Notably, removing the "optional parens" thing would not stop you from defining "property" functions that are called without parens; it would just mean you need to define such functions without parens, as is already possible.

> In other languages with first-class functions, like Python, this 'works fine' [quotes mine]

I'm not sure how this behaviour could be seen as 'more consistent'. If methods that have parens but are called with them stripped are eta-expanded but you can still define no-paren methods, you'd still have confusing behaviour where a no-paren method isn't eta-expanded as expected (because it's a property and so 'special').

Rather than more consistent, this behaviour is just 'more like Python'[0], which has this inconsistency too[1]. Essentially the suggestion in the article would mean:

    @ def x() = 1
    @ val y = x
    resXX: () => Int = $...

    @ def a = 1
    @ val b = a
    resYY: Int = 1

It's also worth noting that IntelliJ IDEA (and possibly Eclipse) warn you if you define a no-argument method with Unit return type (since `def action()`s should have parens and `def pure` values needn't).

IDEA will warn you (and offer to correct) if you call a method with too few parens (`def x()` called as `x` will display a very visible warning), so maybe there's some room to believe that this is a 'problem', but the solution involving automatic eta-expansion doesn't seem like a particularly sane one.

Also, calling `getFoo` without parameters is 'nice to have', but calling `name()` as `name` and `color()` as `color` on imported Java APIs both comes across as more natural in Scala (since Java has no properties, they can't be defined as properties upstream) and has allowed quite a few Java APIs to fit in naturally in Scala where they wouldn't have otherwise, in my experience.

[0]: All the eta-expansion things in this post are to 'be more like Python', which makes sense given Haoyi is also behind https://github.com/lampepfl/dotty/issues/2491 and https://github.com/lihaoyi/Scalite and (say) https://github.com/lihaoyi/macropy.

[1]: Properties in Python also have the underlying design of using descriptors, which are a whole other world of horror.

I don't see the problem with

    val b = a
    b: Int = 1

After all, if you wanted to turn it in a function, you can use `() => a` syntax which everyone is already familiar with.

I don't think it's necessarily wrong, I just think it's inconsistent (all methods are eta-expanded... except properties), just in a slightly different way.

Depending on what this did on conversion to Java-land, this could still be confusing for beginners:

`int foo() { return 1 }` in Java could either:

    @ def x = foo
    resXX: () => Int = $..
    // Why was this eta-expanded - it has no parameters?!?

    @ def y = foo
    resYY: Int = 1
    // This is 'normal'... but it wasn't defined as a property?

You could also argue that right now, if you want to turn any method (lacking generics) into a function, you can use the `someFn _` syntax, which in fairness perhaps not everyone is already familiar with.

Also, removing parens from an upstream method would change the downstream type, which seems more confusing to me from a 'statically typed language' perspective[0] than up-to-N-parens all having the same type - the rule right now is simple to explain as 'you can drop as many empty parens as you like and the call will stay the same (but IDEA will warn you)'.

[0]: In your scheme:

    @ def foo() = 1
    @ val x = foo
    resXX: () => Int = 1
    @ def foo = 1 // Upstream author changes their mind
    @ val x = foo
    resXX: Int = 1 // ???
    @ val x = () => foo // Have to go around changing to this.

... and vice versa (if an author adds parens, all of your property calls start magically eta-expanding instead)

What you say is correct, but I think your mental model isn't the mental model many people work with.

One possible way of thinking of things is "methods" and "function values", where you need to eta-expand something to go from "method" to "function value". This is something people get used to when working in Scala for a while.

Another way of thinking of things is "things you can call" and "things you cannot call". In this case, it does not make sense to care about "method" or "function value": if it's callable, it should remain callable in the same way as I pass it around, until I call it. If you come from a Python, or JS, or other background, this is the mental model you will have.

I personally think the second mental model makes much more sense, and the first is a weird artifact of programming in Scala that you don't see much elsewhere (though I admit C# has a similar dichotomy)

I don't find the "people may forget to leave off parentheses" a convincing argument. People may forget to call `.flatten` on nested lists too; doesn't mean we should go flatten everyone's lists for them automatically. Furthermore, if you as an individual wanted to flatten `() => T` into `T` automatically, you can easily do that within your own project with an implicit conversion:

    implicit def flatten[T](f: () => T): T = f()

The same way you could use an implicit to automatically `flatten` nested `List`s, or automatically call `.get` on `Option`s. Both of those are similar to automatically calling functions to avoid "confusion", and something I think most people in the community will agree would be a terrible thing to do

Hm, so I wrote Python for a decade before Scala (and I still write TypeScript & JS) and I'm deeply sympathetic to the 'passing around a callable' way of looking at things. That said, I find Scala's syntax more consistent.

I would regularly run into that 'wart' in Python, where changing something between a property and a method would completely change the downstream behaviour (property -> method makes downstream 'calls' eta-expand and method -> property makes downstream calls error, or call the eta-expanded function if the property happens to return a function, both of which are terrible IMO).

I guess more than caring either way about number-of-argument-lists, I just disagree (as does a sibling commenter) with automatic eta-expansion, because I think that 'people may forget to call `someFn _`' is unconvincing.

If you currently try to pass `someFn` bare, to something expecting a function, you get a very clear error (and in some cases you can actually do this and it's automatically handled, like in `.map(someFn)`).

On the other hand, I would find it deeply objectionable if the compiler started magically switching the type of an expression of mine based on the upstream's number of parens. If they change `def foo()` to `def foo` I don't expect the type of my `val x = foo` to change from `() => Int` to `Int` or vice versa.

I don't really buy the "what if the upstream author changes their signatures and downstream code behaves differently". Upstream signatures change all the time, and downstream has to deal with it. That's not unique to the act of adding parentheses to a function.

What if the upstream author makes their functions return `Option`s? Should we auto-unbox `Option`s with `.get` so downstream users won't need to change their code?

Of course not, and if you submitted a PR with an implicit conversion that did that, you'd probably get yelled at. So why are we happy auto "unboxing" `() => T` functions using exactly the same logic?

I guess what you see as 'unboxing', I see as 'leaving intact'.

In turn that is, I find your suggestion of automatically eta-expanding to be 'doing something' (auto boxing) and so 'why are we happy auto-boxing?'

As you've said about callables in your sibling post, you see 'doing something' to be changing what is rightly a callable into something not-a-callable (immediately calling the function) and I see it the opposite way (why are we turning what is a value into a function?) This is why I say we're going from one inconsistent approach to another (from Scala-style inconsistency to Python-style inconsistency). The difference is Scala-style inconsistency makes it harder to introduce type errors since all it does is force you to add `_` when you want to eta-expand.

The case that bothers me the most with your approach (again from a static safety perspective) is this:

    @ def foo = () => 1 // There exist less weird examples of this.
    @ val x = foo().toString
    resXX: String = "1"

    @ def foo() = () => 1
    @ val x = foo().toString
    resYY: String = "$...identifier" // No compile error

Now you could argue this is more a problem that functions and `Int`s have any methods in common at all (toString, equality, etc.). With some more time to think maybe I could come up with an example that doesn't rely on that (I've seen a lot more of this error in Python than in Scala, since Python is dynamically typed). But I find it odd here that the compiler changed the type in response to upstream paren count.

Edit: since I can't reply below - thanks for the discussion - as you say, this is probably just different mental models. :)

> Now you could argue this is more a problem that functions and `Int`s have any methods in common at all (toString, equality, etc.)

Now that's something that I should have put in the post! But it was already too long...

Anyway, I think we already fully understand each other's views, and there's not much new to be said. Cheers and thanks for the discussion!

Reply on news.ycombinator.com