Hacker News Reader

-- I'm not convinced that the overhead of copying immutable data doesn't outweigh the benefits of the ability to trivially parallelize the code.

Let me introduce you to my friend, Mister ConcurrentModificationException. He's a wily fellow-- he rarely pops up when you're doing local development, but he has this nasty habit of showing up in production whenever you start hitting a decent load. He's like Keyser Soze. You see him, then you try to debug the place he shows up, and poof he's gone, back into the shadows from whence he came, only to reappear when you try to iterate over that singleton java.util.ArrayList once more.

Now, let me introduce you to my other friend, Miss scala.collection.immutable.List. She's a superheroine. You can run code using her on one thread, or ten threads, or ten thousand threads, she doesn't care, you're always going to get the same iterator back when you call List.foreach. For the very small price of a little bit of garbage (which Mister ParallelGC takes care of very readily), Miss immutable.List will return that filtered list the exact same way, every single time you run the filter method with those parameters you set. You can run code with her in development, staging, production, she's always ready to go, and you'll never run into Mister ConcurrentModificationException when you're dealing with Miss immutable.List.

-- I'm not convinced that a project

What do you mean? All I read was "I'm not convinced that a project". Someone else must have come by and edited your post before I was able to read it.

singleton java.util.ArrayList

So...don't use data structures that are not thread-safe in threaded environments? Wrap it with Collections.synchronizedCollection. Or make copies, if you prefer.

For the very small price of a little bit of garbage...

What small price? If I have millions of things stored in an immutable data structure, do I not have to copy the entire structure (or at least some portion of it) when I want to modify anything inside of it?

It's actually all just pointers to data, so when you change something it makes a new node and copies a few new pointers so the new tree is still correct. All of the old data and pointers that aren't changed stay exactly where they are.

Ah, synchronizedCollection, the...

hang on...

wait a sec...

one more sec, someone's doing something...

now? no, not yet, wait...

how about... NOW! synchronizedCollection, the global lock for Java's mutable datatypes.

Here's the fundamental difference between Java and Scala: if I say, in Scala, val people = List[String]("quacker", "mark242") then by default people is an immutable list. I don't have to do anything special. In Java, either I'm suddenly using the Guava libraries to get something similiar (but not the same), or I'm doing all kinds of funky dances around list iterators, or I'm using Arrays.copyOf, or what have you. In any case, you have all of this extraneous code, when it isn't necessary.

Quick: give me a list comprehension method in Java that takes a list of Strings and returns that list, filtered, of Strings that are only five characters or longer. Make it null-safe and thread-safe. This isn't difficult-- you're thinking about 6-7 lines of Java code in your head, right? Null check, synchronized, new ArrayList, for(s in sList), that kind of thing, right?

In Scala, it's this. Some would argue you don't even need a separate method.

  def getFiveCharacters(s: List[String]) = s.filter(_.length >= 5)

If I have millions of things stored in an immutable data structure, do I not have to copy the entire structure when I want to modify anything within?

Dear god, man, what are you doing wrong in your Java code? This is the exact scenario where you want an immutable list, otherwise you'll be running synchronized code and precisely one core of your CPU will be glowing red like lava, while your other CPU cores sit idle.

I just did this in the scala repl:

  (0 to 1000000).toList.par.filter(_ % 100 == 0)

What that does is grab all of the integers from 0 to 1,000,000, convert them to an immutable List, then filter that List (yes-- an iterator! with a filtered copy!) by only taking numbers divisible by 100. The whole thing takes a couple hundred ms in the repl (which is fantastic, since it's compiling then executing the code) and almost no memory overhead. For code that is absolutely thread safe. And the ".par" makes this run in parallel on all my CPU cores.

> Null check, synchronized, new ArrayList, for(s in sList), that kind of thing, right?

> In Scala, it's this. Some would argue you don't even need a separate method.

It seems a bit unfair to force null checks on the Java code and then present a Scala alternative that throws NPE in two different places.

A good Scala programmer will never write the word null in their code. Ever. If I was overly concerned that the method was going to be called by Java code, I'd put this:

  Option(s).getOrElse(List[String]()).filter(str => Option(str).fold(false)(_.length > 5))

That should do it. I would probably go for

  def foo(s: List[String]): List[String] = s match { case l: List[String] => l.filter(_ match { case str: String => str.length > 5; case _ => false}); case _ => List[String]()}

It's a bit uglier, but I had to develop a habit of not spewing objects everywhere because of the constraints of the application I work on.

You're right that this is primarily a concern if the Scala dev wants to talk to Java. However, if the Scala dev does not want to talk to Java, perhaps a statically typed FP language with an HM type system would be a better fit.

I couldn't grok the original 1 liner, definitely not the second one and no way the last one. If there is a segment of programmers who grok and like it, cool. It still doesn't fix the performance problem of copying 1 million things before you can act on them because the copy would still have to happen on 1 core, or 1 core at a time, unless maybe the Scala compiler optimizes the code to divide the list among the cores available, if not though then immutability doesn't help performance during the copy.

You don't have to use immutable collections. If you want to use them, you should probably consider the cost of the operations you are using. Which particular operations are you worried about? Most updates to an immutable hash table or sorted map should copy only a small part of the collection, for instance.

> What small price? If I have millions of things stored in an immutable data structure, do I not have to copy the entire structure (or at least some portion of it) when I want to modify anything inside of it?

Depends on the nature of the modification; to choose the right immutable data structure, you want to choose the one that has the best behavior under your expected use.

Pushing something on to the head of an immutable list doesn't require a copy, the new list has the new object as the head and the old list (not a copy) as the tail. Similarly, removing the head doesn't require a copy, it just returns the tail of the existing list. If you want to replace something in the middle of the list, you'll need to copy everything closer to the head than the modified item.

Different immutable collections have different "cheap" access patterns.

>Let me introduce you to my friend, Mister ConcurrentModificationException. He's a wily fellow-- he rarely pops up when you're doing local development, but he has this nasty habit of showing up in production whenever you start hitting a decent load.

You do realize that concurrency and parallelization are very different topics right? One is a necessary property, the other is one of oppurtunity. It makes sense that collections designed for concurrency would suck at parallelism.

Here is the magic of Scala.

A collection designed for concurrency:

val users = List("seanmcdirmid", "mark242").reverse

That same collection, designed for parallelism and concurrency:

val users = List("seanmcdirmid", "mark242").par.reverse

The APIs are equivalent and safe, but parallelism is a feature of opportunity. The overhead of using an immutable collection is often not overcome by the use of parallel threads, making the tradeoff not worth it. Even CUDA has brought mutability back to GPUs for performance reasons.

> Let me introduce you to my friend, Mister ConcurrentModificationException.

Thanks for reminding me of him. I have never seen him during my last five years of programming in Java. I hope he's alive and well.

> Scala sits on top of Java

That's not true. Scala sits on the JVM, and has some accommodations to facilitate interoperation with code written in Java, but it does not "sit on top of Java" as a language. You don't need to know Java to write Scala.

Forgive my imprecision.

>You don't need to know Java to write Scala

Is this actually true in practice? Are there tutorials or books that don't assume knowledge of Java? In any case, if you want to use anything in the Java standard library (I assume they haven't converted the entire thing to Scala), then you'll still need to understand Java documentation which requires some knowledge of Java.

> Is this actually true in practice?

Seems to be.

> Are there tutorials or books that don't assume knowledge of Java?

Tutorials, books, and a Coursera course taught by Martin Odersky. Among other things.

> In any case, if you want to use anything in the Java standard library (I assume they haven't converted the entire thing to Scala), then you'll still need to understand Java documentation which requires some knowledge of Java

Sure, if you want to use something in a Java library (standard library or not) for which no one has written Scala-focussed documentation, its likely you'll need to understand Javadocs; that's not a "need to know to use Scala", that's a "need to know to effectively interface with existing Java library code", which applies just as much when leveraging Java libraries from any other language.

OTOH, the structure of Scala makes this fairly straightforward, and its probably a lower barrier if you know Scala but not Java than almost any other non-Java language from which you might call Java.

If you want to use Scala without ever calling or being called from Java, you probably picked the wrong FP language. If you want to use Scala to call and be called by Java, you will need to know Java to debug anything.

I am also posting to disagree. In general, I have been pretty rough on Scala, but the long and short of it is something like this: Scala trades being a usable FP language for being good at Java interop. Everything you could want to do in Java (except for enums) is possible and usually significantly easier and more succinct in Scala. Everything you would want to do in OCaml, Haskell, or Scheme, is possible and usually extraordinarily difficult in Scala.

> -- Scala advocates a completely different style of programming. You have to rethink how you code to match Scala's functional style. This is not a trivial thing if you have a bunch of Java developers you need to convert.

Imperative Scala is both performant and readable. For the application I work on, in many places it is the only style that is sufficiently performant.

> -- I find the Scala documentation is less readable than Java's docs. They are cluttered with type conversion functions and unreadable operator overloads (the filtering helps a bit here, but it's extra work). Documentation is also split between the object/class/trait pages.

This is pretty much true. Good IDE support and #scala@freenode help.

> -- There's a whole class of immutable data structures whose performance characteristics I need to learn, since I'm supposed to be using them.

You can use whatever data structures you want, including the ones in java.util. If immutable data structures are appropriate for your application (for instance, because you want nondeterminism based on Scala's delimited continuations, and repeated invocations of the same continuation should of course see the same data, or because you want to pass a lasting view of the current state of your collection to another thread), then you can use them.

> -- I'm not convinced that the overhead of copying immutable data doesn't outweigh the benefits of the ability to trivially parallelize the code.

This is an objection to a particular way of structuring applications, rather than to a language.

> -- I'm not convinced that a project preferring only immutable data is easier to maintain.

This is also not a Scala thing. My insignificant industry experience indicates that it is true. More importantly, John Carmack seems to think so: https://news.ycombinator.com/item?id=6278047

> -- Programs seem to take longer to compile (sbt is especially slow).

Yeah. I would be okay with this if I got the GLORIOUS TYPE SAFETY from it, but Scala does not really deliver on this front.

> then you need to know all of Scala's features as well.

This is a big burden, to be sure, and Scala introduces some unreasonable gotcha's along the way. Having already learned Scala, though, there's no reason I would sit down and write Java. Unless I wanted an enum.