Hacker News Reader

>Let me introduce you to my friend, Mister ConcurrentModificationException. He's a wily fellow-- he rarely pops up when you're doing local development, but he has this nasty habit of showing up in production whenever you start hitting a decent load.

You do realize that concurrency and parallelization are very different topics right? One is a necessary property, the other is one of oppurtunity. It makes sense that collections designed for concurrency would suck at parallelism.

Here is the magic of Scala.

A collection designed for concurrency:

val users = List("seanmcdirmid", "mark242").reverse

That same collection, designed for parallelism and concurrency:

val users = List("seanmcdirmid", "mark242").par.reverse

The APIs are equivalent and safe, but parallelism is a feature of opportunity. The overhead of using an immutable collection is often not overcome by the use of parallel threads, making the tradeoff not worth it. Even CUDA has brought mutability back to GPUs for performance reasons.

singleton java.util.ArrayList

So...don't use data structures that are not thread-safe in threaded environments? Wrap it with Collections.synchronizedCollection. Or make copies, if you prefer.

For the very small price of a little bit of garbage...

What small price? If I have millions of things stored in an immutable data structure, do I not have to copy the entire structure (or at least some portion of it) when I want to modify anything inside of it?

It's actually all just pointers to data, so when you change something it makes a new node and copies a few new pointers so the new tree is still correct. All of the old data and pointers that aren't changed stay exactly where they are.

Ah, synchronizedCollection, the...

hang on...

wait a sec...

one more sec, someone's doing something...

now? no, not yet, wait...

how about... NOW! synchronizedCollection, the global lock for Java's mutable datatypes.

Here's the fundamental difference between Java and Scala: if I say, in Scala, val people = List[String]("quacker", "mark242") then by default people is an immutable list. I don't have to do anything special. In Java, either I'm suddenly using the Guava libraries to get something similiar (but not the same), or I'm doing all kinds of funky dances around list iterators, or I'm using Arrays.copyOf, or what have you. In any case, you have all of this extraneous code, when it isn't necessary.

Quick: give me a list comprehension method in Java that takes a list of Strings and returns that list, filtered, of Strings that are only five characters or longer. Make it null-safe and thread-safe. This isn't difficult-- you're thinking about 6-7 lines of Java code in your head, right? Null check, synchronized, new ArrayList, for(s in sList), that kind of thing, right?

In Scala, it's this. Some would argue you don't even need a separate method.

  def getFiveCharacters(s: List[String]) = s.filter(_.length >= 5)

If I have millions of things stored in an immutable data structure, do I not have to copy the entire structure when I want to modify anything within?

Dear god, man, what are you doing wrong in your Java code? This is the exact scenario where you want an immutable list, otherwise you'll be running synchronized code and precisely one core of your CPU will be glowing red like lava, while your other CPU cores sit idle.

I just did this in the scala repl:

  (0 to 1000000).toList.par.filter(_ % 100 == 0)

What that does is grab all of the integers from 0 to 1,000,000, convert them to an immutable List, then filter that List (yes-- an iterator! with a filtered copy!) by only taking numbers divisible by 100. The whole thing takes a couple hundred ms in the repl (which is fantastic, since it's compiling then executing the code) and almost no memory overhead. For code that is absolutely thread safe. And the ".par" makes this run in parallel on all my CPU cores.

> Null check, synchronized, new ArrayList, for(s in sList), that kind of thing, right?

> In Scala, it's this. Some would argue you don't even need a separate method.

It seems a bit unfair to force null checks on the Java code and then present a Scala alternative that throws NPE in two different places.

A good Scala programmer will never write the word null in their code. Ever. If I was overly concerned that the method was going to be called by Java code, I'd put this:

  Option(s).getOrElse(List[String]()).filter(str => Option(str).fold(false)(_.length > 5))

That should do it. I would probably go for

  def foo(s: List[String]): List[String] = s match { case l: List[String] => l.filter(_ match { case str: String => str.length > 5; case _ => false}); case _ => List[String]()}

It's a bit uglier, but I had to develop a habit of not spewing objects everywhere because of the constraints of the application I work on.

You're right that this is primarily a concern if the Scala dev wants to talk to Java. However, if the Scala dev does not want to talk to Java, perhaps a statically typed FP language with an HM type system would be a better fit.

I couldn't grok the original 1 liner, definitely not the second one and no way the last one. If there is a segment of programmers who grok and like it, cool. It still doesn't fix the performance problem of copying 1 million things before you can act on them because the copy would still have to happen on 1 core, or 1 core at a time, unless maybe the Scala compiler optimizes the code to divide the list among the cores available, if not though then immutability doesn't help performance during the copy.

You don't have to use immutable collections. If you want to use them, you should probably consider the cost of the operations you are using. Which particular operations are you worried about? Most updates to an immutable hash table or sorted map should copy only a small part of the collection, for instance.

> What small price? If I have millions of things stored in an immutable data structure, do I not have to copy the entire structure (or at least some portion of it) when I want to modify anything inside of it?

Depends on the nature of the modification; to choose the right immutable data structure, you want to choose the one that has the best behavior under your expected use.

Pushing something on to the head of an immutable list doesn't require a copy, the new list has the new object as the head and the old list (not a copy) as the tail. Similarly, removing the head doesn't require a copy, it just returns the tail of the existing list. If you want to replace something in the middle of the list, you'll need to copy everything closer to the head than the modified item.

Different immutable collections have different "cheap" access patterns.

> Let me introduce you to my friend, Mister ConcurrentModificationException.

Thanks for reminding me of him. I have never seen him during my last five years of programming in Java. I hope he's alive and well.