> If the choice is between having to write null-checks or having to understand category theory to read my code
Me too. Luckily this is a false dichotomy.
Me too. Luckily this is a false dichotomy.
def :+[B >: A, That](elem: B)(implicit bf: CanBuildFrom[Repr, B, That]): That
This is from the standard collections library.
In Java, I've got a List.add function. In case it's not clear from the name, 'add', I can click through to the implementation and it's pretty obvious what's happening.
In Scala, I've got +, ++, +:, :+, and a bunch of other nonsensical bullshit, and when I click through to the implementation? Even less sense. Whenever I use a standard scala collection, I have no idea what it's actually doing. Additionally, everything favors allocation-happy overly-clever immutable wrappers rather than a simple ArrayList which will smoke those immutable implementations in real-world performance.
The cure is far worse than the disease, here.
That line alone drives home the point for me, you can read the rest of scala.collection.immutable if you need more convincing.
Also, idiomatic Scala code is perfectly readable. You have no right to complain about readability until you've dealt with spaghetti code written for Megacorp Inc, that completely ignores the fundamentals of structured programming. :)
Scala's collections have some quirks, but not as many as .NET's collections and you're actually comparing apples to oranges, because you won't find the equivalent of an "add" that returns a new collection instead of modifying the old one in .NET.
This is what happens when you pass judgement unto things you don't understand. Working with immutable data-structures is really, really awesome and Scala's API for these collections is very friendly and very type-safe - as in, if you feel the need to use `isInstanceOf` / `asInstanceOf`, then you're probably doing something wrong ;-)
And I really wish that C# would grow up a little in this regard, as modern programming languages need immutable collections as well, with a nice API to go along with it. And btw - working with Option is super awesome, no category theory needed.
That said, as I was saying in another comment, I'm really excited about this announcement, because this is mostly about the runtime, not the language. You can run things built with Scala on top of .NET right now by means of IKVM. And the JVM finally has some credible competition.
As far as not understanding how awesome immutable data structure are.. I'd take a step back before you make assumptions about what other people understand. Do you know anything about cache hierarchy and memory models on modern CPUs? Performance is important to some of us.
Yes.
> Performance is important to some of us.
... and in those cases, you don't have to use data structures that model your problem domain poorly.
The problem isn't "immutable data structures" or "theoretically robust" languages. The problem is finding ways to express computations and their constraints in a way that can be efficiently modeled for your problem domain.
So what will probably happen is that once you see your performance problems, you can pray that it's only some hot spots that you can then replace with faster code - often it's not, so you have 100 places using around 1% of your time budget for example, which is when you can go and start over.
Yes I do. Worked 3 years on a soft real-time system with massive load, profiled the shit out of everything. There's an interesting discussion we could have about when immutable data-structures work best, when they've got problems and when it doesn't matter, especially given the extra benefits in dealing with accidental complexity. This isn't the right place though.
> Performance is important to some of us
Yes it is, but performance problems are fixed by means of profiling and optimizing the bottlenecks. Even in a system that has massive load, in many cases in doesn't matter and in some cases immutability increases performance by eliminating contention on reads. And seriously, most people invoking performance problems are not having those performance problems to begin with, therefore my assumption.
As I'm sure you know, when it comes to real-world situations, reducing inter-thread communication and isolating anything mutable is the key concern. That's why I find immutable wrappers that mock mutability to be a bit of a sideshow. You shouldn't have read contention with locking in the first place unless it's for a very good reason.
True, but you know how it is in practice :-)
For example I found that using persistent data-structures work best when you've got single producer, multiple consumers scenarios - so you mutate some state and you want to signal it over asynchronous boundaries to multiple consumers. With an immutable data-structure you just signal it, worry free and then you can keep on changing that state, completely non-blocking / wait-free and with good algorithmic complexity.
Actually non-blocking logic becomes really easy, as you can always shove an immutable value into an atomic reference (note - I'm not saying "wait free", which still takes a lot of work :))
So really, persistent data-structures are great in a multi-threading context, as long as you don't have multiple producers pounding on the same reference holding such an immutable value - if you do that, things can get bad, when compared to specialized concurrent mutable data-structures - because a good concurrent data-structure is able to distribute the contention in multiple buckets instead of just one. But then again, having multiple producers pounding on the same resource is just asking for trouble and has to be avoided, because Amdahl's law.
Also, as you've hinted at, the problem with a normal linked List is the level of indirection. And in general, persistent data-structures imply the usage of trees, which also implies indirections. More advanced persistent data-structures are much better than the linked list is and this is an active area of research, but on the whole there's still much room for improvement.
On the other hand, in my opinion when speaking about performance, the first problem one has is to actually use the available CPUs (e.g. getting CPU usage over, say 70-80%). Which usually is hard to achieve if you have a combination of CPU-bound and I/O-bound tasks and your I/O stuff is not asynchronous. Only after that you can then move on to optimizing the memory access patterns for cache locality and for minimizing the stop-the-world freezes.
Speaking about GC, that's another topic - persistent data-structures have a tendency to generate junk that is neither short term or long term and that invalidates the assumptions that current GCs are making. The JVM at least has really good GCs, but without paying for a pauseless one (like that one from Azul Systems), you can still end up into trouble if you don't pay attention - but then you fire up YourKit's Profiler, find the source for those STWs, optimize and it works out well.
All in all I encourage everybody to find a good library that implements persistent data-structures and integrate them in their toolbox.
The one thing I disagree with, in many server applications, is using the available CPUs is pretty easy. You've got thread pools handling various tasks, just crank them up. In JVM-land, a very heavy 512kb stack per thread is still not really much penalty to pay as long as you re-use them. Aggregate application performance then becomes a matter of completing tasks faster while creating less garbage.
So it all comes down to what you consider a 'task' and how you handle the handoffs between them. The architecture decisions at this level dwarf the improvements from using an array vs list, as you implied, but they also make the usage of immutable types somewhat irrelevant IMO. Seal off mutable code within single-task boundaries and it doesn't matter how ugly it is, as long as you're passing immutable types (just plain javabeans with final members are fine) between boundaries.
Anyways, just my opinion. Great comment.
Why the atomic reference here? I know that provides CAS but if we're talking about a single writer aren't you okay to just replace things anyway?
Thanks!
Gotta test for any chinks in the armour.
http://msdn.microsoft.com/en-us/library/dn385366(v=vs.110).a...
Admittedly not part of the core library, but installing a NuGet package is pretty darn easy.
>>This is what happens when you pass judgement unto things you don't understand ;)
[edit: Guess I should've refreshed the page to see the prior response before writing this. Apologies.]
I think it's kind of the point if the judgement is on the question "what is easier to understand". I think this is the sentiment many people staring to learn Scala are feeling - the barrier to entry, even if you are coming not from the blank slate but from the background of programming many years in many languages, is pretty high. It's not the judgement on "whether Scala and its collections are good/done right", which is entirely separate question from "whether it is easier for someone to understand how C# collections or Scala collections work".
>>> And btw - working with Option is super awesome, no category theory needed.
Well, if you want to do something like making a function that works on Option from a function that works on the underlying type, you pretty soon find yourself in that general area.
You're rarely actually going to _see_ that signature (the docs actually simplify it for you), and in practice, it's completely meaningless to 99.9% of Scala you'll ever see.
You might as well consider it pixie-dust.
+ for one, ++ to add a collection, the others are generally going to apply to Cons-like. It takes all of a couple minutes to let these sink in.
Use a mutable.MutableList if that's what you want. Or just use ArrayList.
All this is about as pure FUD as I've ever seen...
Your typical Java developer is used to just look inside the code of whatever library they are using, and find a very straightforward implementation. Scala collections avoid a lot of boilerplate with canBuildFrom, SeqLike and suck, but simple and straightforward they are not. It takes quite a while before it stops reading like Japanese.
And IMO yes, a lot of symbolic methods in collections make relatively little sense. Don't forget that list also has ::, :::, +:, :+ and :\. There's more than a few, and there are no textual versions of them, for those that don't have them all memorized. They are a bit of a relic from the time Martin thought that /: was a good idea. It's fortunate that now only the scalaz people keep doing such things, because excessive use of symbolic operations hurt language adoption.
And he doesn't even get into other early confusion points, like how we have =>, <-, and ->, or how decomposing Seqs is not exactly pretty. Last week I had to help a guy that had been using Scala for 6 months to understand the 'punched in the face' operator :_*
So no, it's definitely not FUD. Are they issues that hurt my day to day Scala use? Not at all. Scala is my favorite language. Being able to use it instead of Java or Clojure is worth a good 15K a year for me.But that doesn't mean that I have forgotten some of the little things that made the learning curve tough at first. Thanks the heavens that I managed to end up finding a Scala job where I could learn from one Bill Venners.
Everybody decried Ruby's 107 methods on Array. Then Fowler came out with "fluent-interfaces", and how often do you see someone make the claims that Ruby's Array is indicative of a general badness because it has a lot of methods and you can't memorize them all in an hour as a newbie?
scala.collection is the same deal.
:\? Sure it's not a good idea. I wouldn't debate that. Who uses that? foldLeft/foldRight.
And what's the deal with trying to memorize the entire interface anyways? 8 or so years with Ruby, writing libraries with over 4 million downloads (https://rubygems.org/profiles/ssmoot), and there are definitely methods in Ruby's Array I'm unfamiliar with.
So what?
While :: and ::: look a little foreign, I don't think asking people to learn them if they want to work with Lists in a Functional manner is anymore difficult than learning what the "spaceship operator" does in Ruby. And it's optional. You don't have to use them. But they're usage will probably be the smaller part in the grand scheme of things. Pattern Matching and accomplishing functional recursion with immutable data is the bigger picture. Outside of that context (and the REPL I guess, for convenience) you're just not going to see either of those operators very often (IME).
You don't use a "splat" (aka 'punched in the face'? That's a new one to me) operator very often. It's actually one of the few semi-pattern-matchy areas of Ruby so it comes pretty naturally for me.
You don't become a pro overnight. You can trust I'd be teaching infrequently used idioms in Ruby to a developer who'd only been using it for six months. Been there, done that. ;-)
I guess where I'm coming from is using CBF to claim Scala is a confusing, indecipherable language. Everyone's had that argument. I don't know that anyone wants to stand up and claim it's the best. But it works. Actually using it is a non-issue since you don't actually explicitly use it. And look at the docs. They're actually pretty great overall.
It just bothers me I suppose that someone interested in exploring Scala would be dissuaded by something that's only ever been a problem for 1% of 1% of Scala developers.
If you're just looking to swap in Play to replace Rails, odds are you'll never run into any of these CBF "concerns". At all. That's the very definition of "FUD" IMO.
For every CBF in Scala there's a Calendar in Java. Languages aren't perfect. CBF is probably a wart. But it's a hidden one. If you let it scare you off from Scala that's sad, because it really has about as much to do with day to day Scala development as array.c (https://github.com/ruby/ruby/blob/trunk/array.c) has to do with Ruby development.