Hacker News Reader

Comment by tormeh | Hacker News Reader

Why are parallel collections an "albatross"? I can understand it if he plans to replace the functionality you get by writing "list.par.map(func).seq" with "list.parmap(func)" or something, though. Parallel list functions are one of my favorite features in Scala, and I think it would be a serious loss if they disappeared.

Also, I wonder what the two Scala forkers are thinking about Dotty (the next-gen Typesafe scala compiler) and how they imagine merging their ideas in.

View on HN

It's not a question of whether they should exist at all but of whether they should infiltrate the collections hierarchy to the extent that someone has to maintain code which looks like this. And there's plenty of it: this is only an example.

  trait ParIterableViewLike[+T,
                            +Coll <: Parallel,
                            +CollSeq,
                            +This <: ParIterableView[T, Coll, CollSeq] with ParIterableViewLike[T, Coll, CollSeq, This, ThisSeq],
                            +ThisSeq <: IterableView[T, CollSeq] with IterableViewLike[T, CollSeq, ThisSeq]]
  extends GenIterableView[T, Coll]
     with GenIterableViewLike[T, Coll, This]
     with ParIterable[T]
     with ParIterableLike[T, This, ThisSeq]

As a recovering former C++ programmer, that reminds me of the verbosity of C++ meta programming...

Which language have you moved to?

Well, that's a pretty good reason to fix something, I agree. But what are your thoughts about Dotty? I assume it's insufficient for you, but is Typesafe going in the right direction here, is it not relevant to what you are doing or will it make things harder?

Dotty is unlikely to attain relevance - for me or as a future version of scala - because there's no reason to expect it to be any less of an implementation nightmare than is scalac. It is being written by the same person, applying the same engineering standards and the same checks and balances, so starting with a less unsound type system isn't going to matter.

I worked on scalac for two years, so I know most of the same warts and issues; I also have had my differences with Martin in the past. But you know what: he has been courageously innovative and has come up with code that works pretty well considering the risks taken for Scala. Most people would have thought what Scala was trying to do was impossible (combining FP and OO with type inference? But semi-unification is undecidable!). But he did it, and you know what, scalac isn't that bad either. Heck, I've done much worse, but the point is to just keep trying.

You want everything to be a nice well understood package because its "easier that way?" But sometimes it isn't easy and the code isn't pretty; not because the programmer is horrible, but because the problems are hard and not very well understood. Does that mean we should shy away from the problems and just do something safe and conservative like C#? If everyone thought like this, we would make no progress.

Kudos to Martin for being fearless, and let's hope he doesn't cave.

That's right, I'm the stodgy conservative one and martin is the fearless innovator. Where is this version of the movie playing? I'd enjoy seeing it.

Your second paragraph lands somewhere between completely wrong and "not even wrong" if it is supposed to have some bearing on the matter. You don't appear to know the first thing about what I've said, what I've done, or what is really going on here.

Reflexive loyalty of this kind, unhinged from any objective reality, is what has brought us today's scalac. You should reconsider whether your having worked on the scala 2.7 eclipse plugin nearly a decade ago qualifies you to take a position on this. Then to position martin as the scrappy underdog - "let's hope he doesn't cave" - just let us know when the novel is finished, and I hope it's better than the movie.

You are right, I don't know you beyond these online interactions. I was giving you the benefit of the doubt, but you sound very vindictive, and I get it, you know stuff. Building and designing things is a journey, it won't be perfect unless you set your expectations low.

I didn't just work on the scala plugin, I changed scalac to be completely incremental at the AST level (better than everything being a Diff[T]), which was no easy feat (nor did it last, but it worked and became its own tech [1]). I've dug deeper into scalac front end code (Typers, namers, parsers) than most scalac devs have; I know Martin's style very well as a result.

[1] work on making scalac incremental eventually led to http://research.microsoft.com/en-us/people/smcdirm/managedti...

I am intimately familiar with your work on scalac, and that's all I can bring myself to say about that.

You and miles must be best friends. Its fine. Phew, really glad to be out of the Scala community at this point, it just wasn't worth it to being around those people.

Paul and I don't see eye to eye on many things, but on this we're in complete agreement.

So is life. Its great to see that you fixed the plugin and it works very well now. Congrats!

I'm just glad to be out of it. The Scala community really was becoming toxic, sleeping was hard. Hopefully you guys have all resolved your differences now and things are more harmonious.

The collections library itself is probably one of the worst things about scala.

I've ranted on the mailing list before, but it's overuse of inheritance rather than a type class based approach kills reasoning and modularity. Subtype polymorphic collections libraries are unquestionably a failed experiment.

Why do you feel that it is a failed experiment? As a user of the collections, it seems to me to work out. Is it more of a problem with the implementation of the collections?

Agreed. There is a vocal minority of people (academics mostly) yelling at how subtype polymorphism is broken and unusable. It certainly has broken aspects (with respect to equality) but by and large, it's extremely usable, intuitive and it enables powerful abstractions. The corner cases where it falls apart seem to be irrelevant for most uses.

I should have added that despite me not liking the std lib collections library, it's still stands above all other collection libs I've used in other languages.

The problem is that the std. lib collections tries to unify mutable/immutable and lazy/eager/finite/infinte datastructures into one class hierarchy. So you have methods like "size" or "add" on generic interfaces that mean different things depending on the implementation. So the entire abstraction ends up being leaky.

Scalaz has a few common implementations of immutable, eager data structures and ends up using a type class approach to modularity and code reuse that ends up not being leaky, it's a joy to use.

I don't think there's anything wrong with using the collections, just be aware that if you run into some edge cases, it's not the language, it's design choices made by the library writer and there may be better implementations.

I like dynamic dispatch. Having code which suddenly changes the implementation based on type ascriptions is just not what I want in this case.

Apart from that, I think collections can be improved considerably even while keeping dynamic dispatch.

Reply on news.ycombinator.com