Policy – A fork of the Scala compiler
github.com
github.com
Paul may have been hurting perception of Scala community by his rants a bit lately. But the fork is a greatly welcomed! All the things Paul said make sense. The fork can prove the value of those ideas.
Simpler compiler => more compiler developers => more productive => better compiler => better for the whole community!
As for Typesafe. There are doing great job. Just two releases ago Scala was not compatible even between minor releases. Stability, documentation, training, driving the community, good roadmap for 2.12, etc. Great job! Plus they are build a great framework around akka and play which is gaining traction.
Policy is actually not really a fork. Forks are, imho, projects which diverge code base one day and there's really no sharing between them anymore. Paul is merging from scalac all the time. It really is just vastly simplified scalac with some wise decisions like modularizing some aspects (like parallel collections).
BTW: Anybody worried about a friction while trying to push a bold change in a team of more than one person? Have you ever worked in a team of more than one? :-) Just do it. Prove it. Dismiss the FUD.
Fist questions on my mind: 1) Future? Is this repo intended to pull from scalac or diverge more? 2) Dropping reflection? Does it means removal of macros as well? For the time being or for good? 3) How about leaf node invariance in collections (and other collection stuff you talked about)?
I didn't say I'm dropping reflection, I said I'm dropping the scala-reflect layer. It's a gigantic and useless abstraction penalty. I'll fold the pseudo-compiler into the compiler.
We'll see about collections. Here's where it stands: https://github.com/paulp/psp-std
Some samples (some are quotes, some are paraphrased):
* "The compiler [should be] designed around inputs which are Diff[T]s against an in-memory understanding of the last successful compilation.... As a result, both Repl and Debugger are superfluous as separate tools." (emphasis mine)
* "I want to programmatically generate ASTs and feed them in"
* The AST (the tree) is fundamental, not the language. For example, if you have an AST you're working with, and you hate curly braces, theoretically you could see the same tree "rendered" as text without those curly braces.
* "True power is in restriction" & "unnecessary expressiveness is the enemy" -- whatever your language lets you "say", the "other side" (the compiler or interpreter) has to be able to understand. So unlimited language means horribly over-complicated compiler that cannot do nearly as much optimization as you'd like.
* The more free your code is of "over-precision", the more powerful that code is, because the compiler can do more with it. Example: the well known example of an imperative for-loop versus the functional "map".
* "If you want fast, start with comprehensible."
We did something like this before with scalac back in 2007. But they eventually decided to use lazy rather than incremental evaluation after I left the project (I admit the tech was too crazy and under developed for the time).
But the approach wound up working very well, check out:
http://research.microsoft.com/en-us/people/smcdirm/managedti...
I'd imagine it would work well for Scala today (it took about 6 months to retrofit scalac in 2007, and most of this was just polishing positions for memoization + getting rid of object identities). However, TypeSafe's current approach to reactive programming seems to revolve around Rx and actors, which isn't very useful in a compiler or live interpreter.
> The AST (the tree) is fundamental, not the language. For example, if you have an AST you're working with, and you hate curly braces, theoretically you could see the same tree "rendered" as text without those curly braces.
Ya, but all of this can be done in the IDE. Also, projectional editing isn't quite their yet; I prefer hacks like "Little Braces" for Visual Studio that simply shrink brace-only lines to very small font sizes :) Of course, I think syntax is "language", and having multiple languages flying around isn't great for community building.
Going by the experience with preprocessors in Ocaml, not the best idea IMO.
Excellent talk.
It sounds very much like what he is describing.
I think making Scala more correct and more modular is the answer. Hygienic macros and syntactic extensions are more what I had in mind. Rust seems to be a solid contender here but it's much more of a systems language.
https://news.ycombinator.com/item?id=8258255
https://news.ycombinator.com/item?id=8261896
_____________________________
Many (most?) of the debates regarding the scalac roadmap (and attendant tooling) happened in public on the internals mailing list. The simplest explanation is that nobody agreed on allocation of resources, which were mostly a few Typesafe employees and some (largely unknown but very talented) EPFL grad students. A sample thread https://groups.google.com/forum/#!msg/scala-internals/9Ts3GL...
Don't get me wrong. I know that it isn't necessarily the case that "basic" represents "affects plenty of people." But, especially when it was used in a "hits more people than a vocal group that is complaining about a plugin that needs attention," one does get a sense that you will be hit by these things sooner than later.
Java is at least as transparent as Scala in that respect: every single feature that was shipped as part of Java 8 has been discussed and decided openly on mailing-lists. Everyone knew exactly what was going to be in Java 8 (including what was not going to be shipped) which is a far cry from Scala's operating mode of adding features to the language and then retroactively creating SIPs to give them an air of legitimacy.
I'm siding squarely with Paul Phillips there: I don't have a lot of faith in Typesafe to carry Scala forward any more.
You are either completely deluded or never actually read the mailing list.
But hey, whatever floats your boat. I enjoy watching your desperateness. Really, if you keep posting your talking points in every remotely Scala-related topic, Scala will go away for sure. Really, I promise.
That is, my complaint here is that the platform is unstable. Thus making it scary to adopt. How is that "worse" everywhere else?
Stability is pretty good in Scala, too.
Maybe not "nothing changed since 1995"-Java, but better than C#, Python, Ruby, OCaml, Haskell, etc.
This is exactly the message that Paul Phillips has been spreading these past months.
Personally, I hope he decides to just create his own language, I would be extremely interested to follow this.
That is, how is this helping? If he gets a better compiler out, great. Are the odds high that will happen?
Especially with the history that the community has had with a few high profile individuals either constantly making waves and/or basically turning away from the community after some time... To say that the community seems toxic is an understatement.
Granted... this may convince folks that the truely toxic elements have jumped to other pastures. In some ways, this would be better for the typesafe landscape. That seems a stretch, though.
Oh, for sure. The key though is that the value of "it" which I am making worse is not a value of "it" which I have any interest in making better. Nor would I ever suggest otherwise.
In your mind I may be nothing but an input into your total utility function, but there's no reason for you to expect me to care about maximizing it.
Also, sadly, at the moment none of this is any real input to any utility I control. I have used Scala in the past. I tried setting my plans with Lift, but that didn't go over well with my coworkers. So, as of now, I am merely on the sidelines. And, do note that I am not trying to demand you not do this. Just saying from my vantage point, I'm not at all clear on what this is improving.
There's been drama and each time the community has moved on. Paul, Tony Morris and probably others from before I started paying any attention. Even with this recent "drama", after thinking about it for a few days, I've come to welcome it if anything. Regular Scala can shift its focus to becoming a conservative language while Typelevel acts as the incubator. If the two meet their compatibility goals, everyone wins.
I'd also argue that "constantly making waves" is an overstatement. Scala just celebrated its tenth birthday. How much drama has there really been in this community over ten years? For the most part, I've found it to be very supportive. My interactions are almost entirely just on the mailing lists but I've seen few issues. Maybe I'm just blind to them, who knows.
I recognize that tensions within the community are high but I don't currently believe that they present an insurmountable challenge to Scala's future.
You are absolutely right.
The usual Scala-hate-brigade will jump on this "smoking gun" and will build a few new talking points on top of it how Scala will fail definitely this time for sure ... but outside their bubble nobody cares and just moves on shipping software in Scala.
Stop attacking the people who don't like Scala (real or imaginary). Disagree, fine, but this rampant fanboyism is odd. Yes, the language has a few critics on HN whose critiques are mostly FUD. Argue against the FUD but when you attack them you only cheapen your own position.
I'm just trying to say that there's a way to be effective about this.
(I realize I'm attacking a person about attacking a person, I get it)
I find it kind of funny that we have like usually one username which gifts us with his presence in every Scala-related topic for usually ~6 months and then disappears.
Either it's the same guy who just changes accounts like here: http://www.reddit.com/r/scala/comments/1sdmdq/sockpuppet_acc... http://www.reddit.com/r/scala/comments/25kwb1/cedric_beust_c...; or it's a series of people all completely unaware that dozens of earlier crusades failed.
Both options are amusing. :-)
I guess the big waves I remember are essentially about a post a week from Tony. The hoopla around Colebourne's opinions on the language[1]. And then the mystery/whatever around Pollak's involvement in the community.[2]
The later point is odd just for how much traction lift seemed to be generating, and then the overall community appeared to reject it heavily. Do note that I would still recommend it. Their community is quite nice and helpful.
And I should definitely make clear that it is very possible I just don't see the waves in other communities. I don't really see Scala's anymore; this week being a notable exception. To me, though, high tension is not exactly a good environment. Heck, for all of the flack he gets, Torvalds is usually more cordial than many of the threads I remember from this community.
[1] http://blog.joda.org/2011/11/scala-feels-like-ejb-2-and-othe...
The short version is that he lost trust in the Typesafe team and he thinks they are not competent to carry Scala moving forward. Like he says, he tried to change things from the inside, he couldn't, so now he's trying from the outside.
The problem is on the other side of the Atlantic. Scala is unfixable now no matter the abilities of the people working on it because of the bad decisions which have been made to this point and the unwillingness to revisit them even when the failings were grossly apparent.
Organizations modeled on dictatorships can work fine but they're painfully vulnerable to the dictator's blind spots. As the years go by, the dictator becomes accustomed to being able to settle any question by fiat. He becomes less and less concerned with evaluating the true worth of his own ideas. As long as his power is unquestioned and there is a steady supply of people to do the work, why should he? He's the Decider. He decides.
As I told him at some point "That well does run dry eventually" and indeed it did, for me at least.
I'll be keeping a close eye on what you decide to do next, I have no doubt it will be extremely interesting.
This is somewhat like asking why Richard Dawkins just assumes the Theory of Evolution without justification in his tweets.
Googling Paul Phillips Scala brings up plenty of relevant discussion.
All the talk about scala makes me wonder what the language would have looked like if Java compatibility was never a goal. I'm no language designer and I'm probably wrong but some things that come to mind are 1) no nulls, 2) no distinction between classes and their companion objects, 3) no distinction between traits and abstract classes.
To your second paragraph -- I believe that #2 might have found its way into a Scala-from-scratch, but the others certainly wouldn't. I think the big difference would be in the type system, which could be much simpler if it didn't have to interop with Java's weird generics.
[0] http://www.slideshare.net/extempore/a-scala-corrections-libr...
I don't. You have inserted coercions.
I'm aware of how many bits things have. It's hard to make it through five years on a compiler without gaining some awareness of such details. I consider the introduction of lossy type conversions into the global scope to be a kind of insanity. People should be recoiling from such practices, not looking for ways to justify them.
I don't know much about Scala but I believe that most of its facilities are in the libraries rather than the language itself.
Since he seems to know more about compiling Scala than everyone else I guess it must be a good thing. Lets hope others get to fill the inevitable gaps.
They are the same thing. There is no spec for Scala (no recent one anyway) so the specification is the Scala compiler. And based on everything that Paul has been saying these past months, the compiler is very close to being unmaintainable.
If you're always hearing about how amazingly small scala's specification is, now you know why.
Too soon?
That's going to be a nightmare to Google
Almost as bad as 'play' (which has largely fallen back to 'playframework')
Go is a terrible, terrible name.
Also, I wonder what the two Scala forkers are thinking about Dotty (the next-gen Typesafe scala compiler) and how they imagine merging their ideas in.
trait ParIterableViewLike[+T,
+Coll <: Parallel,
+CollSeq,
+This <: ParIterableView[T, Coll, CollSeq] with ParIterableViewLike[T, Coll, CollSeq, This, ThisSeq],
+ThisSeq <: IterableView[T, CollSeq] with IterableViewLike[T, CollSeq, ThisSeq]]
extends GenIterableView[T, Coll]
with GenIterableViewLike[T, Coll, This]
with ParIterable[T]
with ParIterableLike[T, This, ThisSeq]You want everything to be a nice well understood package because its "easier that way?" But sometimes it isn't easy and the code isn't pretty; not because the programmer is horrible, but because the problems are hard and not very well understood. Does that mean we should shy away from the problems and just do something safe and conservative like C#? If everyone thought like this, we would make no progress.
Kudos to Martin for being fearless, and let's hope he doesn't cave.
Your second paragraph lands somewhere between completely wrong and "not even wrong" if it is supposed to have some bearing on the matter. You don't appear to know the first thing about what I've said, what I've done, or what is really going on here.
Reflexive loyalty of this kind, unhinged from any objective reality, is what has brought us today's scalac. You should reconsider whether your having worked on the scala 2.7 eclipse plugin nearly a decade ago qualifies you to take a position on this. Then to position martin as the scrappy underdog - "let's hope he doesn't cave" - just let us know when the novel is finished, and I hope it's better than the movie.
I didn't just work on the scala plugin, I changed scalac to be completely incremental at the AST level (better than everything being a Diff[T]), which was no easy feat (nor did it last, but it worked and became its own tech [1]). I've dug deeper into scalac front end code (Typers, namers, parsers) than most scalac devs have; I know Martin's style very well as a result.
[1] work on making scalac incremental eventually led to http://research.microsoft.com/en-us/people/smcdirm/managedti...
I'm just glad to be out of it. The Scala community really was becoming toxic, sleeping was hard. Hopefully you guys have all resolved your differences now and things are more harmonious.
I've ranted on the mailing list before, but it's overuse of inheritance rather than a type class based approach kills reasoning and modularity. Subtype polymorphic collections libraries are unquestionably a failed experiment.
The problem is that the std. lib collections tries to unify mutable/immutable and lazy/eager/finite/infinte datastructures into one class hierarchy. So you have methods like "size" or "add" on generic interfaces that mean different things depending on the implementation. So the entire abstraction ends up being leaky.
Scalaz has a few common implementations of immutable, eager data structures and ends up using a type class approach to modularity and code reuse that ends up not being leaky, it's a joy to use.
I don't think there's anything wrong with using the collections, just be aware that if you run into some edge cases, it's not the language, it's design choices made by the library writer and there may be better implementations.
Apart from that, I think collections can be improved considerably even while keeping dynamic dispatch.
Paul, if you read this: why don't you do something completely different? Leave Scala and your personal rage behind.
val color = List(
"Red",
"Yellow",
"Green",
)
If I want to add a 4th element I just ctrl+alt+down, I don't have to go back up and add the comma at the end of the last line, I spend too much time juggling commas in lists for my liking.Some programmers work around it like this (which breaks head-inserting, but that's typically a lot less common) :
List("Yellow"
,"Red"
,"Green"
)
Which is pretty ugly (to me). It's one of those minor QOL improvements where I don't see what the downside is.Frankly I wish every language had this feature.
That's a nice example why 99.9% of developers out there should never design a language.
Paul's fork is him exploring some ideas and not intended for widespread use. To quote the README:
"It's still in development, in every imaginable sense. I would prefer not to publish code in this condition ... However since typelevel has also forked the scala compiler, publishing now and giving them an opportunity to exploit my work seems the lesser evil."
Let's come here again in 6 months and see where we stand.