Scala Native
github.com
github.com
So it is either "Scala like LISP"
or
It is a superset of the Scala language. Meaning all existing stuff will work with it, but if you use new secret keywords it will work "better".
Which seems not quite ideal, because that means most Scala libraries will not be "tuned", and you will need all new libraries.. just like ScalaJS made you need entirely new Scala libraries that did not use reflection.
Worried the Scala ecosystem is turning into a nightmare ;)
Most Scala libraries never used reflection, so most of the ecosystem "just worked" on Scala.js.
I think quite a few things in scala-native are there to show the possibilities of this platform, but in the mid- to long-term those improvements will be supported everywhere:
- @struct and AnyVals: As soon as AnyVals can support more than one value on the JVM (as soon as Oracle finally gets some things done) @struct can go away, because it's equivalent to AnyVal.
- @extern and @native could also end up being the same.
- @inline and @noinline are already supported across platforms.
Do you have personal experience with this? I do not have hard numbers, but for me - basically none of my stack worked.
I had to find a new JSON parser. I had to find new validation for my form stuff. Etc. etc.
It wasn't impossible, but it was a lot of work.
Had some cool parts to it, but not sure I was sold on the overall experience. I hate JS so bad and want to avoid it, but I kind of felt like I was mostly trading evils. Maybe if I worked at it enough it would finally get better? Not sure
I ported a site written in vanilla Javascript. I decided to faithfully port it without any changes before introducing scala specific libraries. That was pretty painful. A bunch of manual casting for UndefOr and js.Function.
Since then I've started using some of the scalajs libraries/frameworks straight away on projects. It has been a much more pleasant experience!
If they just use reflection during compilation: why would that not be compatible with whatever the runtime is? Shouldn't that be unaffected by runtime and purely depend on the compiler's support for reflect?
Or am I looking at this the wrong way?
Macros should always work, reflection is harder as some information needs to be retained at runtime (either via Java reflection or additional data – which can both be problematic in Scala.js).
You have to specify the symbols to preserve explicitly.
Seems trivial with this small example but with large objects it keeps your code much more maintainable and consistent since reading and writing from that case class is the same.
As far as it not working in Scala.js, from my understanding it has to do with how the reflection library is shared between both runtime and compile time implementations.
You can read more about it here: http://docs.scala-lang.org/overviews/reflection/overview.htm...
No, Play JSON doesn't work because it wraps Jackson, a Java library.
Jackson works using reflection. Play JSON therefore at least depends on reflection transatively.
But reflection isn't usually an issue for Scala as Scala code does most of the same tricks at compile-time. Scala libraries that have problems are those libraries dealing with multi-threading. Do you block threads anywhere, waiting for a Future, or on a CountDownLatch? Any await/notify anywhere? Sorry, that won't work on top of Scala.js. That doesn't mean that you can't work around it though and it does take effort on the part of library authors. My own library (sorry for the shameless plug :)) is completely cross-platform: https://monix.io
BTW, Scala.js is very new, but the whole ecosystem wants to support it, every major library is being ported if not ported already and everybody is talking about it ;-)
For me, syntax--good syntax, anyway--makes code far more readable.
It is always a steep curve to dive into other developers code.
I had a very good CS degree in the mid-90's.
We got to use Pascal, C, C++, Prolog, Caml Light, Smalltalk, Oberon(-2), Component Pascal, Lisp, SQL, PL/SQL, x86 ASM, MIPS ASM, Java, across the 5 years it used to take (nowadays thanks to Bologna is no longer the case).
It doesn't mean we got to use many of those languages afterwards.
But Lisp is a special case, if Lisp Workstations hadn't failed in the market or if Sun and others hadn't picked UNIX as their Workstation OS, maybe it would be different IT world, in spite of all DSLs I was referring to.
Having worked with both approaches, there's a time and place for both. Being able to quickly dive into a codebase is not always a good thing. One thing conservative enterprise developers should like about DSLs is that they require new developers to structure their code in a way that fits the intended domain model. I've seen my fair share of code written by developers who "quickly dove in" and they almost never fit the model and end up causing a huge mess which may be unrecoverable. Conversely, if you don't trust your developers to write decent code, trusting them inside a dynamically-typed DSL is probably not a good idea either.
Apart from the base language we'll also introduce some language/library extensions to make lower-level programming that is going to be limited Scala Native "dialect" of Scala. E.g. pointers, structs, stack allocation, extern objects etc.
I could have imagined that it would have been more compatible if Scala-JVM and Scala-Native used the same annotations (using JNI behind the scenes on the JVM).
If you can compile and run C code with Clang on some platform, it's reasonable to expect Scala Native to be run there eventually as we're based on the the same underlying compiler toolchain. Currently we mostly aim at POSIX environments.
Excited about this, keep up the great work!
In a first release we're going to use Boehm GC. That's just a stepping stone, not a long-term strategy. Stay tuned for news on this front.
> How did you get rid of JVM and JRE class library dependencies of Scala?
We're reimplementing / porting from Apache Harmony all the things we need.
> What's the debugging story?
We're going to integrate with lldb.
> Is this in any way related to vmkit?
Not related in any way, apart from being based on LLVM.
Doesn't that create licensing issues? You already seem to have code from Harmony in the project, but the project's license is neither Apache nor does it mention it.
Additionally, won't that cause trouble with Oracle, considering the lawsuit against Google?
Would GCJ be any use? They had native compilation over a decade ago.
(Although developers moved on to IcedTea, iirc)
(I can't recall for sure but I believe I had this conversation with Martin in a taxi ride not too long ago :)
Will probably have more questions after I browse through the code but some basics: Q: How much of the collections library is covered with full support? Partial support? NYI w/ ETA? Q: Cross-platform support? If I wanted to run this on iOS and Android today, what do I have to do? What about on Windows or OSX? Q: How does GC work? Is there a way to take more control over GC for Scala objects? Q: Is it possible to ‘link’ in external dependencies, i.e. Joda Time. How do I do this right now? Q: Does the compiler use a translation layer? If so, is it consistent with Java 7 or 8?
On another note, the c extern stuff is excellent — exactly what I’d like to see for portability and performance (without the headache of something like the JNI).
Non-parallel collections should work in first developer preview (to be announced.) Parallel collections may took a few release to get working.
> Q: Cross-platform support? If I wanted to run this on iOS and Android today, what do I have to do? What about on Windows or OSX?
If you can compile and run C code with Clang on some platform, it's reasonable to expect Scala Native to be run there eventually as we're based on the the same underlying compiler toolchain. Currently we mostly aim at POSIX environments.
> Q: How does GC work? Is there a way to take more control over GC for Scala objects?
Boehm GC for now, but that's not our long-term strategy.
> Q: Is it possible to ‘link’ in external dependencies, i.e. Joda Time. How do I do this right now?
We can link with libraries via C ABI. Java libraries need to be ported over to Scala.
> Q: Does the compiler use a translation layer? If so, is it consistent with Java 7 or 8?
We strive to keep the same semantics as Scala/JVM.
Bare in mind that scalajs has the same issue and a lot of work has already been done!
I have been so extraordinarily impressed with Scala, and with the contributions its made in the PLT field, I'm very excited for Scala Native to start picking up steam.
Can we expect the same level of excellence from Scala Native as we've received from Scala for JVM?
The reason I ask is that I see on GitHub that you are the only contributor so far. This project is quite a large commitment for a single person, I'm hesitant to play around with Scala Native without knowing more about future plans for support, milestones, etc.
Having said that, this project looks awesome. Nice work so far!
That's very common for academic projects.
It died for technical reasons because .net supports reified generics, which made it pretty much unable to support Scala's type system.
People who keep saying that erased generics is a stupid idea have no idea what they're talking about.
scala-native is facing insurmountable difficulties, the kind that will certainly not be conquered by a single student who will stop working on it as soon as he graduates.
No, the reason for why Scala.Net didn't happen is because nobody cared. To find proof of this, you only need to look at Clojure. Its .NET implementation is well maintained by David Miller, yet it's very unpopular. And the reason for why .NET developers don't care is because they don't have an open source culture. Or in other words, if it doesn't come from Microsoft, then it doesn't exist.
So, in a sense, no on cared -- because neither of the available options was anything anyone wanted.
I'm interested in it breaking away from the JVM to become more performant, since that has been one of my concerns about Scala; there is a lot of good about JVM-based languages, but when in the ring with other languages like Go, you need to be quick to win.
I hope that others join him, but there's no reason to believe he can't do it if he sets his mind to it.
The author of this project is most definitely a smart and capable individual. However, at some point this project will reach a point where it requires more work than one individual can achieve, hence the question regarding official support - is there any financing, plans for new contributors, etc.
For example, here is the story of how Python got its start from Guido van Rossum (quote from 1996):
"Over six years ago, in December 1989, I was looking for a "hobby" programming project that would keep me occupied during the week around Christmas. My office ... would be closed, but I had a home computer, and not much else on my hands. I decided to write an interpreter for the new scripting language I had been thinking about lately: a descendant of ABC that would appeal to Unix/C hackers. I chose Python as a working title for the project, being in a slightly irreverent mood (and a big fan of Monty Python's Flying Circus)."
And the history of how Ruby got its start by Yukihiro Matsumoto (quote from 1999) is similar:
"I was talking with my colleague about the possibility of an object-oriented scripting language. I knew Perl (Perl4, not Perl5), but I didn't like it really, because it had the smell of a toy language (it still has). The object-oriented language seemed very promising. I knew Python then. But I didn't like it, because I didn't think it was a true object-oriented language — OO features appeared to be add-on to the language. As a language maniac and OO fan for 15 years, I really wanted a genuine object-oriented, easy-to-use scripting language. I looked for but couldn't find one. So I decided to make it."
Denys Shabalin is working under the author of Scala and corresponding regularly with the author of Scala.js, so he has more support than either van Rossum or Matsumoto did, and neither of them had the plans you spoke of- they just wrote the language because it is what they wanted to do.
EDIT: Thank you!
https://github.com/densh/talks/blob/517b20c30dd4aaf390785039...
Will scala native generate some c-linkable files?
Scala Native generates LLVM IR that can be compiled to C-linkable code but for now we focus on "one statically linked application at a time" use case.
* Any plans to strengthen the Scala core lib so one doesn't have to reach out to JVM for mundane tasks like I/O or threading?
* Is there a simple way to use raw C / C++ libraries from Scala native?
* Is there a http://search.maven.org/ equivalent?
2. @extern objects are an easy way to call C code. There is no C++ interop at the moment.
3. Not yet.
I heard some rumors that someone (you) was doing it as a graduation thesis under Odersky, is that true or complete bollocks?
def erased[T](xs: List[T]) : String = {
xs match {
case ns : List[Int] =>"Natural numbers"
case fs : List[Double] => "Floating point numbers"
case _ : List[T] => "Something else"
}
}
System.out.println(erased(List(1,2,3)))
System.out.println(erased(List(1.0,2.0,3.0)))
case ns : List[Int] =>
val listInt = listAny.collect { case x:Int => x }
val listDouble = listAny.collect { case x:Double => x }
The only way this would make sense would be if you are talking about a List[Any] deserialized from somewhere (like some shitty JSON library). But then in Scala we don't really work with Any, that should never happen and if you see libraries that use Any in their API, drop them like they are hot ;-)
> Which is purely because of a JVM runtime limitation
No it's not. Talk to compiler authors. This is actually a freedom, because by introducing reified generics in the runtime, then language designers either have to deal with inefficiencies due to boxing because of extra conversions, or to limit their type system, so for example you can say goodbye to higher-kinded types. A runtime that has reified generics is not a multi-language runtime. In fact many languages are doing type erasure. Haskell is doing type erasure. And why shouldn't it, I mean, type casting and isInstanceOf checks make no sense in Haskell.
Will you be using native libraries/the data blobs installed on the host system? Or will you ship these things with each binary?
What is the rationale for doing this, given that the Graal / JVM guys are already developing an AOT compiler which can compile Scala to native binaries?
Is this motivated by performance? The desire to write C libraries in Scala? What?
Given the high level, dynamic nature of Scala I am skeptical LLVM+Boehm GC will result in faster code. I expect it would result in slower code.
EDIT: OK, I just saw the link to your talk below. I feel you should know a few things:
1) The Hotspot team are adding AOT compilation to the JVM. There is a talk on it here. This mostly eliminates warmup time:
https://www.youtube.com/watch?v=Xybzyv8qbOc&list=PLX8CzqL3Ar...
You can't eliminate warmup time entirely because profile guided on-the-fly optimisation is actually quite powerful and an AOT compiler that doesn't have profile data to work with can't generate code that's as good. But they support tiered AOT, where the AOT compiled code does basic profiling of itself, and then it can still trigger profile-guided JIT compilation if you want to.
2) Interop with native is being heavily worked on in project Panama: another JVM upgrade project. With the current prototype you can feed it arbitrary header files that are parsed with clang, and it generates Java interfacing files that are efficiently compiled (no overhead). There's a Pointer<T> type and so on.
3) Stack allocation isn't so easy to do unless you are willing to pretty wildly violate the safety of the language. It's also been found to not help much when very adventurous JVM implementors tried it anyway (Azul), because a good generational GC makes short lived allocations so cheap. Valhalla is doing value types on the JVM but that's about more than just stack allocation.
4) You can do manual memory management on the JVM already, via the Unsafe class. I'm sure a Scala DSL would make it have a more convenient syntax.
5) Java 9 will have a static linker equivalent that generates stripped, standalone, optimised application packages that don't depend on any separate JVM. The AOT compiler will be a plugin to this static linker.
So I don't mean to discourage you if this is a fun research project, but you should be aware that the daydream is not one that the actual Java team are ignoring for mysterious reasons. They're working on it too, and are better funded and taking a more general and compatible approach. By the time you have a tool that's usefully complete it'll take years and by then there might not be much difference between the approaches.
It's actually a pretty good question. A lot of Scala's design is full of compromises made for Java inter-op. A huge motivation for using Scala is "I want a modern language which allows me to use all these existing Java libraries and tools". If you remove the ecosystem and the inter-op, what is there left for Scala? Why not use a better language?
Can you provide an example of what you think is a better language?
Scala gets a bad rap precisely because people are big fans of their own way of writing code, call it better, and think that anyone that wants something else is deluded. The Scala way is to say 'sure, you can do that too, to hell with purity'. I'd rather have power over purity any day.
That said, in my opinion Scala's blunders are many. You can find out about many of them from Paul Philips, ex committer of Scala, but if he sounds too bitter to you (he does to me; at this point he sounds like there is bad blood between him and Odersky), here are some of mine:
- Null. Null will come back to bite you in Scala code, whenever you call Java libraries, and the occasional Scala library that didn't get the memo. Sure, you can wrap every suspect value with Option(...), but why should you do this?
- Type inference is not as powerful as in a language with a cleaner type system such as Haskell. I'm told this is because Scala tries to be both OOP (inheritance) and FP at the same time. Here is a clear case of Scala's attempt at being a "jack of all trades" resulting in it being inferior than the sum of its parts.
- Type signatures in Scala collections and core types are extremely hard to read. You do not need to read them, but if you try, you are in for a world of hurt. This in itself is not a mortal sin, but of course if you're willing to forgive Scala this much complexity, surely you're willing to forgive other languages as well?
- Tooling is bad. It's getting better, but it's still bad. SBT is painful to use. As for IDEs, IntelliJ is now the recommended choice; I've tried both Scala IDE and IntelliJ, and they both suck. Slow, spurious compilation errors, uncomfortable to use.
- And finally, telling bit of personal experience: I do NOT use Haskell professionally, yet whenever I want to try a quick idea and see if it typechecks, I find it easier to open ghci (the Haskell REPL) and try my idea than do the same with the Scala REPL. What does this say about Scala?
Scala devs deprecate and remove things that haven't worked out well. They have an established track record of making these migrations easier with each release.
> Null
Upcoming versions will make references non-nullable by default. If you want things to be nullable, you will have to opt-in explicitly with T|Null.
> - Type inference
Type inference is not a huge priority, because people think that it's a good thing that declarations require type annotations (just like it's recommended in Haskell, too). One of the last pain-points, higher-kinded unification of type constructors, has been fixed recently which will make type inference "just work" in exactly those cases where the lack of type inference was most annoying.
> - Tooling
I think tooling is pretty amazing. Sure, Java has still some advantages in some places. But the tooling is superior to almost all other languages out there. SBT is amazing. IDE support is getting vastly better, not only have Eclipse and IntelliJ improved vastly, but also IDE support for Emacs, Vim, Sublime, etc. is coming along at a rapid rate, and it's just amazing to use. There are so many tools in the Scala ecosystem which just don't exist in this combination anywhere else.
I can see how the opt-in null references might help prevent you from writing Scala code that uses null, but how does it help when interacting with Java code?
Regarding your question about nulls and Java:
I think that there is not much Scala can do here. All existing evidence from other languages that tried this shows that there is a large semantic gap between "nullable values" and "values of unknown nullability".
As Scala is much more independent of Java it is a much smaller issue, but improvements with Java interop require action from Java library authors first.
The approach of supplying external nullability meta data has largely been a failure, because
a) authors are not aware of the stated, external constraints, so they could break these assumptions in future releases without even knowing
b) it's really really hard to retrofit nullability guarantees into libraries which have been designed without this requirement in mind
c) the existing ecosystem is not very amenable to these guarantees, as nullability metadata would be largely limited to final classes and members, because everything else could be subclasses or overridden in third-party code, breaking these assumptions
As soon as Java library authors themselves start using @Nullable/@NonNullable annotations everywhere, there is a good opportunity of reading these annotations and typing the values accordingly, but without it, benefits are very slim.
The planned changes are valuable for dealing with nullable types, but as mentioned "unknown nullability" needs a different solution, and I think it's currently not worth adding something to the languages as long as there is still hope for widespread adoption of nullability annotations.
- Being a "jack of all trades" means Scala has the superior module system. In Scala you can have abstract modules, the way you have in Ocaml. In Scala type-class instances are lexically scoped, whereas in Haskell they are global. Haskell's type-classes are anti-modular, which is why there are people avoiding type-classes. A big part of what makes Scala so good is OOP. Haskell needs extensions to achieve similar functionality in a half-baked way and modularity is the main complaint of people coming to Haskell from Ocaml.
Btw, if you ask a C++ developer to describe OOP, he'll say it's the ability of having structs with a VTable attached. Most people think OOP is about state. That's not true, OOP is about single (or multi) dispatch and subtyping (having a vtable), hence it's not at odds with FP, unless you want it to be ;-)
- SBT is amongst the best build tools ever available. I could rant all day about the clusterfuck of Javascript (npm, Bower, Grunt, Gulp, Brunch.io, etc.) or Python (easy_install, setuptools, virtualenv) or .NET (MSBuild, Nuget, dotnet) or Haskell (cabal). For all its quirks, SBT is by far the sanest dependency and build management tool I've worked with. In fact, amongst the best reasons for preferring Scala.js is being able to work with SBT and avoid Javascript's clusterfuck completely.
- I've been working with IntelliJ IDEA with the Scala plugin for the last 3 years. I've had some problems with it, but all minor and it's amongst the best IDEs available, giving you everything you expect out of an IDE. Other platforms either don't have an IDE (Haskell), or require extra commercial plugins to behave like an actual IDE (Visual Studio).
And let me give an example: in IntelliJ IDEA I can click on any identifier in any third-party dependency and it automatically downloads and shows me the source code and I can set breakpoints and debug any third-party dependencies that way. Such a simple thing, yet it's a tough act to beat. Try it out in Visual Studio sometimes.
Yes, and I'm speaking in practice! If your language allows nulls, they will be used. I've experienced plenty of NPEs in Scala code, both from Java libraries and from misbehaving Scala code, to know this is real. Lucky you if you haven't experienced them! (As an aside: avoiding NPE is almost never a matter of simply "reading the docs". Many times there aren't docs at all, and even when there are, nulls are seldom documented).
> Being a "jack of all trades" means Scala has the superior module system
Your comparison with Haskell modules is fair. But that's just one aspect. Being a jack of all trades means Scala has poorer type inference, way worse type signatures, and generally it feels less clean than both a purer OOP language and a FP one. Scala does fine in all fronts, but not great. If you want to do FP, there are far better languages. I assume it's the same with OOP.
> SBT is amongst the best build tools ever available
If true, that's... unfortunate. SBT is uncomfortable and bizarre. Before you mention it: Maven looks likewise bizarre to me. These are tools to suffer with resignation, not to celebrate. Talking about Haskell, I'm trying to learn stack, which some say makes cabal more bearable. Are you familiar with it?
I agree IntelliJ is now an acceptable IDE for Scala. It's still ages from the comfort of using Eclipse with Java (at least what Eclipse used to be, not the unbearable beast it is now), and to be honest, IntelliJ only recently became usable. A couple of years ago (definitely less than 3 years), the IDE choked on Scala code and highlighted compilation errors left and right where there were none -- and I'm talking about vanilla Scala code, nothing advanced.
- C# with VisualStudio + JetBrains addon
- Java with IntelliJ
Why not the others you mentioned?
- IDE support for F# is not very good.
- VB is too dynamically typed to be reliable.
- C++ IDEs seem to be constantly fighting with constructs that manage to break the IDE's understanding of the code. It's gotten better with LLVM, but even VisualStudio is far away from providing a reliable experience.
- JavaScript is dynamically typed, so IDE support is not reliable.
- F# does lack some Microsoft love, but Visual F# Power Tools does improve the experience a lot
- Netbeans and Visual Studio 2015 are quite good in JavaScript support, specially on code where JsDoc comments are available
- It's unclear whether SBT wants you to write build.sbt or build.scala project files. It's recommended you use build.sbt by default, but I've seen plenty of "simple" projects, sometimes examples but often templates generated by tools, which default to build.scala. And now I've read build.scala is deprecated...
- The syntax of build.sbt is confusing. Is it Scala? Is it a DSL? Knowing Scala is certainly not enough to understand SBT (even for build.scala files!), but if I remember correctly one of its selling points was that it was "just Scala".
- Let's get not started about build files for build files...
- The structure of the file is confusing. I never know when defining a key is going to work or not. There seem to be tons of ways of doing something, which is fine in a general-purpose language but undesirable in a configuration format!
- The many third-party plugins are confusing to configure and are often poorly documented, which reminds me of Maven's discouraging "plugin hell"... I don't think I've ever seen a plugin for Maven which was adequately documented, which means you end up copying someone else's configuration without fully understanding it. I fear SBT is the same.
- Neither IntelliJ nor Scala IDE are entirely happy compiling SBT's project files. Thankfully the support has gotten better, but it's not 100% there yet. This used to be maddening in the recent past; a "compiled" project file which your IDE fails to properly understand is unusable!
This also essentially describes C++. This is also why both languages can be so terrible to use: every library has its own dialect.
Are you talking about compromises in the language or the standard library? In the language itself I can only think of type erasure for generics having to follow the JVM choice.
Have a look at the adoption of Scala.js. The ecosystem of Scala.js is larger than the "compile-to-js" ecosystems of "rust, Haskell, go, c++" combined.
One of main strengths of Scala is that people get things done. It's the only language of the ones you mentioned which has reasonable support across three vastly different platforms.
> One of main strengths of Scala is that people get things done.
This is also true of rust, go, c++, and arguably haskell. I don't see anything about scala that inherently makes it easier to "get things done". If anything, I've spent a good 10% of my scala development time (over hundreds if not thousands of hours) debugging library, compiler, and documentation bugs and inconsistencies. This is not good for a language where you get things done.
This is false. You can see a few commercial uses in "Built with Scala.js" at https://www.scala-js.org/community/
I have also talked to people out of a few tens of companies who have told me they were using Scala.js in production.
Some people from losing ecosystems like to spin it that way¹, but the truth is people are happy, adoption is great, and companies see that pain-points get addressed.
Of course there are companies which drop Scala, but often like in LinkedIn's case it's not caused by a dissatisfaction with Scala, but new leaders making different decisions like "we are using 10 different languages, we should consolidate on X and Y!".
The last unhappy company which is commonly mentioned is this social-network company Yammer. That was half a decade ago, and most of the issues mentioned have been addressed in the mean time. (Yammer has been bought by Microsoft, so it also made some sense to use a language that makes getting bought out easier.)
¹ I remember there was some very very bitter Groovy evangelist a while ago.
Kafka is deprecating their Scala clients because of maintainability issues as well. Scala keeps making breaking changes on dot releases, making it impossible to maintain long term.
Scala is developed by the EPFL, Lightbend, and ScalaCenter. That's three, not one.
> Scala keeps making breaking changes on dot releases, making it impossible to maintain long term.
That hasn't been true for more than half a decade.
Very silly.
That doesn't mean you're wrong; but it does explain the confusion. I'm a fan of Scala but I'll admit I was confused by this as well when getting started with it since I've been trained to see everything after the first decimal as "minor".
The new Kafka client library is in Java just to reduce the number of dependencies, the server is still written in Scala. Strangely, using Java hasn't stopped Confluent from making breaking changes on dot release of the client... which really puts the lie to your explanation.
SBT makes it fairly straightforward to cross publish for different dot releases of Scala, and libraries in the Scala ecosystem do it all the time.
Scala dot releases are far apart. It's been over 2 years since 2.11, and 2.10 released in 2012.
By contrast, the JAVA client that the Kafka project published in November is already undergoing breaking changes for the current release candidate. It's pretty clear that avoiding breaking changes is not an overriding concern for them, and even if it is, it's equally clear that Scala was not the inherent problem.
Apache Groovy has always been good for scripting stuff on the JVM, including for Grails and as a DSL for Gradle. Unfortunately, one of its backers tried to retrofit it to do static compilation, and pitched it as a alternative to Java instead of simply an accompaniment. That's when the trouble started. Groovy's static compilation can't be as good as that of a language designed from the ground to be statically compiled, e.g. Java, Scala, or Kotlin. To this day, Groovy's own codebase is still Java. Even though some people talk anonymously of "1 million line Groovy codebase where static compilation is required", Groovy's own "million line codebase" isn't statically compiled Groovy and until it is, any claim of such codebases is just a claim, nothing more.
Sure, companies aren't dropping existing code in Scala, but I would be surprised if large organization were to push for new code to be written in Scala instead of Java 8.
Edit: link
Of course you can write Java in Scala and then the benefits over Java aren't that high (but are still there), but Scala is so far ahead in terms of language compared to Java that most current Scala libraries just _can't exist_ in Java.
Adoption these days seems to be great and accelerating, and given the growth of the library ecosystem there are more and more reasons to leverage the benefits of libraries written in Scala especially compared to legacy stuff written in Java.
Source: I work there.
- Can this reuse existing Scala code?
- How does it compare against Rust/GO/Swift? Why use this over them?
- How about libraries?
Java should have gone the native route for deployment, instead of having a few architects at Sun being religious against AOT compilation.
The browser, well personally I am not a big fan for anything besides interactive hypertext documents. Even though I worked several years as web developer, I tend to favor native + network protocols instead.
Many researchers jumped into the JVM because it provided a fertile ground for language research, without having to build their own.
Apparently LLVM brought a change to that and now everyone is using it instead of the JVM for language research, with the benefit of always having JIT and AOT toolchains, with GCC trying to follow up on that.
What I dislike in the JVM was the religion against AOT compilation (only third party commercial JVMs offer it) and missing out on value types, even though Eiffel, Oberon and Modula-3 where all having them.
Personally I think all language toolchains should offer JIT/AOT, with the developers using the best one for each deployment use case. Although for dynamic languages, AOT is probably not a good use case.
2. It's mostly the question of what language you're most comfortable with. I personally love Scala sans the JVM and that's why I'm developing Scala Native.
3. Subset of java.* is going to be supported. Pure Scala code that uses that subset and other Scala libraries should just work on Scala Native.
Now the type system of course is entirely different.
val label = "The width is "
val width = 94
val widthLabel = label + width
val first = "2"
val second = 3
val together1 = first + second
val together2 = second + first
Hopefully Scala at least has some very well defined and easy rules for exactly how string concatenation works, but I think it's an uphill battle to argue that '+' is a good concatenation operator.
(1) Passing an int/string when a function expects the other type? Yes, I had that happen lots.
(2) Concatenating a string and a (coerced) int?
It's either a much narrower subproblem of (1), or exactly what I wanted to happen in the first place (e.g. print "Your score is: " + n + " points".
Does that function that gets the user supplied value for height automatically convert to a numerical type, or is it a string?
What about the function that gets the user supplied value for width?
Is it stored in a back-end format that strictly types those fields to ensure that if the data is set from some less frequently used method it has the same type (e.g. are you storing as JSON, and can this JSON be created from multiple code paths?)
Given a simple statement like "foo = bar + baz", why should there be ambiguity when there doesn't need to be? The glib answer to the problem is to tell people to use better variable names, or document their functions better, or any number of suggestions that put the onus on the programmer, when this is a very simple to solve problem. Don't overload operators in the core language for conceptually different actions. We don't "add" strings, we concatenate them. Add is at best, shorthand for "add to end of" which means to append.
Coercion can be a useful tool, except for when it creates confusion. There are well known and tested ways to combat this confusion, so I'm not sure why we haven't adopted them widely where coercion is in use.
P.S. It's not just addition. Testing for equality is another common case where coercion can cause problems. The answer to that is equally as simple, use a different operator (such as "eq" and "ne" for string equality in Perl). Numeric equality is not the same as string equality (10.0 is equivalent to 10, but "foo." is not equivalent to "foo"), and making the programmer think (not not!) about what will happen when using an overloaded equality operator on two variables just shifts a small amount of up-front cost when learning the language (we have different operators for string concatenation and equivalence testing, learn them) to a cost imposed every time the programmer has to use those operators.
$varOne . $varTwo
"$varOne$varTwo"
There are some thoughts about migrating people away from + toward string interpolation. But automated conversion tools are required before this can happen.
>>> 1+2
3
>>> "1"+"2"
'12'
>>> "1"+2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: Can't convert 'int' object to str implicitlyI don't see making string concatenation ambiguous with property access being a real improvement. At least use `..` or some other unused punctuation.
auto greeting = "Hello" ~ " World!";
immutable x = 9 + 99;
auto:immutable::var:val
But don't mistake that for lack of practicality!
Good refactoring support in IDE can save you a lot of time and errors, especially in the statically typed languages. It has nothing to do with language being unbearable, but a lot to do with the size and complexity of the code you have to work with.
I have nothing against Haskell, and before sticking with Scala I've seriously considered it. However, having things like Akka in Scala, an actor/OTP framework that is the only one that is even remotely comparable to what Erlang has, was a huge benefit.
Other tools like Play, Spark also made choosing Haskell an unpractical decision for me.
Besides, after a couple years of excursion into pure FP-only approach to development, I've understood to myself that OOP is not in any way in opposition to FP, but can be a rather welcome addition. Especially when you work with big code bases and you plan to maintain them for many years to come. And that is an area where Scala really shines.
After writing and maintaining a bunch of code in Clojure, Erlang and functional style Ruby, I feel like OOP+FP gives me greater flexibility in my design options and allows to architect better solutions to my problems.
It seems to me Scala is potentially much more "general purpose" than Haskell. In other words, "more practical".
Haskell is currently a practical general purpose language. I don't see any evidence Scala is any better at this. Are you speaking from experience?
Hard and soft realtime both require deadlines to be met, the difference being that in soft realtime a missed deadline is not considered a fatal error, just highly undesirable.
Examples: Soft realtime: A game where 60 FPS is desired for smooth animation. Lower frame rates degrade the experience but are tolerated.
Hard realtime: Flight surface control software in a fly-by-wire aircraft. Missed deadlines potentially result in a crashed plane - a true "fatal error".
Any ahead of time compiled language with deterministic memory performance may be used for hard realtime. Absolute performance isn't required, although it is desirable. As long as memory primitives are available for native Scala that permit pre-allocation, and the GC can be turned off (trivial), it should work fine even for hard realtime.
As an aside, I expect for many things native Scala will equal C++ in performance. On the JVM is extremely close to Java in performance.
"but Haskell can be used to write simulators and games."
Realtime simulations and games? It seems hard to reconcile immutable state with time-based simulation in any efficient way. Then there are garbage collection cycles and laziness to deal with.
"Haskell is currently a practical general purpose language. I don't see any evidence Scala is any better at this. Are you speaking from experience?"
GC based languages in general aren't good choices for the types of systems discussed above. GC is also a problem on smaller hardware (embedded systems for instance) since there should be a much larger memory pool than actually used for good performance.
Haskell also throws in laziness, which leads to non-determinism.
Scala also supports mutable state and the OO paradigm if they are better or more convenient for a particular system.
> As an aside, I expect for many things native Scala will equal C++ in performance
Are you familiar with Haskell performance?
> [can Haskell be used for] realtime simulations and games?
Yes.
As I said, I can't speak for hard realtime, but then again, most sims and games seldom require it.
> GC based languages in general aren't good choices for the types of systems discussed above.
You seem to be arguing out of theory (which is why I asked you if you were speaking from experience). We're also not discussing GC based languages "in general"; we are discussing Haskell, which is a very practical language which can and has been used successfully in a variety of real-world applications, and which qualifies as a general purpose language. If in doubt, talk to an actual Haskell practitioner instead of arguing out of theoretical positions, and you'll be surprised. Many such practitioners frequent HN, and I'm sure they are eager (no pun intended) to discuss the systems they've worked in.
For that matter, will Scala Native not be a GC language?
> Scala also supports mutable state and the OO paradigm if they are better or more convenient for a particular system.
As I argued before, in practice this means Scala is not a great language for OOP or FP. It's understandable that experienced practitioners of either style will prefer cleaner & better languages.
I don't predict a particularly great outcome for Scala outside the JVM, since I think the JVM is its main selling point. Once you remove this point, other languages become distractingly attractive :)
Anyway, I wasn't trying to disparage either of them (or this project). Just poking fun at the seemingly common "I want to use Haskell but I'm forced to deploy on the JVM" use case for Scala.
Except that Martin Odersky himself has explicitly said that the primary influences were SML/OCaml and Java.
> In particular, do notation (for comprehensions),
I'll give you that one
> pattern matching
Which pre-date Haskell by nearly 2 decades in ML
> lots of standard lib classes (ex Maybe (Option)),
Also pre-date Haskell by nearly 2 decades, in addition to be named exactly the same as ML.
> many of the methods in the collections library (map, fold, take, etc)
Which pre-date Haskell by almost 4 decades with origins in LISP.
> Just poking fun at the seemingly common "I want to use Haskell but I'm forced to deploy on the JVM" use case for Scala.
Which is where the Scalaz folks come in. They want to piggy back off of something successful, as opposed to Frege, which is closer to what they actually want (avoiding success at all costs?). Who knows, maybe they actually like the strict-evaluation by default of Scala/ML, which is quite pragmatic, but definitely more in line with ML than Haskell.
As a Scalaz committer and as a developer working at one of the largest scala shops (that makes heavy use of Scalaz), we don't try to write Haskell on the JVM. We do try to write pure, functional code as much as we can, and so what we end up with falls somewhere in the middle between an OCaml and Haskell.
Funnily enough, Scala's implicit-based type classes were the inspiration for the coming modular implicits type classes in OCaml (as opposed to the Haskell approach). It wouldn't be the first time that two languages have mutually inspired each other (The Rust and Swift teams have both acknowledged certain design decisions inspired by the counterpart).
Is it documented anywhere that this was inspired by Haskell? To me, Scala for comprehensions seem syntactically more like a generalization of Python comprehensions than Haskell's monadic do notation.
I think Scala was influenced by Haskell's list comprehensions.
0: needs citation, though I remember reading something semi official about pythons comprehensions being influenced by Haskell. Maybe Python.org's page on Haskell.
> - Can this reuse existing Scala code?
I think that's the plan. Would be a pretty pointless exercise without that, right? :-)
> - How does it compare against Rust/GO/Swift? Why use this over them?
Rust: Scala and Rust have different niches. Rust is more focused on low runtime overhead, while Scala is more focused on low development overhead. This means Rust can be potentially faster to run, but Scala is faster to develop.
Go: Go is utter shit that only survives due to the devs name-dropping "Google" every 5 minutes.
Swift: Scala is more mature, simpler, better designed. Not a knock against Swift, but there is a difference between a language where changes have been tried experimentally for years before either adopting or removing them, and Swift where things get added at a frightening rate.
> - How about libraries?
Libraries without Java dependencies should work, libraries with Java dependencies depend on whether their Java dependency is provided by Scala-Native (just like on Scala.js).
Woah!
To say interest in is divided into either hate or love is disingenuous.
Having standardization on things like channels in the language makes for a much nicer consistency in the language. Scala has 8000 ways to do anything, half of which involve an exotic DSL. I'll take plain and simple please over esoteric and unmaintainable.
People don't like writing hundreds of lines of repetitive boilerplate the way go's lack of generics forces you to. They'll look for a better way, and they'll come up with hacks, just as they did in Java.
As for Go, I'd say that's the most honest description of the language I've seen in a while.
Scala is pretty good in the low boilerplate department. I'm not a fan of Scala in the slightest, but in terms of code more or less directly expressing ideas it isn't bad.
Scala certainly doesn't have high development overhead either. Build system more or less just works, plenty of libraries, minimal ceremony to do much. Don't get me wrong, I kinda hate Scala¹, but compared to _many_ languages it does have low development overhead.
¹ Disclaimer: haven't used Scala since 2.8. Maybe it's better now. Don't get your panties in a wad, but Scala is basically really shitty OCaml on the JVM in my mind.
I wouldn't want to go back to 2.8 even if people paid me a ton of money.
I love ML, but think OCaml is a poor ML. That's why I love Scala. It's extremely consistent, and design decisions are very considerate.
It's an amazing language and I can express my intentions much better than in OCaml (no typeclasses, no higher-kinded types, etc.).
Thankfully I'm no longer working with the JVM, so I doubt I'll use it much, but it could be fun to take another look at.
Oh, and OCaml is possibly getting typeclasses (via modular implicits—not unlike Scala actually, for better or worse). It's not very painful if you use an alternate standard library (i.e. Core) that actually takes advantage of OCaml's amazing module system.
Making blanket statements like that without any substantiation makes me want to just write off everything else you have said.
For better or for worse when most people hear a new statement or opinion coming from someone they will trust it as much as they trust the least believable thing they have heard that person say.
I think most people on this planet are able to consider each point at its face value.
Humans have a pretty strong demonstrated tendency to develop positive or negative emotional attachments to pretty much everything, including sources of information/arguments, and see the world strongly filtered through those. This is, while it can be misleading, an important evolved survival mechanism in terms of attention/resource management.
Most people probably can, by expending effort, mitigate that to a certain extent, but that doesn't negate it completely.
Perhaps this is exactly the filter I want to have.
One can see most work on Scala in git commits is done either by few people at a research institute(EPFL) or at a consulting company(lightbend) whereas Go has quite varied committer profiles and there are far more committers.
I would strongly disagree with the implication that Scala isn't used to make interesting or popular products (e.g. Spark or twitter), or indeed that it's "exploring" anything "cutting-edge"; it is very much a production-quality language (backed by theory sure, but theory that's been established for decades now), and I don't see that it contains anything more novel or experimental than e.g. Go's concurrency model. It's a decent platform for doing research in, but that's mostly just a question of being a good language.
> One can see most work on Scala in git commits is done either by few people at a research institute(EPFL) or at a consulting company(lightbend) whereas Go has quite varied committer profiles and there are far more committers.
We're talking about a factor of 2-3 in number of committers, not that I have any idea what that number is supposed to tell anyone. Certainly Scala has substantial industrial support and a wide base of contributors.
The language it's self has no beauty and is not fun or interesting like scala. But I still love it and would totally base my company on it if I was planning on hiring thousands of developers and training them to use it. I would be scared of doing that with scala.
Not sure what you mean by that.
I still think that you are not really thinking about how many non-language pain points go takes care of. You don't really worry about a library being the wrong version, you need extremely little special training for a new developer to start on a go code base (compared to FP which requires a paradigm shift for most people), the people that have to setup the servers generally appreciate something that only requires a static binary to be dropped on the server and executed. Almost none of the things that make go great are actually related to programming languages, where as some of the best things about scala are all of the elegant ways you can express computation.
It's a dumbed down language, not a simplified language.
Things are always added at a frightening rate when a language is young. Happened for C#, happened for Scala, happens for all languages.
Not really something to worry about.
Apple's Swift message is more or less "Stop writing Objective-C if you can and start using Swift – oh and by the way – you will have to rewrite your code with every major Swift release which we will be releasing at a rapid schedule".
It will be very hard for Swift to get rid of all the cruft they accumulate.
In Scala every major release addresses one or two pain points and migration is very smooth – no "rewrite all your stuff".
Yes, the Java way. Take ten years to implement basic stuff.
I much prefer Swift and C#'s approach to this: first major version bump, things get deprecated. Second bump, they get removed. It worked great for C#, it's going to work great for Swift, which has already seen incredible adoption on iOS.
Scala is moving at a glacial pace in comparison, really not worth bothering with.
As you surely know it already, the deprecate-remove cycle is pretty much how a) Scala works and b) C# doesn't work.
We detached this subthread from https://news.ycombinator.com/item?id=11678317 and marked it off-topic.