GraalVM: Run Programs Faster Anywhere
graalvm.org
graalvm.org
Graal: https://github.com/oracle/graal (GPLv2 with CPE)
JavaScript with Node Integration: https://github.com/graalvm/graaljs (UPL - BSD license)
Ruby: https://github.com/oracle/truffleruby (EPL, GPLv2, LGPL)
R: https://github.com/oracle/fastr (GPLv2)
Python: https://github.com/graalvm/graalpython (UPL - BSD license)
LLVM/Sulong: https://github.com/graalvm/sulong
Yes there is a Java embedding API.
http://www.graalvm.org/docs/graalvm-as-a-platform/embed/
> (provided that it was run under JDK 9)
It can run on 8. It'll be fast on JDK 9 or 10 because these have the interface to use Graal.
> Are there ways to call out to Java like JRuby?
JRuby compatible interop:
https://github.com/oracle/truffleruby/blob/master/doc/user/j...
Our own interop:
https://github.com/oracle/truffleruby/blob/master/doc/user/p...
> Having some difficulty finding documentation around this.
It’s good for quite a lot though.
It's a little bit more complex than that in that we can run a Ruby application ahead of time up to a certain point where we compile it, and then when you run the application that state when it was compiled is restored. So we load the core library during compilation for example.
Will this be the case for truffle python, js,etc ?
The problem is that the semantics of Ruby are such that you can't compile it efficiently out of the box. TruffleRuby can but only by making tons of assumptions at runtime that might not be valid. So it compiles code on the assumption your program is normal and not weird, and if those assumptions are violated, it goes back to interpreting the source code.
Therefore even if you could pre-compile Ruby in some pedantic sense, you'd still need the source code to fall back to when the JITC has to bail out.
The slow-path version of the code that violates runtime assumptions can still be compiled as an alternative variant - so instead of switching from fast compiled code to the interpreter, the code violating the assumptions can switch to slow-path compiled code.
Dir.glob("some dir/*").each do |filename|
require filename
end
But other times it is used as a plugin mechanism, where the expectation is for the code to be provided by a user or other developer, and processing this statically at compile time would remove capabilities that the user would expect.
You can't automatically determine things like that, because it is down to developer intent and the code does not document the intent.
So the problem goes deeper than being about being able to revert to a slow path: To compile Ruby in a meaningful way you need to make some level of trade-off in what aspects of Ruby you decide to treat as "ahead of time" vs runtime, because there is no clear rule that says "now the program has been loaded; everything after this is runtime".
I would think that the fact that Ruby constructs that look like what would be compiler directives in a more static language (e.g., Ruby's “require” vs C’s “#include”) have their semantics defined in a way which both depends on and affects runtime state means that AOT compiling Ruby in the strict sense gets you very little, whereas running it to a certain point AOT and saving the runtime state gets you closer to what you'd expect from compiling based on usually-compiled languages.
Select your start and your target language and it will show you an example. Its a very basic example, but it basically works with all Java types. We will provide better docs soon.
Or have I misunderstood this totally? I thought I could do something like ruby script -> graal ruby --> jvm bytecode --> native image
Update:: Doesn't look like I read it wrong - from this reddit comment[1] from 3months ago
Ahh, I think there's some confusion here. What we do with the SVM is ahead of time compile the TruffleRuby interpreter, not an application. This is akin to what you'd do with MRI. Instead of treating Java as a language you compile to a .class file and load with the JVM, the SVM's native image generator compiles it to a native binary.
There's likely no reason we couldn't compile Ruby applications into a static binary.
I would also love to be able to distribute a ruby binary as a single statically compiled executable. Any pointers/directions?
[1] https://www.reddit.com/r/ruby/comments/7o3wy9/graals_substra...
I think this was a theoretical observation.
What is more likely in the near future is to be able to initialise your Ruby gem at compile time, so when you start your application it is already required and you can start execution as if all your require statements had already run.
Why is this ? We are currently on JDK 10. Also - why is there no official Graal Python ?
And there is an official Python alongside the other languages.
For Clojure I don't know. But from what I understand Clojure produces quite inefficient bytecode, so it may benefit a lot.
One of a million examples, this happens on the front page of HN at the same time:
https://www.reddit.com/r/javascript/comments/8d0bg2/oracle_o...
I am excited to see such a beast coming from a major company with a strong track record in VMs, and particularly given that it is based on their existing JDK, and am particularly excited that it now seems to be available!
I have some questions that I did not managed to easily get answered from looking over the website.
Can I AOT with this (via SubstrateVM) and deploy to "mobile" targets, and specifically iOS?
Are there any examples of embedding GraalVM in an existing native application (such as the noted idea of having GraalVM accessible inside of MySQL)?
Is there an existing embedding for node.js? (I don't mean "can I run node.js apps on GraalVM?"; I mean: from node.js, can I instantiate a GraalVM and give it code to execute in a new context, or will I need to build my own bindings to the GraalVM embedding API for that first?)
I did find some documentation for using an embedded GraalVM in a Java application, but while I work on many projects in many languages, none of them have been written in Java now for over a decade. I guess I was expecting a C/C++ API to be documented somewhere, not a Java API (due to the listed prototypical embeddings, and based on my long experience with embedding other virtual machines, including the Oracle/Sun JVM).
> Can I AOT with this (via SubstrateVM) and deploy to "mobile" targets, and specifically iOS?
Technically there is no reason to not support it. But support is not implemented yet.
> Are there any examples of embedding GraalVM in an existing native application (such as the noted idea of having GraalVM accessible inside of MySQL)?
Yes. Oracle MLE[1] and MySQL already work. Oracle MLE you can try already. We have a native embedding API (its very similar to the Java API). We could not yet quite finish the documentation of it for website. But its already included look for a file polyglot_api.h in the binary.
That's what their "node" program actually is. It's regular node but using GraalVM instead of V8. The semantic difference you're getting at doesn't seem to exist.
I guess I was expecting a C/C++ API to be documented somewhere, not a Java API
SubstrateVM has a C API that can be used to start it up that's not the same as JNI, however, bear in mind Graal can be used as a plugin to regular HotSpot too. So you can just embed the JVM the usual way and ship the Graal compiler with it, enabled via a command line flag. So yes you can do this and it doesn't require any new documentation.
No, because what I'm trying to accomplish is "I'm inside of a node.js program and I want to instantiate a GraalVM virtual machine instance context, load a random Java class (or whatever), and run a function on that class". Even if I'm running my node.js code on top of GraalVM, the semantic difference fundamentally exists. Like, you are trying to tell me that somehow running my node.js program on top of GraalVM is the same thing as being able to run GraalVM from inside of my program (which just so happens to be written in node.js, but that's arbitrary and meaningless in a way: I just want good bindings for the language I happened to write it in) on arbitrary code, but that clearly doesn't make any sense. I mean, it certainly isn't true for their Java embedding API, why/how would it somehow be true for their node.js API?
> SubstrateVM has a C API that can be used to start it up that's not the same as JNI... So you can just embed the JVM the usual way and ship the Graal compiler with it...
But SubstrateVM is the offline VM for AOT use, not the runtime VM for JIT use, so I really am not that interested in the SubstrateVM API... it isn't at all relevant for the use case I'm describing for embedding GraalVM; and the Graal compiler in this context is also for AOT use, which again isn't relevant for the use case of embedding the GraalVM.
What makes GraalVM interesting here is that it is a JIT. Just take a step back for a second and imagine you have a C program, and you want to boot that JIT and use it, as a library, loaded into your process. That's the use case here. You seem to be confusing that with either running the C program inside of the VM (which doesn't even change the problem) and then later with pre-compiling the thing we want to load and bringing the compiled code into the process (which bypasses the wonderful goal of working with the JIT).
The way you do this using their Java embedding API is clearly documented, and it is absolutely not the same thing as running a Java application in the GraalVM: it is a set of APIs that let you instantiate Graal Context objects, eval() code in various languages, and then work with the proxy Value objects. I am asking if that API has been exposed to any languages other than Java yet. It seems (from the answer from someone who actually works on Graal from 8 hours before your comment) that the answer is "yes" for C/C++, but I'm still guessing the answer is "no" for node.js.
So if you wanted to run JavaScript using GraalJS, from C, you would link against libjvm, use JNI to initialise the VM with Graal as the JIT compiler (there's a special flag for this), and then use the Polyglot API via JNI.
To bind from JavaScript to Polyglot without having Graal be the host would presumably require you to write a V8 extension that then loads the JVM using JNI and proxies back and forth. I think there are projects that do this already. But you wouldn't get the performance or convenience improvements than Graal/Truffle are pioneering.
In the end though, embedding the JVM this way - at least for node apps - would not be better or more efficient than using their node implementation, but ignoring things like performance and debuggability the results are presumably identical unless your code depends on an implementation detail of V8. So I don't get why you'd do that.
How feasible would it then be to use Graal NFI for FFI from vanilla Java? That seems useful given the performance limits of JNA and the complexity of JNI.
The GraalVM in MySQL will be released soon and that will be the best example of embedding GraalVM in native projects.
The API for embedding in native applications is still experimental: it might change slightly in the few following versions. You can find the API with documentation in the polyglot_api.h file in the GraalVM distribution.
If I have a choice between running our stack 50 times slower (and thereby 50 times the cost) or running anything Oracle, I know which choice is cheaper and doesn't come with an existential threat to my company.
Only governments and big corp are rich and reckless enough to take on this kind of risk willingly.
There is like a bajillion people out there running Oracle Java, MySql, Virtualbox and so on. All of them Oracle products, and with permissive licenses. Where is this "existential treat" thing coming from?
Oracle. (see, I can be sarcastic too)
Ignoring the fact that most of those users were likely acquired back when those products were still known as Sun Java, Sun MySql, and Sun Virtualbox, those "bajillion" people are not lucrative enough to warrant the effort. The moment one of those people becomes large enough, Oracle's lawyers will come knocking. If you're large enough, they will even go as far as to change how copyright law itself is interpreted to get their fill.
With a company like that, entering any form of contract or agreement is a major liability and an existential threat.
Can we see an example of big Java user being called by Oracle's lawyers? Excluding Google, they created their own Java with no regards to license, so yeah they got sued. So... did anyone that just downloaded Java and used it, got sued? I do not see it. Nor do I see what they could possibly be sued for.
You don't need a license to build a clone.. well, you didn't need one until recently thanks to Oracle.
While not exactly the same thing, I'm still glad that IBM didn't pursue IBM-compatibles using oracle's absurd logic (or ford and other car manufactures for that matter). I like competition.
> did anyone that just downloaded Java and used it, got sued? Perhaps not sued, just threatened into paying. See sibling posts for your anecdotes.
Fast forward a bit and Oracle sales people where calling him and trying to convince him that he needed a paid license - to run default MySql on AWS(!).
They were so scary he almost gave up and paid.
All of those examples have dual commercial/copyleft licenses, none of them have permissive licenses. Moreover, the Free licenses are without explicit patent grants which may, therefore, be at risk of patent claims.
No issues were raised regarding Java or MySql. Bottom line, I don't know the difference between this and that license, what I do know is that Oracle licensing for Java and MySql gave us zero grief. For the record, we just use Java and MySql as provided. I can see how Google making their own "Java" for Android can get messy. But that is kind of unusual, who makes their own Java? I know of several vendors, and they usually try and get certification/TCK and so on. Google did not, well, that is Google's problem.
I also want to echo others' concerns about the license, lock-in, and lack of explicit patent grant. It's a shame that Oracle's (well-deserved) reputation is tainting so many excellent engineering efforts. I hope that these comments serve as some form of feedback about the very real cost the Oracle brand has suffered as a result of its legal activities.
http://www.graalvm.org/docs/graalvm-as-a-platform/implement-...
Here's an example of the addition operator from Graal's SimpleLanguage:
@NodeInfo(shortName = "+")
public abstract class SLAddNode extends SLBinaryNode {
@Specialization(rewriteOn = ArithmeticException.class)
protected long add(long left, long right) {
return Math.addExact(left, right);
}
@Specialization
@TruffleBoundary
protected SLBigNumber add(SLBigNumber left, SLBigNumber right) {
return new SLBigNumber(left.getValue().add(right.getValue()));
}
@Specialization(guards = "isString(left, right)")
@TruffleBoundary
protected String add(Object left, Object right) {
return left.toString() + right.toString();
}
protected boolean isString(Object a, Object b) {
return a instanceof String || b instanceof String;
}
}
By default it uses an arbitrary precision adder. But node rewriting on the AST (predicated by the "guard") provides specializations for long and String types. The long-type execution is a "fast path" that executes machine instructions directly; the String-type execution is a string concatenation.
A research paper provides further details:
https://dl.acm.org/citation.cfm?id=2658776
The full tutorial is at:
https://github.com/oracle/graal/blob/master/truffle/docs/Lan...
https://github.com/oracle/graal/blob/master/truffle/docs/Lan...
If not. Please let me know.
https://github.com/frodwith/jaque is the project though, which is an implementation of the Nock VM from Urbit
But the performance seen on a trivial Ackerman function was very good -- Nock via Graal got close to C performance. That's impressive for what's basically a Lisp.
BTW.: I'd recommend upgrading one Truffle release at a time. So it should always continue to compile. Just fix the deprecation warnings and continue with the next version until you reached the last version. We always keep deprecations for at least one version before we remove/break APIs.
If you want to discuss something in more detail feel free to join us in the Gitter chat: https://gitter.im/graalvm/graal-core
We don't bite.
Not at all! Thanks for the advice, and I'll take you up on the invitation to come chat before too long.
> Are you working on a Java application? Do you want a faster runtime? Enhance your code with JavaScript!
First, the JVM already has the Nashorn JS engine. Second, if I'm "working on a Java app", that means that it's already running in a JVM. Where does the faster runtime come from? Is executing JS from Java using Graal faster than running actual Java code? Is it just faster than Nashorn? Something else?
"Are you working on a Java application? Do you want a faster runtime?"
Graal provides an ahead of time compiler for Java, which can speed things up by removing JVM warm up time. I'm unsure how much faster Graal is compared to steady-state JVM.
"Enhance your code with JavaScript!"
Embedding Graal is an alternative to Nashorn.
Here is a full list of reasons: http://www.graalvm.org/docs/why-graal/
Here are some (maybe a bit outdated, but independent) benchmarks comparing it to Nashorn: http://blog.nidi.guru/tech/javascript/2018/04/02/javascript-...
We have some way to go to beat V8.
- graal has superior compiler technology based off of a research paper [1] that:
-- has superior ability to remove costly object allocations in many scenarios
-- has better inlining and more aggressive speculative optimizations
-- has smarter jit compilation
[1] http://www.ssw.uni-linz.ac.at/Research/Papers/Stadler14/Stad... (pdf)
http://lafo.ssw.uni-linz.ac.at/papers/2013_VMIL_GraalIR.pdf
https://dl.acm.org/citation.cfm?doid=2245276.2232084
http://aleksandar-prokopec.com/resources/docs/graal-collecti...
edit: lol: talk about a low blow!
> The community version does not support DWARF information. The enterprise version supports all native tools that rely on DWARF information, like debuggers (gdb) and profilers (VTune).
Otherwise with GraalVM you could run your non-Java environments on Graal and go at it the other way around.
Are you calling from Java? Then there are more examples here: http://www.graalvm.org/docs/graalvm-as-a-platform/embed/
Also some of the examples on the website use interop: http://www.graalvm.org/docs/examples/
1. Community Edition (CE)
> GraalVM CE is available for free for development and production use. It is built from the GraalVM sources available on GitHub
2. Enterprise Edition (EE)
> GraalVM EE provides additional performance, security, and scalability relevant for running critical applications in production. It is free for evaluation uses and available for download from the Oracle Technology Network.
"performance, security, and scalability" words don't present in p1. Is it just me or...?
All major commercial JDKs do support AOT compilation and ExcelsiorJET is free for open source projects.
However bear in mind you can already bundle JavaFX and the JRE together into native installers for each platform that don't require a JRE or JVM, and don't mention Java anywhere, using the javapackager tool. I've used it several times, it works well enough.
https://news.ycombinator.com/item?id=6232240
That was the earliest post I remember from HN. But I record there some other information scattered around a little before that.
So this is nearly 5 years coming. I am amazed what at first seems like fantasy, impossible work is now actually thing. 2012 / 2013 was the year when VM and JIT has lots of innovation. Rubinus, LuaJIT, SpiderMonkey and all those Javascript VM.
So like the JVM or JDK, there is nothing stopping you from for forking the code, as long as you don't call it Java, and "your" JVM would not get the test suit.
GPL 2 with Classpath exception has been with us for so long. I know you hate Oracle, but I don't see any downside to it. I mean half of the Internet literally runs on JVM and MySQL.
I had to look hard and it doesn't seems to have any strings attached.
How large is the resulting executable? There was some mention of shares libraries — is it possible to build a runtime that is callable from C?
Relatedly, does the GraalVM java executable support JNI?
I’m also curious about the runtime footprint in general; any idea what is the smallest possible VM for a real but relatively-small language like Lua?
A minimal executable is about 5 MB.
> There was some mention of shares libraries — is it possible to build a runtime that is callable from C?
Yes you can compile a JAR to a shared library with C function entry points.
> Relatedly, does the GraalVM java executable support JNI?
Yes.
> I’m also curious about the runtime footprint in general; any idea what is the smallest possible VM for a real but relatively-small language like Lua?
Real language VMs are quite a bit bigger - a hundred MB or so.
> Real language VMs are quite a bit bigger - a hundred MB or so.
Don't know if you're trying to say Lua is not a real language, or just skipped the part about Lua.
I'm not passing any opinions about any languages or VMs here.
Is this mostly JRE? Would Jigsaw help, or is SubstrateVM already doing something like that? (To be clear, this tech is really impressive! Just trying to get a sense of embeddability constraints)
Language VMs compiled using SubstrateVM are larger than you may think because as well as the native code they also need to contain a representation of the code that the compiler can read (IR) so it can dynamically compile the methods that are combined to form the compiled version of your user's code, and we also need to compile a sort of runtime debug information that allows us to jump from optimised code back to the native code when an optimisation proves to be wrong. This is a novel thing - most language VMs jump from optimised code to a bytecode interpreter - we have to jump back to the native code as there is no bytecode interpreter. Finally, a language VM also needs to include all of the Graal compiler itself for dynamic compilation, which is quite a bit of code.
Currently we have the same APIs for CE and EE. So applications are easily portable between those two. We have no plans to change this.
But. You don't have to take our word for it. CE is open source and you may build, extend and maintain it yourself if you want to.
What it actually provides is a form of "universal abstract syntax tree". So if a C program wants to access a JavaScript object by writing "a->b = c" then the dereference is turned into a snippet of JavaScript code (in effect) and inlined right into the C function.
From the website:
> In order to provide foreign polyglot values meaning in languages we have developed the so-called polyglot interoperability protocol. This interoperability protocol consists of a set of standardized messages that every Graal language implements and uses for foreign polyglot values. The protocol allows GraalVM to support interoperability between any combination of languages without requiring them to know of each other.
This technique is based on some research we did in 2015: http://chrisseaton.com/rubytruffle/dls15-interop/dls15-inter...
GPL.
No comment beyond that but I thought I might save someone a few clicks.
It's not that all the stuff is bad or that all the 'free' things turn sour, most of the things are technologically sound, but there is just so much lawyering, licensing and IP fighting that takes all the joy out of life. This isn't unique to Oracle, but not even IBM is as passive-agressive as their non-tech people and documents are.
I’m surprised you called it FUD, given the high-profile case against Android (google).
I was about to recommend we try it at my company and gave up as soon as I read “Oracle” in the company name.
Call it FUD if you want but Oracle does have a damaged reputation.
"James Gosling: Oracle vs Google"
Oracle has done lots of nice things for Java, some of them with uncertain future or even tabu at Sun, like Graal's percusor MaximeVM or AOT compilation.
It might not be a company loved by FOSS, but other than IBM there wasn't anyone else caring that much for Java's fate, Google specially did not even bothered.
So I appreciate every improvement Oracle does to Java.
We are all aware of the danger of using techs from those companies.
Recent story about Oracle exercising copyright over the word "javascript": https://news.ycombinator.com/item?id=16862949
Remember when Oracle sued Google for re-implementing the Java standard library.
Remember the OpenOffice -> LibreOffice thing. Remember the Hudson -> Jenkins thing.
There's the enterprise sales/licensing tactics which I guess is not very familiar or visceral for most of us, me included. But I'm sure there's more I'm forgetting ...
Ask that to the civil victims of drone attacks and Apache pilots having fun with civil convoys, as proven by many available video footage.
> Recent story about Oracle exercising copyright over the word "javascript": https://news.ycombinator.com/item?id=16862949
It was a decision done by Apple, Oracle did not move a finger.
"As you are likely aware, Oracle owns US Trademark Registration No. 2416017 for JAVASCRIPT. The seller of this iTunes app prominently displays JAVASCRIPT without authorization from our client. The unauthorized display of our client's intellectual property is likely to cause consumers encountering this app to mistakenly believe that it emanates from, or is provided under a license from, Oracle. Use of our client's trademark in such a manner constitutes trademark infringement in violation of the Lanham Act. 15 U.S.C. § 1125(a)(1)(A). In order to prevent further consumer confusion and infringement of our client's intellectual property rights, we request that you immediately disable access to this app. We look forward to your confirmation that you have complied with this request."
> Remember when Oracle sued Google for re-implementing the Java standard library.
Not only do I remember, I stand by it.
"James Gosling: Oracle vs Google"
https://www.youtube.com/watch?v=JQ7xVO9lqD0
> Remember the OpenOffice -> LibreOffice thing. Remember the Hudson -> Jenkins thing.
I do, other companies did similar things without getting the half the hate Oracle does.
Many people here even would probably like to work for those companies.
The quote you posted was clearly sent to Apple by lawyers acting on Oracle's behalf, which was then forwarded to the developer by Apple.
I base this on the author's repeated use of the phrase "our client's IP".
That's not true. If you read the text, the request comes from Oracle's lawyers. "our client's intellectual property":
> "As you are likely aware, Oracle owns US Trademark Registration No. 2416017 for JAVASCRIPT. The seller of this iTunes app prominently displays JAVASCRIPT without authorization from our client. The unauthorized display of our client's intellectual property is likely to cause consumers encountering this app to mistakenly believe that it emanates from, or is provided under a license from, Oracle. Use of our client's trademark in such a manner constitutes trademark infringement in violation of the Lanham Act. 15 U.S.C. § 1125(a)(1)(A). In order to prevent further consumer confusion and infringement of our client's intellectual property rights, we request that you immediately disable access to this app. We look forward to your confirmation that you have complied with this request."
The idea of interoperating languages is great though. From the other polar end of the corporate world, we have .NET and its open source versions, and I hear IronPython, Javascript etc work pretty well in the .NET ecosystem. Not my cup of tea though.
[1]: http://www.graalvm.org/docs/reference-manual/languages/ruby/
BTW. In my experience so far, when it comes to software - a bird in the hand is worth ten in the bush
For dynamic languages like JS you can’t actually make these optimizations AOT, which is why we JIT JS in the first place.
GraalVM improves the performance of Ruby, Python, etc and also allows you to run C and C++ code in the same VM as a bonus with good performance. Unless you're using an intermediary bytecode like WebAssembly or JVM Bytecode you will also need to ship full sources with your C or C++ application.
https://blog.plan99.net/graal-truffle-134d8f28fb69
discussed at https://news.ycombinator.com/item?id=13343845
The idea there: "Graal is a research compiler. ... Truffle is a framework for implementing languages using nothing more than a simple abstract syntax tree interpreter."
As a bonus, you get high speed, good instrumentation for debugging and profiling, with great garbage collection, and hotswap code as you develop.
"The OCA gives Oracle and the contributor joint copyright interests in thre code."
What does 'thre' mean? ;P
They want it physically printed and signed.. sigh, I haven't printed anything in years.
Is this normal to require a scanned signature for CLAs? I noticed Apache CLA is the same (they want scanned paper signature), but all others I've seen are an online form.
"One VM to rule them all" reminds me of Parrot VM (for Perl 6). Looks like development slowed down a year or two ago. Back then, it got me excited as it looked like the "open source community" alternative to "Big Corporate Java".
Ref https://github.com/oracle/graal#license, appears that the parts you would want to distribute have the classpath exception which allows linking without messing with the license of your code (same as the JVM). The FAQ also covers this: http://www.graalvm.org/docs/faq/.
There is a community edition (CE) and an enterprise edition (EE) of GraalVM. The community edition is distributed under an open source license. It is free to use in production and comes with no strings attached, but also no guarantees or support. The enterprise edition is available from the Oracle Technology Network under an evaluation license. It provides improved performance and security for production deployments. If you are interested in using the enterprise edition in production, please contact graalvm-enterprise_grp_ww@oracle.com.
A lot of language specific libraries incorporate foreign language (often C) libraries and add additional functionality thst fits the ergonomics of the target language, rather than being a thin wrapper around a language neutral library. This is more than aesthetic when the target language has, for example, differences in type system or fundamental guarantees from the underlying library's language.
OTOH, sometimes there aren't C (etc.) libraries, and it's more straightforward to implement a library in (say) Python. But once you have a Python library, calling into it from Rust, C, Erlang, or even a broadly similar language like Ruby is, to put it mildly, non-trivial.
And that's just the top-level differences. It gets worse as you get into the details of API style and the costs of the indirection layers to convert into the local style, assuming conversion is even technically possible.
Even if you write with the lowest possible common denominator of C or Rust with no runtime, you'll still encounter a significant subset of those issues. And of course we still will use such things; all serious languages can communicate in some manner with a C implementation of a library. (Not for any theoretical reasons, really, C has more opinions than people often realize, but because it's still the baseline current systems are built on.) But it can't be as good as a native implementation.
Heck you will encounter those issues in a large project using only one language!
You mention Python that they say has an easy FFI with C. That's nice but it's usually used for the performance more than for reusing code.
(I'm having doubts if this comment belongs here. But it's my genuine reaction, for what it's worth.)
LLVM is providing most of the interop without strings attached.
I don't know a single person that's said a single good thing about Oracle, including people that work there. I know a ton of people that have been burned by Oracle and their lawyers.
Except a VM with a more liberal license could easily beat it in popularity.
These licenses prevent Graal VM from being used inside e.g. OSes and browsers with a more liberal license.
The fact that some people will use the code without giving back is secondary to that.
That is not just my opinion, because if a more liberal library comes into existence (inevitable), it will prevail.
And on that day you will miss when GNU/Linux mattered.
But hey, we will always have freeware and public domain.
The concept lives on in the form of two "new" Perl 6 projects, namely MoarVM (started in 2012; a multi language VM) and NQP (started around 2008; a virtual VM and multi language compiler toolkit).
These can appropriately be compared with Graal and Truffle. Imo their main weakness is they're not backed by Oracle but rather the Perl 6 project.
I think MoarVM/NQP are strikingly similar in their approach based on what I've learned so far about Graal/Truffle (which isn't much; perhaps a few hours of exploration; eg reading/watching the One VM to Rule Them All, One VM to Bind Them paper/video). [1]
Given the large number of Oracle employees working on Graal/Truffle (40 in 2016 according to the video) it doesn't surprise me that it's currently a lot faster.
Did you use Graal/Truffle friendly benchmarks (just as was done in the video)? Have you tried MoarVM friendly benchmarks to see where MoarVM shines? In particular, have you accounted for the new JIT that recently landed after 3 years of non-master branch development? [2]
Would you please elaborate a little on how MoarVM is insufficiently flexible?
TIA for any reply.
[1] https://www.youtube.com/watch?v=FJY96_6Y3a4
[2] https://perl6advent.wordpress.com/2016/12/09/a-preview-of-th...
Does MoarVM use a meta-compilation technique? Pypy for example uses meta-tracing and Truffle uses Partial Evaluation to get from an interpreter specification to dynamically compiled code without duplicating the language logic. If yes, could you please point me to further information?
Meta-compilation is the primary innovation in Truffle. It allows to compose languages together and to compile them as one unit.
If you want to know more about the Truffle approach to meta-compilation see this PLDI paper from 2017[1].
[1] http://chrisseaton.com/rubytruffle/pldi17-truffle/pldi17-tru...
Its NQP that does metacompilation, not MoarVM (afaik).
I linked to NQP's (spartan) readme above.
Some other points/links that might be helpful:
* nqp is written in nqp.
* nqp has a first class grammar construct. See wikipedia's Perl 6 Rules page.[1] (nqp's grammar is defined using nqp's grammar construct. So is Perl 6's.)
* nqp and MoarVM implement a protocol for abstracting behavior (eg methods) and representation (memory layout and use) called 6model.[2]
To try set your expectations appropriately, let me emphasize this note from the NQP readme: "NOTE: there's no end-user support for NQP and the behaviour can change without notice. It's a tool for writing Perl 6 compilers, not a low-level module for Perl 6 programmers."
NQP is a tool for writing compilers, not just Perl 6 compilers, and is in fact much more than that, but the Perl 6 project is currently very focused on the Perl 6 language and definitely doesn't want ordinary Perl 6 programmers thinking NQP is a tool for them.
Doc is sparse, again reflecting the focus on the Perl 6 language and the Rakudo compiler that builds atop NQP, analogous to your team focusing on just one of your guest languages.
The best doc for getting into NQP is not even in the NQP repo. It's the support materials for a 2-day workshop held in 2013. The abstract is "This intensive 2-day workshop takes a deep dive into many areas of the Rakudo Perl 6 and NQP internals, mostly focusing on the backend-agnostic parts but with some coverage of the JVM and future MoarVM backends also. During the course, participants will build their own small compiler, complete with a simple class-based object system, to help them understand how the toolchain works.".[3]
Beyond that, there is a doc directory in the nqp repo.[4]
Please write an update here on what you find out.
----
[1] https://en.wikipedia.org/wiki/Perl_6_rules
(nqp rules are Perl 6 Rules grammars but the general purpose DSL (!) for writing procedural code embedded in an nqp grammar is nqp's general purpose DSL rather than Perl 6's general purpose DSL.)
[2] https://github.com/perl6/nqp/tree/master/docs/6model
(MoarVM directly implements 6model -- MoarVM is named for "Metamodel On A Runtime". 6model is how one nqp based language is able to do magic like subclassing a class written in another language (whether nqp based or via an existing compiler) despite them having distinct OO semantics and implementation strategies.)
[3] https://github.com/edumentab/rakudo-and-nqp-internals-course
I'm also curious if .net standard/c# support is being planned.
Being able to reasonably use R stuff from clojure (and other languages better for general purpose programming) is my data science dream. Though I can't say I'm sad with my current unix-y solutions.
We don't test all of them but many.
We have considered .NET support but we are not actively working on it right now. But GraalVM is an open platform. Everyone can develop a language for it: http://www.graalvm.org/docs/graalvm-as-a-platform/
Edit: It works (for Ruby and Python)
Here is how to set up GraalVM for the Linux Subsystem on Windows 10:
1. Enable and Install Ubuntu for Windows 10: https://docs.microsoft.com/en-us/windows/wsl/install-win10
2. Download the EE edition of GraalVM to Windows: http://www.graalvm.org/downloads/
3. Mount the folder that has the tar file: https://stackoverflow.com/a/42586455/2548452
4. Unzip the files (tar xvfs graalvm-ee-1.0.0-rc1-linux-amd64.tar.gz)
5. Create a .bash_profile for setting the $PATH (my personal preference)
6. Follow the instructions from the following up until the "Running Programs": http://www.graalvm.org/docs/getting-started/
7. Follow instructions for installing LLVM and locales: https://github.com/oracle/truffleruby#system-compatibility
8. Follow instructions for installing libssl-dev: https://github.com/oracle/truffleruby/blob/master/doc/user/i...
9. NOW continue on with everything _after_ "Running Programs": http://www.graalvm.org/docs/getting-started/
Getting npm to work:
I could run node -v, but could not run npm and I was getting this error:
jvm library could not be loaded: /home/raj/graalvm/jre/lib/polyglot/libpolyglot.so: cannot enable executable stack as shared object requires: Invalid argument
I fixed it by running the following:
1. sudo apt-get install prelink
2. sudo execstack -c ~/graalvm/jre/lib/polyglot/libpolyglot.so
Going to do some experimenting, but so far so good!
If you really insist, invoke instead as .exe so you can still use bash and other tools, but run the actual compile or runtime using the system-native implementation.
So, any plans to support Windows and MacOSx for CE?
Also, any support for cross-compilation to generate native-binary for a different target(something like Golang GOOS) ?
Can anyone comment on how mature the R and python support is? I'm coming at this from an ML/data science background, so anything that speeds up the above while maintaining broad library support would be a great help.
- Graal.python is pretty new and its really early days. So even basic language things might break right now. But we are investing a lot of resources in Python support.
You can check for compatibility of packages here (not yet for Python): http://graal-staging.us.oracle.com/docs/reference-manual/com...
Would love to use this in Rstudio for exploratory analysis which would also need ggplot to work. Quite often have to cancel long running tasks when exploring a data set, so even if its a bit slower for interactive tasks but much faster for longer running tasks would be huge. E.g. typical interactive finished in 1 second instead of 50ms, but large loops that would take an hour finish in 10 minutes.
Here is the correct link: http://www.graalvm.org/docs/reference-manual/compatibility/
(Accidentally posted the internal link :D)
https://gist.github.com/havenwood/6fed7dd17131330ae4fd36a06f...
*( Scroll to the bottom for Summary )
Thank The Maker.
Until then, areas like web development always will be a torture.
For ClojureScript, it would mean full circle. You could run your ClojureScript back on the JVM, and have Java interop from it.
There are technical reasons we cannot have a CE for Mac OS as well. But we are working on that.
gu -c install org.graalvm.ruby
or from a local file
gu install ruby-installable-linux-amd64.jar
Can anyone comment on TruffleRuby ? What remains to be done for it to be production ready ? GraalVM page still mentions it's experimental.
It's very hard to produce another Ruby implementation without breaking some of those assumptions, but we can hopefully get it down to a small set so the changes needed are minimal.
There are other things like throwing Ruby exceptions across C library boundaries which may work in MRI, but whose behaviour may not be intentional and could already have subtle problems, and I don't think we should be aiming for compatibility in those areas at all. Finding and fixing things like that is normally a good thing to do anyway.
I’m working on Discourse at the moment and can run the dB create and migrate tasks and I’m working on the asset compilation pipeline, but it’s all pretty rough and won’t be our final solution.