Graal and Truffle could accelerate programming language design
medium.com
medium.com
"Mostly" in this case means semantically, not syntactically. It's things like concurrency model; floating point behavior; memory layout; FFI; semantics of basic libraries; performance characteristics; and level of dynamism. You can write a Python frontend for both the JVM and CLR, and it will look like Python but act like Java and .NET, respectively, with no guarantee that native CPython libraries will work on it.
The problem is that this is where basically all the interesting language-design research is. I wouldn't use Rust for the syntax; I'd use it because I want properties like complete memory safety, manual control over memory allocation, easy (and fast!) access to C libraries, and short startup time. These are all things that Truffle explicitly does not deliver.
It's a great tool if you work in the JVM ecosystem and want to play around as a language designer. But most of the interesting languages lately have created their own ecosystems, and they succeed by solving problems so fundamental that people will put up with having to learn a new ecosystem to gain the benefits they offer.
On IBM i the language surface is RPG, Cobol, C, C++ and Java.
On other ones you could eventually consider the micro-coded CPUs as a kind of cross language VMs.
"Little languages" can have significant advantages over APIs in terms of flexibility, optimization, and conceptual fit with specific domains. Why not create infrastructure to not only de-duplicate work creating tooling for languages while ensuring everyone always had 1st-class tooling and making "little languages" as easy to create as APIs?
Except that it's not really exposing much of the JVM from what the article mentions, it just uses the JVM as a base. That is, Truffle languages can talk to other truffle languages, but I'm not sure how easy it is for a truffle language to talk to a JVM language.
From everything I read, it seems like people thought Parrot was a good idea so decided to rewrite/reimagine it using the JVM instead of C as a base.
As for not moving in the right direction, it was probably far too ambitious to try to develop something like Parrot and something like Perl 6 at the same time, with dependencies on each other. Both projects would have been better served if they did their own things, and Parrot was just seen as "a good candidate for a second implementation" from the Perl 6 perspective, rather than the primary target. It's hard to target a VM that's not done, and it's hard as a VM to support a language that isn't finalized (and has crazy requirements).
It's perfectly possible and manageable to create cross-language VMs and to translate pretty accurately between languages. However, what breaks down, as you alluded to, is that people expect the standard libraries, and the libraries of others, to work interchangeably, and that is orders of magnitudes more complex (mostly because you are, by definition, working closer to the metal).
So, I need to ask, how much research is being done in giving the standard library as much consideration as the language? Do we have a grasp on what it would take to treat the stdlib as a "first class" citizen?
One of the solutions to this problem could be having package management in your language be a first-class citizen, then treating each part of the stdlib as a separate package. If using stdlib code is just as simple as using 3rd-party libraries, then why need a stdlib at all?
Without a decent stdlib, you end up with the pre-STL C++ situation, where every library & framework declares its own string, vector, and hashmap classes and you need slow, verbose, and error-prone routines to convert between them. The main benefit of having these in the stdlib isn't that you don't have to re-implement them, it's that every third-party lib knows exactly which class represents the concepts of text, sequences, and dictionaries.
The same goes for many other types, eg. Promises, Paths, URIs, Dates, Input/OutputStreams, Iterators, etc. Indeed, when one of the stdlib types is misdesigned (eg. java.util.Date), you get an utter mess as the community converges on an alternative (eg. JodaTime) and then the alternative API is folded back into the stdlib (Java 8).
Luckily, we're learning just which types need to be in the stdlib and which can be outsourced to a third-party package manager. In particular, a lot of serialization/parsing formats are better off in a package manager, as long as the language defines an annotation system that can be used to define which fields must be saved. And giant systems like webservers, webframeworks, or RPC formats are often better off as a third-party library.
> I wouldn't use Rust for the syntax; I'd use it because I want properties like complete memory safety, manual control over memory allocation, easy (and fast!) access to C libraries, and short startup time. These are all things that Truffle explicitly does not deliver.
Truffle itself does not do it, but it is possible to have a Truffle interpreter with these requirements. Memory safety can be enforced at the frontend compiler level like Rust does I believe. Memory can be allocated directly with Unsafe or calling libc functions. GNFI [1] allows for fast calls to native libraries. Startup can be improved radically with SubstrateVM, but it is currently closed-source. There are many ways to improve the JVM startup, but not necessarily convenient (but then you don't need to wait to compile your program either to start running it).
Pretty much every professional job I've had used polyglot programming of some sort - when I was in financial software it was mixed Java/C/Fortran numerics, when I founded my first startup it was polyglot Javascript/ActionScript, and at Google it was a large server that was first Python/C++ and then Java/C++. The boundary between languages is always problematic. And the reason for that is because you choose memory layout to make certain trade-offs around the access patterns for that data. Do you inline data structures in a vector, or chase pointers? Do objects carry their type information with them so you can manipulate them dynamically, or do they throw it away at compile-time for greater efficiency? Do you get O(1) string indexing or native UTF-8? Do you allocate on the stack or the heap? Do you copy, borrow, move, or COW?
And then when you go to switch representations to call into another language, you pay a cost to convert all the data structures you might be touching. In many cases, that cost could be more than you saved by using optimized representations in the first place.
If you already buy the JVM's premise of "just let the compiler do it, we'll figure out the most efficient representation", then this is a non-issue. But there are still programmers out there who believe that Rust or C++ or Go or CPython or whatever presents a better memory layout for the tasks that they wish to accomplish, or language designers who think they can do better than all of the above, and these are the users that Graal+Truffle is trying to win over. What's the story for them?
Sorta like SWIG++? If you could do SWIG but never require that an end-user write a typemap or debug a crash themself, there'd probably be a big market for that.
Not actually true. `ByteBuffer` lets you read integers from arbitrary offsets, which is all Cap'n Proto really wants.
An example is JRuby+Truffle where in its C extensions, pointers to Ruby objects act to the C code as if they are pointers to MRI data structures, but behind the scenes Graal compiles code that accesses them like the JRuby objects that they are.
The C-memory to Java GC transition doesn't work as well, but you can get around that by implementing C-memory using the Java GC like the Managed-C paper does, alternative implementations only have to keep the same behaviour, not the same underlying data representation.
What if it's not the C extension in your project that does the allocation, but a closed-source third-party library that your C code calls into? Do you need to recompile all source code to generate the appropriate typemaps?
> The JVM works if your language is mostly like Java (Java, Clojure, Scala, Kotlin).
I'm really curious why you're characterizing Clojure as "mostly like Java"?
LuaTruffle appears to implement the syntax of Lua but none of the semantics. For example, no metatables and no coroutines. Lua is unparalleled in its support of coroutines, and metatables are what make optimizing Lua _difficult_. Without implementing either of those things you've neither demonstrated the ability of the framework to implement novel control-flow semantics (asymmetric coroutines that can yield across multiple, unadorned function invocation) nor performance capabilities (JIT-optimizing Lua tables is non-trivial).
I'm not even sure LuaTruffle implements Lua's lexical closures or tail call optimization, both critical to Lua's ability to support functional programming patterns.
Of course, it definitely doesn't implement the Lua C API, which is part-and-parcel of what makes Lua preferable as an extension and glue language. But I was willing to overlook that if it could easily implement the former.
The beauty of a good DSL isn't the syntax (I know, hard to believe!), but in novel approaches to code flow execution and other deep semantics. Golang's goroutines are beautiful. Rust's ownership analyzer and prover are what _defines_ the language. Haskell's lazy evaluation open up whole new dimensions for problem solving (and headaches). If your language framework doesn't at least preserve the ability to implement those things cleanly and performantly, it's not adding much value and is basically a toy. It's not like writing a lexer for a DSL is a serious impediment. (Using Lua's LPeg, for example, you can write parsers and build an AST for most major languages in a few hundred lines of code.)
If you want to know about how well we can implement the semantics of an existing language then look at JRuby+Truffle - it passes more Ruby language tests than any other alternative Ruby implementation. The only stumbling block is co-routines as the JVM doesn't have these and they're hard to implement efficiently with other constructs.
That's a pretty critical feature of Lua that somewhat defines it(along with it's low overhead + embeddability, both of with you won't get with the JVM).
It seems like engineers always want to build One Tool to Solve Them All(tm) and yet there are always tradeoffs to be made. The reason 90% of this industry is still employed is because we don't have these one size fits all problems that we'd so love to solve.
SubstrateVM solves the embeddability part.
For context we use to run VM + tuneable data + game logic code in a 400kb block allocated to Lua on the PSP. I'd love to hear how SubstrateVM compares. Lua has a really rich history of being embedding in some pretty small targets.
Some of this is just simply a design trade-off. E.g., in JRuby+Truffle, we've implemented much of the core library in Ruby. This has allowed us to achieve a high level of language compatibility in a fairly short period of time -- implementing 3,000 core library methods in Java would be quite the undertaking. As a consequence, the static binary must include those full Ruby sources (compressed, but they're not in an optimized bytecode format).
Having said that, if we wanted to optimize for size over a restricted subset of the language (think mruby), that would be straightforward to do.
Except it really hasn't. The industry is going to use whatever language has the bare minimum feature set they value at the moment and no more.
Call me cynical but my point of view is: if the industry were really striving for perfection, there would be no COBOL or BASIC, for instance. Lisp had already been invented. Garbage collection was a thing already. We had macros. A REPL. A little later, OOP and multiple dispatch. The list goes on.
They had to ditch the (almost) perfect wheel for a cheaper square one. Fast forward a few years, and the square wheel is now polished but just as square, the insanely great wheel now just is as cheap as it is, but we won't use it, as the axles expect the square wheel.
And then we invent Java and XML...
COBOL was a huge improvement at the time, an innovation in readability. BASIC is a good demonstration of why the platonic ideal language is wrong: it was designed for beginners (that's what the acronym stands for), and it did a reasonably good job at that. Beginners might be overwhelmed by OOP, multiple dispatch and a list of design patterns. Sometimes experts are too.
Garbage collection is a nice feature, but it doesn't fit everywhere. Sometimes you want to manage your own memory. That's why we have more than one language.
That's often my point when people bring up arguments against a language that seems a bit more complex (Perl), or for language simplicity (Python). Some complexity falls into a sliding scale where there is more cognitive load while learning, but it pays of in the every day usage. The siple example of an extreme end of this is APL. If you can internalize and understand the extremely concise and powerful syntax and semantics of the language (I haven't), you can do amazing things very quickly[1].
Now, doing amazing things very quickly isn't the only metric by which we judge a language, but it is a useful metric, and it's probably worth having a language on that end of the spectrum in your tool set.
Not to mention figuring out how to type APL -- that's where I got stuck.
I think quite a lot of complexity falls into this scale, and the quest for simplicity - or rather, "user-friendliness" - in computing is utterly misguided. The difference between a toy and a tool is that a tool exchanges more upfront learning requirement for ability to get more done. Not just more efficiently, but more. We're doing ourselves, and the world, a huge disservice by expecting that everything - be it a website, an IDE or a programming language - should be able to be mastered in first 5 seconds of exposure. The only way to achieve that is to dumb the product down.
In other words, we need more learning culture (or even RTFM culture), less "user-friendliness".
Sure. But these systems were created from the ground up, every single time. Given a good foundation, which could be Lisp or something even better, you can whip up the "simpler" languages in no time. Even ditch the parentheses if you so desire. While getting all the benefits of the underlying system.
> Garbage collection is a nice feature, but it doesn't fit everywhere. Sometimes you want to manage your own memory. That's why we have more than one language.
Sure. But I disagree with that's the reason why we have multiple languages. There's no single reason, it is probably a mixture of preferences, previous knowledge, budget, time to market (JS!) among others.
We can manage our own memory from higher level languages. We can compile them. We can even make them spit out assembly custom-made for a given, niche application (see the Galileo magnetometer patch).
Sorry if I sound like a smug lisp weenie. But even that description is not accurate: had we focused our collective resources towards "perfection", as that sentence implies, we would have something way better than Lisp itself. Or anything else that we currently use.
It is as if we had invented jet engines before planes. But then placed steam engines on them because they were cheaper. Or that more people understood them.
Good programmers can write good code in any language, lousy programmers will write lousy code in every language. Malbolge might be excluded from that, but with a decent macro pre-processor, even brainfuck can be workable.
Times have changed now? Sure, but history didn't wait.
https://en.wikipedia.org/wiki/PreScheme
It's a Scheme created as a C alternative for low-level programming used in a verified Scheme48, VLISP. It was quite efficient. LISP's doing more of that kind of thing might have fueled more use of the language.
Lisp usually required expensive mainframes and workstations.
Of course C got the adoption of the 80's hipsters with such price tag.
In practise, that foundation (for many programs) is Java byte code.
Which is probably why a technocracy in which every voter must pass a fact-based proficiency test before being allowed to vote might trump democracy. Not sure, just wanted to chime in with being cynical. :/
Why would that group not use that to their advantage, instead of to the benefit of everyone?
> The majority never picks the best and markets do not optimize as much as some people think. Whether it's movies, arts, books, music, politicians, keyboards or programming languages, the most popular choices are practically never the best ones.
The question is not whether the market picks the optimal outcome, but whether a committee of experts would do better. The experience of central planning within the Warsaw Pact should hint at the answer...
The experience of the entire Internet seems to suggest a different answer, so it's not that central planning is always bad. Note that the best, most reliable, most stable parts of the Internet were invented and codified long ago, when there weren't many people on-line and there wasn't that much commercial interest in it. Today we can rarely if ever agree on any kind of protocol or standard, and if we do, it's usually a huge bloated mess.
So the question becomes, how do you know whether you're going to get a huge bloated mess or the Internet, and how do you influence the outcome?
As the number of potential 'experts' increases and the number of people involved in choosing them increases, you get something that looks more and more like a market, but without price transparency...
Also, the facts can be discussed openly and if there was a problem with some of them, there could be a system of revoking or modifying the set of questions similar to the recounting of ballots.
For the committee of experts, that was your idea, I never suggested that, because it obviously makes no sense.
The majority and optimizing markets are at odds in your statement. Look at qdb/k, it is very expensive, programmers cost a bunch, but it is used by the big financial houses. I am sure they are glad it is not too popular and affordable.
I do agree that I don't like central planning, and design by committee, but those are different than your market statement or technocracy comment.
Movies, arts, books, music...horses for courses, best for what moment? I like action films, I like high-brow, arthouse stuff too. Same with books. I am glad I have a choice for the moment at hand.
When I was a young intellect, I succumbed to that elitist attitude of 'the best', and 'they don't know what's good for themselves' which is funny because I grew up in a poor, working-class neighborhood in Brooklyn in the late 60s/early 70s. Elitists were the guys from NY as well, but who wore black berets to film class at TSOA/NYU (three in my class alone!).
Lisp was slower than C when speed mattered most. If you can't see why that would be an issue, I suggest trying to write a game for a home computer from the 1980s.
Parser [1] shows how lack of sum types makes code messy. Lots of "factory.createStringLiteral" kind of calls. At least Java has a mature parser generator.
Implementing interpreter nodes [2] requires a pretty large amount of boilerplate, each individual node is relegated to its own file, some classes are littered with probably autogenerated getters/setters. While functional languages can be too terse, Java shows its exceptional verbosity here.
[1] https://github.com/graalvm/truffle/blob/master/truffle/com.o...
[2] https://github.com/graalvm/truffle/tree/master/truffle/com.o...
I don't cut slack for Obj-C on iOS either.
I've had great fun with two products for Android/iOS and Mac/PC/Linux:
[1] Godot game engine. I followed the introduction and had an app on my Android phone in 10 minutes. A little hack in the background, but hey, for gaming it is great.
[2] 8th - a dialect of Forth - you write Forth-like 8th, and voilà it's running on your Anroid, PC, Mac, iOS device!
Oh, and I've heard React Native's not so bad for mobile.
* Very customizable interface * Lots of tutorials on YouTube and elsewhere * Good examples provided * Script is very similar to Python, so not too hard to pick up, and again, lots of examples. * Easy packaging for Android, Windows, and Linux on my end * MIT license, commercial or non-commercial * 2D/3D and live debug and scripting on the fly
Really worth trying out, since it is easy to install and dive in.
I'll have a look at React Native, but I am not a JS programmer, and all of these 'hot' JS libs, modules, etc... seem more cluttered than plain old JS. I think if I were to branch out, I'd try Purescript instead.
It's probably more worth checking out that most of what's out there.
I would love a lightweight toolkit that made for easy language development. VMKit[0] is dead, MicroVM[1] is a nice idea but not full fledged. Many like myself looking to implement a toy language would love to be fast out of the gate and would rather not mess with the OS-specific constructs and specifics of LLVM IR. So we end up cross-compiling to existing languages or marrying ourselves to a certain runtime (e.g. the CLR). Any other lightweight VMs or AOT compilers that are cross-platform that language designers can use these days?
Some tangential things mentioned are proprietary though.
It may be unlikely but Oracle can simply remove highly paid staff from above mentioned projects and put in some closed source alternative/analogous projects. Open source project may then be left to rot. As it is almost every single developer on Graal etc is Oracle labs or Oracle funded research group at university.
Also we still have deployments using EOL Java versions that don't get upgraded, because IT doesn't want to change those servers.
Previous discussion on HN: https://news.ycombinator.com/item?id=11555527
I agree there is a lot of interesting stuff going on here but lets not forget the prior art. Even OMeta I think already did a lot of these things way back when. Going further back there's META II (https://en.wikipedia.org/wiki/META_II).
I think building self-specializing interpreters is the main trick. If that becomes easy then a whole bunch of other magic is also possible. But I'm just an amateur so all of this is still very impressive.
There's a nice paper comparing the two approaches and their relative merits:
http://stefan-marr.de/papers/oopsla-marr-ducasse-meta-tracin...
If you want to write a language that is semantically like C for the most part, than go ahead and use Truffle, but that's not where interesting language design is happening. Show me that Truffle's Ruby implementation hasn't removed callcc (yes, Ruby has callcc), and maybe I'll reconsider.
Java is my main lang and currently I'm working on my hobby project - implementing Scheme r5rs in Java. That is hard! And I doubt I'll be able to implement full r5rs even close.
I have no idea (yet) how to implement Continuations: In theory yes, you can implement Continuations in any lang which has Exceptions (the only way in Java to unwind a stack).
Like: https://www.politesi.polimi.it/bitstream/10589/108685/3/2015...
Proper TCO:
AFAIK it is impossible in general case in object-oriented lang. That is one of the reasons why Rich Hickey decided to have explicit recur in Clojure,
Yeah, you can implement your own stack and stop using Java's stack, but then you lose performance and interop: "Interop, speed, TCO -- Pick two." Rich Hickey
See https://news.ycombinator.com/item?id=4922848
Then Full Numeric Tower is hard! Mostly due to the fact that Java's type system is not strong enough.
I think Kawa is the best implementation of Scheme for JVM you can get now.
But even Kawa has some limitations (due to JVM limits): https://www.gnu.org/software/kawa/Compatibility.html
Are you still using Java stack, Java calling convention, still able to do Java interop?
When you are writing your own language on top of Java (Scheme, for example), you have full control of your lang AST (S-expressions, for example). How does it help to implement TCO and continuations (assuming you don't want to lose Java interop, Java calling convention, Java stack and performance)?
Now, as you point out, you lose Java interop that way. However, you can imagine using Truffle + Graal to also build a similar interpreter for Java. For interop, your interpreter 'simply' has to merge in the AST for the Java interpreter (with its own 'execute' methods). As I understand it the RubyTruffle project uses a similar trick to do C interop.
http://cesquivias.github.io/blog/2015/01/15/writing-a-langua...
ZipPy (an incomplete Python implementation) supports Python's coroutines, which are faster than any other implementation. In theory these might be directly generalizable to get continuations.
Let's say you're in a particular function at a tail call node. Since the tail call node is semantically (from the point of view of the Java program) many function calls deep into the implemented language's function and we want to get back to the start, we want some way of exiting a whole stack of function calls in one jump.
Exceptions offer a simple and neat way to do this. You throw from the call site and catch back at the start of the function. Graal will inline all of the internal function calls up to this point, since Graal uses inlining extremely heavily. This means that Graal will see the throw site and the call site next to each other in one block of code, and consider it to just be a jump.
Because Graal is just inlining Java functions, and the actual target language's call stack is stored separately as a particular object, the target language's calling convention for a large part doesn't matter. The jump to the first node doesn't invalidate any of the target language's stack - that has to be done by the language implementor manually at that point.
"For example, the TruffleJS engine which implements JavaScript is competitive with V8 in benchmarks. The RubyTruffle engine is faster than all other Ruby implementations by far. The TruffleC engine is roughly competitive with GCC."
It is not surprising in sense Java is competitive to C/C++ as per many Java developers.
In many use cases, C++ might win the benchmark game, but the end user will not notice any difference in human time.
The biggest issue is of course than one needs to learn how to optimize code, use the right algorithms and data structures in first place.
I haven't worked with a single Java based GUI application that did feel smooth, fast and efficient.
I wonder why that is.
Additionally Swing has bad defaults, so it requires effort to make the required set of calls to make them look better.
Back in the Sun glory days, there were Sun blogs like Filthy Rich Clients, which lead eventually to a book http://filthyrichclients.org/.
It was a consequence of Sun not getting what it means developing GUIs for the consumer systems, think that their Solaris UIs (SunView, OpenWindows, CDE) weren't that great in that area.
So most Java developers stick with the defaults and as such the majority of Java based applications send the wrong message.
You still have to GC at some point and when that happens chances are you're going to drop frames. Unless you're very aware of the garbage you're creating you'll take more than ~5ms which is usually enough to push you over 16ms with the other work that goes on during a frame.
Dalvik is well known on Java world for having a JIT and GC implementations that worse than what most commercial embedded JVM are capable of.
Things have improved with ART, but even there there are quite a few performance improvements that Google could eventually do.
Soft real time Java GCs for embedded devices are being used in ground station controls of missiles and a couple of US Navy weapon systems. They surely don't want GC glitches in battle situations.
The Android team seems to only bother with "good enough" in what concerns Android performance.
As a side note, just check how many Android releases are they going through and yet real-audio support isn't quite there. Even Windows Phone has better support for it, and they do have WinRT and .NET on them.
No they don't. Point me to one windows phone audio app that has low latency audio.
https://blogs.windows.com/buildingapps/2014/05/15/real-time-...
https://rtaudiowsapps.codeplex.com/
https://channel9.msdn.com/Events/Build/2014/3-548
And I just found this, but not sure how good it fares.
https://www.microsoft.com/en-us/store/apps/audio-meter/9wzdn...
Sometimes being good at something isn't enough.
I am no real time audio expert, but as someone that does work on Windows, I saw the information I have posted back when it went live.
So it is possible, that in spite of the support being there, no one cares about it due to the sad state of the Windows Phone market and not because they suck.
Or I suck at understanding how real time audio is supposed to work and the support isn't really there in spite of what Microsoft says.
Actually, there is. As of Android 6.0 Google's CCD includes a section for Professional Audio devices
If a device implementation meets all of the following requirements, it is STRONGLY RECOMMENDED to report support for feature android.hardware.audio.pro via the android.content.pm.PackageManager class.
The device implementation MUST report support for feature android.hardware.audio.low_latency.
The continuous round-trip audio latency, as defined in section 5.6 Audio Latency, MUST be 20 milliseconds or less and SHOULD be 10 milliseconds or less over at least one supported path.
If the device includes a 4 conductor 3.5mm audio jack, the continuous round-trip audio latency MUST be 20 milliseconds or less over the audio jack path, and SHOULD be 10 milliseconds or less over at the audio jack path.
The device implementation MUST include a USB port(s) supporting USB host mode and USB peripheral mode.
The USB host mode MUST implement the USB audio class. If the device includes an HDMI port, the device implementation MUST support output in stereo and eight channels at 20-bit or 24-bit depth and 192 kHz without bit-depth loss or resampling.
The device implementation MUST report support for feature android.software.midi.
If the device includes a 4 conductor 3.5mm audio jack, the device implementation is STRONGLY RECOMMENDED to comply with section Mobile device (jack) specifications of the Wired Audio Headset Specification (v1.1).
They even had Samsung on stage to talk about their Android extensions for real-time audio.
So with Android 6.0 barely over 10%, still with issues fixed in Android 7.0, it is as I said "real-audio support isn't quite there".
If the program runs on the user's machine, I'm pretty certain the users WILL notice the difference, memory wise.
Previous discussion
I've had the pleasure of discussing some of his jRuby work with him in the past (I occasionally plod along with own Ruby compiler in Ruby - it's nowhere near complete) and they've done a lot of impressive work to demonstrate how compiling Ruby into fast code is largely about assuming people will behave sanely (e.g. people won't usually give Fixnum#+ side-effects, or make it return weird stuff), but add fallbacks for when they do.
This is the big challenge with Ruby: The subset of Ruby that people actually tend to work in is largely very predictable and possible to compile efficiently, but there's a lot of stuff around the fringes that people expect to work on the very rare occasions that we use it.
JS VMs also tend to have multiple tiers for that reason - interpreter, baseline, and full JIT, etc. - while AFAIK the Graal+Truffle/JVM approach has just 2 (which is optimal for Java, but not for startup times of JavaScript and other dynamic languages).
There also are projects underway to bring AOT compilation / caching of compiled code to the JVM. Right now they're only available to commercial customers due to their experimental nature and the needed support but if I remember the talk correctly they're supposed to be added to openjdk eventually
Instead, dynamic languages tend to do tiering, which in principle is something Graal+Truffle could do, but it might be a lot of work.
[1] Nitra - https://github.com/JetBrains/Nitra
Also, anything from Oracle which is partly open and partly closed is scary. They've done that with Java and MySQL, which drives people away from both.
However, I assume one goal of the project is to make the JVM more competitive, which it certainly does.
Xamarin. iOS too, although that depends on what you mean by 'recent'
Didn't a lot of Xamarin get open sourced after the acquisition? Swift is also open source, although Xcode isn't.
>Swift is also open source,
Interesting that they've gone as far as running C extensions "internally", making it easier to run existing ruby apps without running up against the barrier of a native gem and having to change the code.
...unfortunately. I see PLs as mere material - sure, you can improve on them but at far more important is how we architect our systems (the PL-independent ways we create and organize our systems into interfaces and components) is where I see the software practitioners of today flailing - and no PL is going to save us there.
We need to have a better way to analyze systems on these architectural measures and a better way to train people to build better architectures, not more PLs.
That's only part of their point. It's as if we were in the building industry, and we needed new tooling for each specific material. When I'm at a hackspace, the same chop saw will let me cut wood, delrin rod stock, steel linear rails, and aluminum extrusion. The same drill press and bits will operate just fine on most of the above as well. What we have in the programming world is a situation where I'd need a different toolset for every material I'd listed above.
As a Lisp programmer, I regularly create new languages in a matter of hours.
You can do that pretty quickly in IO language (admittedly, that's not Java)
You can. Reader macros let you put whatever syntax you want on top of Sexprs. For example:
Welcome to Clozure Common Lisp Version 1.10-r16479M (DarwinX8664)!
? (spark-init)
#P"/Users/ron/devel/spark/ergolib/init.lisp"
? (require :parcil)
...
? (in-readtable infix)
|NIL|
? infix(x=1.23+4.56)
5.79
? sin(x*x+1)
0.03341171
?
> Lisp can only metaprogram its own syntax, you can't introduce a C-like syntax.
This is a flat-out false statement, and it reflects a deep but sadly common misunderstanding of how Lisp works and why it's cool. Even if Lisp did not have reader macros as a standard feature, you could still write a C compiler in Lisp more easily than you could write one in any other language. The whole concept of "metaprogramming a syntax" (so that you can talk about whether or not a language can "only metaprogram its own syntax") is non-sensical, a category error. Metaprogramming is simply writing programs that write programs. What makes Lisp cool is that it separates the syntax from the program. A Lisp program, unlike all other languages, is not text. A Lisp program is a data structure. Lisp happens to define a textual surface syntax (S-expressions) that allows you to easily convert text into the particular data structures that are Lisp programs (and also incidentally data structures that are not Lisp programs) but you can also produce these data structures in other ways, like, for example, writing programs that produce them. The textual surface syntax is a detail. Writing an infix parser in Lisp is an elementary exercise, and you could use that parser to parse C-like code whether or not you had reader macros. All reader macros let you do is seamlessly integrate that C-like syntax into the Lisp REPL rather than having to embed your new language in strings or read it from files.
On a different note, would you say that Lisp's syntax is better because it makes an easy compile target (i.e. it's easier to compile C into Lisp instead of x86) or because it makes writing macros/AST transformations easier? I ask because my biggest beef with Lisp has always been that I thought its S-expressions to be less readable than other languages, but if you're arguing for Lisp's value as a compile target, then that makes a lot of sense.
Not quite. Lisp's syntax is (mostly) irrelevant. Lisp's design is better because you don't have to worry about syntax to use Lisp as a compile target.
There's a huge conceptual difference between
(eval '(some code))
eval("Some code")
(eval (cons (quote some) (cons (quote code) nil)))
(eval (some-function-that-generates-code))
There's no reason a regular macro couldn't implement a basically C-like syntax (though it would have Lisp, not C, tokenization rules, to the extent that those are different) on what is passed as its argument. The actual call to the macro would have typical Lisp syntax, but what the macro consumes would be interpreted with whatever syntax was implemented by the macro.
(And, a reader macro could implement more deeply C-like syntax.)
Which is why I said normal (rather than reader) macros could do a "basically C-like syntax" but with Lisp tokenization rules.
((:include "stdio.h")
(const char * message = "Like this.\n")
(int main ((int argc) (char * * argv))
(printf message)
(return 0)))Using a parsing framework (e.g. parser combinators, parser generators, ometa, etc.) to parse a language which has a well-specified syntax (e.g. a BNF grammar) is pretty routine and mechanical; usually it just requires a one-to-one translation of the BNF form into the parser framework's syntax, then fiddling with ordering and precedence rules until your tests pass.
Starting from scratch, I could knock out a C-like parser in maybe half an hour, e.g. using parsec or ometa. I'm sure Lisps have equivalents (I know Racket has built in support for defining new concrete synax)
This will, however, not "play" like C without a great deal of work. You will be dragging an entire Lisp VM around and have to shake all of that out to actually produce something as lean as a C program at the end. You're probably best suited to take the Lisp compiler and build a "independant code" generator to generate binaries that don't rely on Lisp. But I'm waving my hands, you'd need to talk to the SBCL or CCL development teams there.
Implementing C as a parser that builds s-expressions followed by a collection of macros to translate it into Lisp is certainly possible; I did it in 1984. I wouldn't say it was trivial, though; nor was the performance anything special (even then).
I will definitely take a closer look at Graal and Truffle.
This looks... backwards?
I guess some people from the Java world could find something useful there, but it's very unlikely to attract anyone else.
[1] https://lafo.ssw.uni-linz.ac.at/pub/papers/2016_PLDI_Truffle...
Consider all the improvements to the above software that came after Oracle's purchase of Sun. There was a boatload of them and they keep coming. Oracle is paying for that.
They are all free, and that is because of Oracle, not because of Sun as Sun does not exist anymore.
you have sizable groups of people who work like one team, one person submits and others up vote
In the Digg days I was approached by some guys in Eastern Europe who offered to get me diggs for $$. I declined. I was hoping HN is harder to game, but I don't see any evidence that support that.