Racket-on-Chez Status
blog.racket-lang.org
blog.racket-lang.org
In Common Lisp, one has pthreads-like concurrency capabilities which seem very well suited for fine-grained parallelism as well as for server tasks. I am thinking in such things like parallel backtracking and optimization algorithms for robotic control or board games, for example.
However, threads are notoriously difficult to handle cleanly and safely in larger programs. Here, Clojure (which in my eyes is a very Scheme-like Lisp) offers a very elegant solution with its concurrency primitives of Futures, Atoms, Agents, and STM. After some experimenting, I believe they are very, very attractive for many concurrent server tasks (like a web server), but not that well suited for computing-intensive parallel algorithms like the ones I mentioned above, which I am highly interested in. Part of the reasons are that such algorithms can become quite GC-heavy. Also, the Clojure compiler has limits on how much primitive types can be passed as parameters in one functions. This means that a complex backtracking algorithm in Clojure can still be two orders of magnitude slower than in C (which is somewhat disappointing, but one has to remember that Clojure was not designed for this).
Racket has had, so far, only limited concurrency capabilities. It had Futures, however they could easily become blocked by GC. It also has Places, which are a very safe and clean solution of splitting parallel computations into separate processes. However, I think that places are not the first choice for heavily parallel algorithms with strong interdependencies.
Now, Racket can run on top of Chez, and Chez has fine-grained concurrency capabilities on top of pthreads, which seem to be on par with Common Lisp. Also, Racket has strong support for functional and side-effect free programming, including some data structures. In my impression, this seems to open a wide range of new possibilities, including providing look-alike primitives for Clojure's Futures, Agents, and Atoms. I would be very interested to know more whether this impression is correct.
[1] https://www.gnu.org/software/guile/
[2] https://www.gnu.org/software/guile/news/gnu-guile-300-releas...
For example ephemerons, that need some magic to cooperate with the garbage collector. https://cisco.github.io/ChezScheme/csug9.5/smgmt.html#./smgm...
It seems that the one of the motivations for starting this effort, besides performance, was to move away from a C code base. Is Chez Scheme primarily written in Scheme?
What that other Scheme is based on is a secondary consideration. Chez Scheme is also said to be implemented very well, however.
About the only notable thing with it (apart from inspiring the target for Idris 2) is that it implements the C FFI, so it will handle Idris programs made to target C.
Not that it wouldn't be possible to make the C code fast. It is just that the chez backend was a one man job.
Actually, Gerbil used to be written on Racket. As you can see, it now benefits from Gambit's strengths without having to re-implement them...
Racket was a great tool to teach people lisp and initiate them into that paradigm.
Is the main syntax still good for long term use?
And even in the original announcment it was stated " * `#lang racket` is not going away and will always have its current parenthesis-oriented syntax. In the same way that Racket still supports `#lang scheme` and `#lang mzscheme` and even `(module <name> mzscheme ....)` and even top-level programs, the Racket compiler and runtime system will always support `#lang racket` programs. We believe that Racket's `#lang`-based ecosystem makes it uniquely positioned for trying new language variants while preserving and building on our past investments."
They have only made that statement stronger since then.
Renaming doesn't help the case much here, if Racket's future is abandonware.
From there:
Phase 1: Brainstorming (months)
Phase 2: Iterative Design (years)
Phase 3: Conversion (months or years)
Phase 4: Transition (years)
I am a nobody in front of people like Matthew Flatt, but this feels like Seconds Systems Effect taken to its extreme definition: https://en.wikipedia.org/wiki/Second-system_effect
Plus as a former Perl programmer having watched Perl 5 lost almost everything, chasing a never to have come to reality Perl 6. I can say, given all this, the future of Racket as a language, for its core uses and users is pretty much dead in the years to come. Like dead totally. Racket doesn't even have as much the share of dev mind share or a resource like CPAN at its disposal. Languages like Racket will fade away into oblivion a lot more quickly.
It can be hard to see this in all the enthusiasm in the early stages of the project. But when you embark on multi year project journeys. People's priorities change, people get into health crises, lose jobs, move on to better projects, recessions happen. Core teams that started these projects change so much, newer one's pretty much give up and see no point in it after a while. Many people prefer a perfectly working tool improved over time, than a pie-in-the-sky idea that will never come to see the light of the day. While all this is happening, your existing language, libraries, dev mindshare, tooling suffers. Merely supporting small time fixes means nothing, because no one likes to use a tooling in hospice care. Your core dedicated set of users, who did most of the evangelism for your cause move on to newer tools and languages, for the obvious reasons that they don't see their future with existing tool they like, the newer one is taking for ever and isn't even the same goodness as the current one. Once you lose those users, the newer users you wish to attract from the crowd of Java and Python programmers won't even bother to try, why should they? They have something proven to work in production with tooling, libraries, books, and production success stories for decades.
That is when you realize you lost both your old and new language.
Racket was awesome for what it was. An awesome lisp, a top successor for Common Lisp and a playground for experiments and initiating people into Lisp.
It's not late. Probably it's time to stop right now.
Net result is a stagnated Racket, and a non-attractive Rhombus.
Plus C based languages have their winners already. Racket is a kind of a winner in the Lisp family. With Clojure it makes the only other choice. Now if you change the syntax, you are going after users who don't really care what you have to offer, at the same time, you are taking away what your existing users like.
Racket is a language with first class support for building languages on top of it. And so no, Rhombus does not need to be a "successor" to Racket. It does not need to "replace" Racket. It will have all the current Racket goodness to begin with, because all of its semantics will come from Racket - they will be the same language. Nor will it steal valuable developer attention - a bug fixed in Rhombus is a bug fixed in Racket, anywhere except the syntax parser. They will share all core libraries. The whole point of Racket is that #langs are mutually compatible. And the stress test of using Racket in earnest for what it is actually designed for will make Racket better, not worse.
Consider PyonR [0]. This is a similar and yet much more ambitious goal than Rhombus - Python as a Racket #lang, with 100% compatibility with Racket and Python including external modules. And yet, it's a one-man project, written as a thesis and now pretty much abandoned. Writing Rhombus, a mere surface syntax for Racket, should be much easier. The hard part isn't writing it, it's deciding what it should be.
The Racket team are the only ones that I can see who are taking all the "write your own DSL" Lisp hype and actually trying to apply it in a structured way.
This is how it rings to my ear. Racket as is, is in some way frozen when it comes to features and big changes. Rhombus is where the next set of awesomeness will happen. Lispers who happen to be core users of Racket have no use at the time for non-existent Rhombus, and its not Lispy so their whole use case ends here. Racket won't be continued to developed in the same breath as Rhombus, so Racket is also dead here. C based language devs have a lot of good stuff already, so they don't need Rhombus either.
Eventually this could lead to the exact situation Perl was in the 2000s. The core development was frozen for 5+ years. The language usability just fell year over year. Perl 6 took forever to even start with the implementations. Pugs, then Neitza, then Rakudo. Eventually they came to a point, where they announced Perl 5 development would just run parallel as a separate project compared to Perl 6. By then a lot of dev mind share was lost to Python/Java. Now neither Perl 5 nor Perl 6 have significant mind share. Perl 5 certainly has lost a lot of its users.
Also in general trying to be everything to everybody is a bad idea in general. Ask Parrot VM people. It doesn't work the way you expect. You won't solve anyone's specific problem, and it takes forever to get done.
You still seem to be misunderstanding how Racket and #lang's work. Rhombus and Racket will be the same language, but with different parsers. Big features of Rhombus will be added by adding those features to Racket.
Languages in Racket are not developed independently of each other, they all share the underlying language machinery.
Exactly the point, one will have to bring Rhombus to Racket's feature parity, which by the link I posted, and the Racket people themselves acknowledge will take several years to happen. That much dev resources have to be dedicated to build a new language which will have the same feature parity as Racket. This effort could be expended to taking Racket forward. This is non trivial effort.
At the same time, you are hoping to attract users from Java/Python land(Who very likely won't come, given where your language will be(absence of killer libs, frameworks, books, q&a support etc)). While turning away existing lisp users. This ultimately becomes a disaster.
One of the biggest selling points of Racket was that it was continuously being worked on, unlike CL whose spec is frozen(putting it mildly, the real word would be abandoned).
Looks like Lisp has a lot of self destructive tendencies than anything else. Competition seems to be non existent. But Lisp communities just can't agree to work on things. A super massive pivot that just won't fix any real problems to existing users, adding very few incentives to new users, plus act as a resource drain, and cause your perfectly fine existing language to stagnate.
https://news.ycombinator.com/item?id=20503742
A similar argument is made in the above thread.
Regardless, of all this serious software also commits itself to things like stability and takes backwards compatibility seriously.
The idea is that the libraries written in Racket will be usable in Rhombus and that the libraries in Rhombus will be usable in Racket. Racket already ships with multiple internal languages, so one more is not a big difference. [2]
Note that some constructions that are special cases that must be implemented by the language are just libraries in Racket. For example most of the implementation of `for` and `match` are in their own library, and even more weird things like at-expressions is just a library https://docs.racket-lang.org/scribble/reader-internals.html Some of them may need some tweaking to make them more idiomatic with the new syntax, but most of the implementation will be useful as is.
[1] Most comments are nice, but let's add some drama to make the story more interesting.
[2] Some languages have more impedance mismatch and need some macro magic to make sharing easy. In some languages sharing libraries is straightforward.
I was under the impression that SBCL has monthly releases and runs on a bunch of current platforms. It also has a AOT native code compiler for 20 years, which Racket is just switching to. http://www.sbcl.org/sbcl20/
My Linux/ARM64 board runs a current commercial Common Lisp, which is also available natively on a bunch of current platforms incl. iOS, Windows, macOS, Linux, FreeBSD, AIX, Solaris. x86, x64, ARM32, ARM64, SPARC, POWER, ... It has a portable native code AOT compiler since 30+ years.
Recently, Perl 6 has indeed become a lot less real, as it has been renamed to Raku (https://raku.org using the #rakulang tag on social media). But it is still very much a real thing (with regular releases since December 2015). And it actually has its own IDE: https://commaide.com .
I know that some well-known Lisp hackers left for Racket, but I've never been clear on how it is really a successor. It does have a great community and ecosystem, but it inherits a ton of the mistakes of Scheme, is unspecified and as such is susceptible to derailments like Rhombus. I think that performance is not that great either, which isn't necessarily a huge deal for many problems but is for some.
What I am genuinely curious about is: from your point of view, what makes Racket a preferable successor to Lisp (which can't easily be grafted into a Lisp environment)? The two things which leap to my mind are the sane read case and the community mindset.
As good as its development environment might be for learning purposes and research, it is no match for neither LispWorks nor Allegro Common Lisp, so it is hard to be a top successor.
In fact there isn't a single commercial Scheme has successful in the market as those Lisp environments, with exception of ChezScheme, used commercially by Cisco.
One could even consider Scheme as more widely used than CL.
The biggest production use case for Lisp today is to learn Lisp.
The biggest use case for Scheme, CL, or Racket is to use to learn Lisp. Racket that way was quite successful.
With all this going on now. Scheme is the only serious Lisp for learning purposes. What new in sights will Rhombus provide here? The biggest use case for trying out scheme was while working through books like SICP.
The fact that CL is used enough to keep two commercial vendors in business, shows how relevant it still is across some industries, even if doesn't do newspaper headlines.
More so than ever Scheme ever was.
Clojure is also non-existent in production to the software world at large.
The latest Clojure survey had roughly 2500 people giving feedback. That's a drop in the ocean of many millions of Java, C, C++, C#, Python and Javascript programmers. Rough estimates say that there are >7 million Java programmers world-wide.
https://www.zdnet.com/article/programming-languages-python-d...
The number of Clojure programmers is probably less than 1/1000ths of that.
I think you are mistaking working time for bloat; they can be inversely related; it's easier to add lots of ideas, it's harder to pare things down.
> Plus as a former Perl programmer having watched Perl 5 lost almost everything, chasing a never to have come to reality Perl 6. I can say, given all this, the future of Racket as a language, for its core uses and users is pretty much dead in the years to come. Like dead totally.
This isn't Perl 6.
> Many people prefer a perfectly working tool improved over time, than a pie-in-the-sky idea that will never come to see the light of the day.
Pretty much everyone does, but Racket remains the former. Neither the schedule nor the ambition of new Racket development has been scaled back for Rhombus. It's not even clear that they would once Rhombus was generally available and hosting Racket, though there would also be a place for improvements that didn't fit in Racket.
Indeed. And even Perl 6 is not Perl 6 anymore: it has been renamed to Raku (https://raku.org using the #rakulang tag on social media).
An example: Je suis allé chez Steele et Sussman roughly translates to: I went to the Steele's and the Sussman's homes