The WebRacket language is a subset of Racket that compiles to WebAssembly | Hacker News Reader

The WebRacket language is a subset of Racket that compiles to WebAssembly

github.com

157 pointsmfru38 comments

Hi All,

It's still early days for the WebRacket project.

Racket is a huge language, so be patient wrt features.

To keep motivation high I decided to implement a subset that can be used to built practical applications - and then extend the supported features from there. Hopefully, this strategy will also lead to some early adopters that can help me prioritize which features to add.

Some features are simply "more of the same". In this category falls more types of hash tables. Supporting bignums are also a matter of just doing it.

Other features require more work. I have already done some work on implementing modules in terms of linklets. When linklets/modules work, we can reuse the existing implementation of regular expressions.

Adding continuation marks and delimited continuations require adding a CPS-pass. This is certainly doable. Postponing it has been great though. Having a direct style compiler means the generated code follows the structure in the input source code. And that makes debugging easier. Now that bugs have become rarer, it makes sense to look at CPS.

Enjoy.

/Jens Axel

Are there any architectural similarities or crossover with Hoot (Guile Scheme -> Wasm) or are you taking a completely different approach?

I am using a similar representation of immediates as Hoot and wasm_of_ocaml.

The representation is explained here:

https://github.com/soegaard/webracket/blob/main/compiler.rkt...

Internally the compiler uses a series of passes implemented using Nanopass.

    (generate-code
     (flatten-begin
      (closure-conversion
       (anormalize
        (categorize-applications
         (assignment-conversion
          (α-rename
           (explicit-case-lambda
            (explicit-begin
             (convert-quotations
              (infer-names
               (flatten-topbegin
                (parse
                 (unexpand
                  (topexpand stx)))))))))))))))

The code generator is inspired by "Destination-driven Code Generation" by Dybvig, Hieb and Butler. There are some differences however. The code generator in the paper generates "flat" code (assembler) whereas I generate nested Web Assembly instructions.

This approach generates reasonable code without having to implement a register allocator. Also, I believe I saw a Wasm to Wasm compiler that improved register allocation (maybe it was a switch for wasm-tools?).

If (when?) WebRacket becomes a success, we can always switch out individual passes.

> Adding continuation marks and delimited continuations require adding a CPS-pass.

Have you considered targeting the stack switching proposal?

Another idea, wasm_of_ocaml [1] compiles effects using either a CPS transform or the JS Promise integration proposal for Wasm [2].

[1]: https://github.com/ocsigen/js_of_ocaml/blob/1b1fcf7b06c12324... [2]: https://github.com/WebAssembly/js-promise-integration/blob/7...

Maybe. My main problem is to get light-weight support for continuation marks.

If I need a CPS-pass for continuation marks, I might as well use it for continuations as well.

It would be great if it were possible to avoid a CPS-pass though.

With stack-switching you won't (shouldn't?) need a CPS pass.

I need to study the stack-switching proposal in more detail.

However, I don't see an obvious way of attach and probe continuation marks to the continuations (including the current one).

I am not an expert in continuation marks, so I'll just link to this presentation by Matthew Flatt (which you probably already know).

https://github.com/WebAssembly/meetings/blob/main/stack/2021...

I read the WASM spec and became somewhat of an expert in it for the purpose of eventually designing a low-level language specifically for wasm, to be the most efficient and lowest-level language you could possibly have for wasm, and then gradually add high level concepts into it to make it more convenient. Kind of like building C on top of asm, and then gradually evolving it into C++. That project never came about though due to lack of community interest and time on my part, but I like seeing languages that take a fresh look at how to integrate with wasm using novel techniques to aim for maximal efficiency.

> to be the most efficient and lowest-level language you could possibly have for wasm, and then gradually add high level concepts into it to make it more convenient. Kind of like building C on top of asm, and then gradually evolving it into C++. That project never came about though due to lack of community interest and time on my part, but I like seeing languages that take a fresh look at how to integrate with wasm using novel techniques to aim for maximal efficiency.

I wonder how much efficient would such wasm be compared to native itself theoretically?

I really like libriscv as well, what are your thoughts on libriscv and now the recent project created by legendary fwsgonzo about looongarch.

Although I would say that libriscv/loongarch are more focused on trying to start the fastest as much as possible instead of pure performance (for which if atleast for the purpose of sandboxing, fwsgonzo has also created tinykvm which is really quite close to native performance)

A SIMD ECS language would probably be that today and be more modern than C.

I'm working on something like this right now, targeting WASM GC. I started with functions, basic numeric types, arrays, and structs. Then added blocks, control flow, and strings. Then interfaces, mixins, classes, and extension classes. It's now something like a statically typed mashup of TypeScript, Swift, and Dart, all done in the best way I could figure out specifically for WASM GC.

It's been a really fun side project.

I do think there is a market for something like this - optimizing an existing language for WASM, or ephemeral networked code delivery in general, can be really hard. And a statically typed, WASM-oriented, very familiar high-level language, that can give very good static errors, and quickly run in a secure sandbox might be a good target for LLM-generated code.

I had the exact same idea around 5 years ago, and actually built a language. But without a stdlib it’s a bit useless and as there was no component model at the time it was impossible to have one.

People into this sort of stuff might be interested to know that s7 Scheme also runs really well in WASM. It's 100% ansi C and uses its own GC, so getting it compiled and running in WASM is very simple. I use it in both my audio project (Scheme for Max, an s7 interpreter in Max/MSP) and in a browser based set of music practice tools I am working on as a solopreneur. It's fantastic to be able to write the music theory engine in Scheme instead of JS.

An example page (not mine) of it running in the browser is here: https://actondev.github.io/s7-playground/

I really hope the racket effort gains traction too! Excited to see this. In comparison, s7 is much more minimal. Though this also means the FFI is dead simple too, so extending it and bridging to js functions is much easier, and everything in s7 is available now in WASM - CL macros, keywords, hashtables, first class environments, continuations, etc

Is there any shared lineage between this and Whalesong, a previous Racket->JS compiler?

Of course both projects have the same maintainer if I recall, Jens Axel Søgaard is a rockstar :)

Jens Axel Søgaard is cool and involved in many things. We collaborated on SICP support.

Original developer of Whalesong was Danny Yoo. https://www.hashcollision.org/whalesong/

There was also this: https://docs.racket-lang.org/racketscript/

Dave Herman worked on various JS-related libraries for Racket (or PLT Scheme) before he was involved with Rust.

ah that's right! apologies

No, there is nothing in common with Whalesong.

Whalesong used the built-in bytecode compiler and compiled the bytecode to JavaScript. Reusing the bytecode compiler is in principle a good idea - but each time the bytecodes are changed, Whalesong needs to be updated.

And after the move to Chez Scheme as backend, the bytecode compiler is no longer a part of the main compilation path.

JVM languages always target bytecode because it’s much simpler and stable than Java as a language. It almost never changes and when it does it normally won’t break code generation since it’s only adding type system information, for example, as with records.

Is Racket bytecode different?

JVM bytecode is well specified. [0]

Racket is not [1]. It's just the internal representation that the compiler uses. Sort of like marshalling in Python.

[0] https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.ht...

[1] https://docs.racket-lang.org/raco/API_for_Making_Bytecode.ht...

> Is Racket bytecode different?

Changes to the bytecode representation were indeed rare also in Racket.

The Whalesong project was written as part of a dissertation - and when people graduate and get jobs, projects are often abandoned.

I love this. Racket is the future we were promised.

Speaking of prolific Racketeers... Noel! Just an hour ago, on a walk, I was thinking, "I should work through that one LLM book, and implement it in Racket." (But have started job-hunting, so will probably be Python.)

Which one LLM book?

I've got so much other stuff I'd rather learn and code I'd rather write (C/wasm backend for my language), but I've also started job hunting and probably should understand how this latest fad works. Neural networks have long been on my todo list anyway.

I'd guess it's Mark Watson's:

https://leanpub.com/racket-ai

I noticed the --expose-gc. Does this mean it's using the (now standardized) Wasm GC feature?

Yes. I am following the Scheme tradition of representing immediate values as tagged pointers. And (ref i31) is the obvious choice when using WebAssembly. I am happy you and the team added GC to WebAssembly.

Details on the representation.

https://github.com/soegaard/webracket/blob/main/compiler.rkt...

I am more or less only using the linear memory for the JavaScript FFI. FASL-encoded values are passed back and forth to JavaScript.

Hoot is another interesting one: https://spritely.institute/hoot/

Is a (Web)Racket engineer a racketeer?

They're certainly a schemer. :-)

It is interesting to see another language target WebAssembly especially one like Racket. The fact that it compiles to a subset of the language limits its utility right now. I think it is a neat proof of concept but it needs full language support.

check out grain! https://grain-lang.org/

I don't know that there are any similaries besides the names -- well maybe something thematic about distributing the future or what looked like it at the time to more programmers -- but the handful of times I've run across Grain (probably all on HN) I'm reminded of Wheat https://web.archive.org/web/20050215032130/http://wheatfarm....

As far as I know compiling full Racket to WASM is impossible because of continuations.

You could probably model continuations using the wasm GC feature since you can then pass around function references and strongly typed continuation objects, but making it work certainly wouldn't be trivial.

I wouldn't say compiling full Racket to WebAssembly is impossible. But I think the consensus is that one can't add a WebAssembly backend to the compiler in the same manner as the x86 and arm backends. These backends manipulate the stack in ways WebAssembly prohibits.

This forces an Racket implementation to make continuations explicit. And that will most likely mean a WebAssembly backend will be slower than the native backends.

It's possible to do continuations on Wasm now and it will be made even easier by the stack switching proposal.

For folks curious about the supported language subset, here's a summary:

- Modules aren't implemented yet, but are high on the list

- Continuation-based control flow isn't supported yet, including break and promises. Tail calls are supported though.

- No support for complex numbers, bignums, weak hashtables, immutable hashtables, prefab structs, regexp, or file I/O support

Most of the rest of racket/base should work, according to the README. There's also a FFI for javascript, including bindings for Math, DOM, Canvas, MathJax, XTermJS, and JSXGraph. Overall feels like you can use most of the language right now unless you need odd Racket-isms or call/cc.

Reply on news.ycombinator.com