LO – simple WASM native language | Hacker News Reader

LO – simple WASM native language

github.com

221 pointsglebbash63 comments

Hi, it's my first time posting here, just wanted to share the language I was working on for some time now.

I've been working on integrating it with WASM-4 runtime this past week. And with minimal changes to the compiler I created this demo game: https://rawcdn.githack.com/glebbash/LO/da8305293f5438967619e...

Game source here: https://github.com/glebbash/LO/blob/main/examples/test/demos...

Straight to wasm is a good choice. You could get something close to a native binary by packaging the result with wasm3 or similar.

you might want to consider changing the title to a "Show HN" https://news.ycombinator.com/show

Do you have (or plan to support) a Wasm build of the compiler so you can compile from within the browser?

Given you have includes I guess you'd need to provide a loadFile(filename) callback to provide it with data.

Yes. Compiling single files is supported by calling the exported `compile` function, example setup here: https://github.com/glebbash/LO/blob/60b80d1b325136ce1ceaf4dc....

For includes to work compiler uses WASI which can be implemented in the browser. LO playground and VSCode extension use this approach. Playground link: https://glebbash.deno.dev/lo-playground/ and source: https://github.com/glebbash/glebbash.dev/blob/main/static/lo...

Very nice project! I love the fact that you have almost no dependencies for the compiler. I'm interested in learning how to build a compiler for web assembly and your code seems very clean, simple and easy to follow. Thanks!

Great to see some attention to WASM-4, if it plays nicely with that I'll definitely give it a go

I like that you have a vscode extension already - I'm making a similar language project, and you really don't notice how much you miss even basic syntax highlighting until it's gone

Doesn’t tree sitter make it pretty easy to add highlighting?

For editors that support it, yeah. Sadly vscode doesn't natively support tree-sitter.

Cool project . I am also designing a Wa language for Wasm (https://github.com/wa-lang/wa), we are currently improving support for the WebGPU library and LSP extension.

Cool project you too :) It's fun to see how the language of the first compiler affects a lot of the syntax of the languages we build.

It's funny to watch how often new programming languages resemble visually (and conceptually) the language they are written in. You can put it the other way: just look at a syntax sample and try to guess the implementation language.

I can see some problem here. Probably, it means the author is under heavy influence of the implementation language, that limits their thinking and creativity scope to the concepts of that language.

I'm not sure if that is really a problem.

A single individual is likely to miss a lot of edge cases that a larger organisation (like the Rust foundation) has thought of while creating the language.

In that sense, I believe following the conventions created by a large and well-known entity is likely to produce better results.

I was going to say: I bet this is written in Rust, they use https://docs.rs/syn/latest/syn/ to not write their own parser/lexer, and this is how you get Rust-looking languages... I'm surprised.

It's funny yeah but it makes sense and is not really a problem.

You would write a compiler in the language that is the closest to your perfect. Trying to fix the original language by making a "clone" of it and adding/removing features to your liking.

That is what I want to change here by making a very simple language that if someone would like they could just fork it and add/remove the needed features instead of building one from scratch.

If this idea won't work then at least a very simple compiler would be a good learning resource, so a win win anyways.

Right, this is not the problem par se. To me this is a rather philosophical question. Related to the motivation behind the project.

If someone has some particular domain problem to which they need to introduce the new programming language, then, probably, it should look somehow different to all existing languages, otherwise they would be used.

Alternatively, the motivation could be: OK I need ExistingLanguageX(-alike) but in the domain/environment where it's not present yet, for example, WASM.

The motivation question is always of interest to me since it allows to judje for the long-term perspectives of the project.

However don't take this as a discouragement rant, learning case is a good motivation/raison d'être too.

Very cute project!

I personally wouldn't call anything Rust based "simple" (I'm not trying to hate on Rust, it's my daily driver.), so I have to ask: Wouldn't be the simplest possible wasm language, a small macro system for WAT, that bootstraps a slightly more highlevel lisp dialect?

Wasm already seems like an increadibly high-level language, given that it's a low-level compilation target.

I don't see how WASM is that high level. It doesn't have types except numbers (ignoring reference types and the GC proposal), which makes it lower level than LLVM IR. The control flow constructs are also much closer to SSA form than anything viable for humans.

If you ever try to write WAT by hand you realize that it's very much a compilation target, not something you'd ever want to manually write.

Unlike LLVM, control flow is actually structured, no arbitrary jumps, functions, tables, real harvard separation of code and data.

The restriction to only number types, is more a byproduct of trying to keep the 1.0 spec as mvp as possible, and reference types are gonna be a big shift in that regard.

I mean compare these two. One is a (very low level) language, the other is (high level) assembly.

https://developer.mozilla.org/en-US/docs/WebAssembly/Referen...

https://mapping-high-level-constructs-to-llvm-ir.readthedocs...

That was how it started actually, I wanted to make a superset of WAT. But then I switched to a more familiar C-like constructs and that gave me a big boost in productivity.

I think the superset of WAT idea is very viable and someone more comfortable with s-expressions should explore that for sure.

Thank you for your insights, again very cool project!

> a small macro system for WAT, that bootstraps a slightly more highlevel lisp dialect

Lisp traditionally has immutable data structures and a garbage collector, in which case I would consider it more complicated than LO.

I would be very interested if it’s possible to create an extremely simple garbage collector or an ergonomic Lisp that isn’t garbage collected, but a solution doesn’t spring to mind.

A classic mark-and-sweep GC for a heap of uniform structs (like cons-pairs) can be implemented in a few lines of code. Garbage collection is not inescapably complicated, it just gets more challenging as you impose more constraints: multithreading, generations, compaction, minimizing bookkeeping overhead, etc.

While I'm a huge fan of immutable persistent datastructures, I don't think that they are a neccesity.

And linked lists with reference counting are really easy to do, and don't have any leaks if you don't allow cons cell abuse. (which would mess with immutablity anyways)

You could also do linear types like carp [0], or you could just do manual memory management.

I feel like most people will call something with macros and s-expressions, a reasonable, if brutalist, lisp.

0: https://github.com/carp-lang/Carp

> Lisp traditionally has immutable data structures

Can you provide a source for this? Common Lisp and its predecessors certainly do not have immutable data structures out of the box. The usual data structures are cons cells, lists (composed of cons cells anyway), vectors, and hash tables.

> I would be very interested if it’s possible to create an extremely simple garbage collector or an ergonomic Lisp that isn’t garbage collected, but a solution doesn’t spring to mind.

PreScheme immediately comes to mind for me as does ulisp. The latter in particular is designed for microcontrollers with limited memory.

That much be a very special kind of Lisp, given the amount of variations since the Lisp 1.5 Programmer's Manual came to be.

Pairs in Lisp are mutable, traditionally, and there are other mutable data types like vectors. For targeting wasm only there's no need to figure out a GC solution as that is something the host provides in wasm 2.0.

You could probably do something similar to what Roc is doing: https://www.roc-lang.org/

it would be assembly script, the most simple possible typed language to make scripts for wasm. its almost identical to js.

While that may be easy it's definitely not simpler than WAT.

https://developer.mozilla.org/en-US/docs/WebAssembly/Underst...

Yes, this would be similar approach how Lisp Machines, and Interlisp, came to be.

Low level forms for systems programming, and then everything else was built on top of it.

Picking something like "An Incremental Approach to Compiler Construction", but instead of Scheme, stay in WAT and move from there, adding features.

http://scheme2006.cs.uchicago.edu/11-ghuloum.pdf

Or stay in Scheme ;) https://spritely.institute/hoot/

Indeed.

FYI the VS Code extension Link in the README is broken, as it links to /lo-extension instead of /vscode-ext

And is it intentional that the repository does not have a LICENSE file, or do you intend to open-source (by OSI definition) it?

Fixed the link, forgot that markdown references use double square brackets.

Hmm, I haven't thought about it the license actually, I would default to MIT, but wanted to keep the number of files minimal in repo, hehe.

Will figure it out asap, thanks.

This is very nice! What is missing for me is the answer to question "Why?"

Mostly for recreational programming. And then for exploring what the languages with minimal infrastructure could look like.

If you would like to learn how to build your own then check https://wasmgroundup.com/

Disclaimer: I'm the co-author

How is this project related to the work you do? e.g. instadeq, gloodata

Why do you need WASM?

it's not that related, I like to create compilers/transpilers/weird languages, see the "Language Projects" section here: https://marianoguerra.github.io/

But there's a relation, two years ago I prototyped a thing that tried to make a headless version of instadeq that could run in the browser and the server, the idea was a dataflow language where the data was represented as apache arrow tables where filters and transformation expressions where compiled to wasm. The prototype kind of worked and I had a lot of fun, I thought about sharing the fun by writing a small book about creating 10 toy languages that compiled to Wasm, I shared the idea with Patrick, he said he was thinking about something similar and the idea evolved into the current book.

In-browser data analysis is a great place to use Wasm. Javascript is great, but larger datasets really benefit from SIMD + tighter memory control/controlled memory layouts.

Some interesting data-related Wasm projects:

- DuckDB Wasm | https://github.com/duckdb/duckdb-wasm

- Row Zero | https://news.ycombinator.com/item?id=39551064

- PGLite | https://github.com/electric-sql/pglite

Thanks for this answer

Neat project! I liked the dev log writeups (despite the multiple rickrolls), hope you keep doing these. It was interesting seeing your reasoning. I'm starting a for-fun language project that should compile to wasm and native, and was debating between wasm -> wasm2c or llvm -> emscripten. Your posts about ffi/bindings issues were helpful, I'm probably going to try wasm (or just wat) first and then binaryen for wasm -> wasm optimization.

Looks interesting! Always glad to see more WASM native languages coming. You may take a look at MoonBit(https://github.com/moonbitlang/moonbit-docs/tree/main). The language is Rust-like with gc support, and we also have a full toolchain like cloud IDE, compiler, build system, and package manager.

Looks very cool! I can see myself using this for some Wasm experiments.

Please add a license so it's clear how people can / can't use the project!

Yay a first user :) Added the license.

Fantastic!

How does LO compare with AssemblyScript?

AssemblyScript is very strange to me. The whole point of wasm is you can avoid javascript/typescript language and do stuff in a browser in go, rust, or now LO!

I wouldn't say that's the whole point of Wasm. Speed & efficiency are also key goals, which are what AssemblyScript play into, giving access to that speed potential in a format that's easy for web devs

And the similarities between AS and TS (or especially JS) are very superficial. They share some grammar, and utilize many of the same type system features, but AS is fundamentally a different language with fundamentally different semantics (which are much closer to WASM, and then much closer to those other compile-to-WASM choices).

Nice work. Out of curiosity, did you consider using pest (https://pest.rs/) to help build your parser? Or is it too much for what you are doing?

Well the goal for this project is to go dependency free so, yep, it's not a good fit here.

If you run it in chasm you can run it absolutely everywhere: -JVM -Browser -Mobile -Desktop

https://github.com/CharlieTap/chasm

This looks super impressive! Any chance you could explain how memory management works? Eg is it like rust in that sense? Or is it GC'ed ? Thanks

Absolutely. The memory management is manual like in C, but with addition of defers it is actually quite good to work with.

Here is the source for the allocator: https://github.com/glebbash/LO/blob/81f96bb09e3a0b1455b32497...

It also links to where I learned how to build the "proper" allocator from scratch.

Any plans to use wasmgc?

I don't see a need for it. Maybe for some higher level scripting, but I don't have any plans for that.

A big advantage of WasmGC is it would allow your binaries to be smaller, since you wouldn't need to bundle an allocator for heap::alloc and free. Kotlin and Java binaries can in some cases be be much smaller than C++ and Rust for that reason.

Alright, will take a look at it sometime. But the more interesting proposal for me right now is the component model.

So far I’m loving this. Would love a language tour doc!

Thanks, I will definitely do that once the feature set will get stable enough. Don't want to update docs constantly :)

As for now, you can check the examples folder and explore the compiler sources, tried to keep everything simple.

Reply on news.ycombinator.com