The Onyx Programming Language
wasmer.io
wasmer.io
The language marketing insists it is "data-oriented", even in the footer of every page. But I couldn't find any more detail on this.
My favourite example of this is Austral, which has a detailed rationale[2] that justifies the choices made and the tradeoffs. It really helped me decide whether I was interested in the language.
Odin, cited in the blog post, has a shallow rationale on its FAQ page[3], but seems to have had more time/maturity to document specific "data-oriented" features like built-in SoA[4].
[1] https://docs.onyxlang.io/book/Overview.html
[2] https://austral-lang.org/spec/spec.html#rationale
Austral rational documentation is indeed really interesting. My preference goes into another direction (notably toward expression-based languages), but it's refreshing to read well explained design decisions.
The section about linear types is also a gem to this regard. I would even consider this as a reference source to introduce the motivations and principles behind the Rust type system and borrow checker. Only the syntax would have to be made a bit more rusty.
[1] https://austral-lang.org/spec/spec.html#rationale-linear-typ...
That's a delightful sentence, and in context of how the design makes opinionated choices based on its goals entirely reasonable.
(I don't know if I'll agree with all of those choices after I've thought about them sufficiently to be comfortable having an opinion, but I do very much appreciate that they're explicit, deliberate, and explained)
Similar to what you are suggesting.
We get a bytecode runtime to rival JVM and CLR, but with a diversity of stake holders (no single vendor trying to push a language or enterprise offering), it's JITed even on Apple devices and on the damn web! Pretty amazing.
What makes me excited about this: Getting to poke around, at runtime, my compiled .NET C# code using PowerShell (in Snoop) was pretty eye-opening to me. The "standard" interoperability between PowerShell (Core?) and .NET objects as well, of course.
It feels great to have a common runtime making calling out to other languages so seamless. Yet it seems to me like it never reached the popularity it deserved, but maybe it will get another try, with a lot more enthusiasm (no more Oracle/MS brake-foot).
What problems does this uniquely solve that other languages & environments don't?
Basically, it's really all about certain WASM use cases, and everything right down to its memory management strategy is optimized for those cases.
It looks like it fits pretty squarely in the "if you want to do C level programming but you'd also like nice lambdas, sum types, and type inference"
>A modern language for WebAssembly.
1. Is it value based (C++, Rust, Go, TCL) or reference based (basically everything else).
2. How is memory freed? GC?
3. If GC how do you deal with the fact that Wasm GC is still in progress?
4. What about concurrency? Is it single threaded?
5. How does error handling work?
6. Does it support modern type features - Option<>, sum types, etc.
7. It's imperative, does that mean things that should be expressions (e.g. if/else) are statements?
1. It is value based, like C. There is obviously passing by pointer, but you do that explcitly.
2/3. Memory is manually managed. To make this easier, like Zig and Odin, everything that allocates should take an allocator as a parameter. This allocator can be the general purpose heap allocator, or it can be an arena, ring buffer, or even tracked allocations that can be freed all at once. More details on that to follow in the documentation.
4. It does have multi-threading support, thanks to the atomics WASM proposal. The standard library currently has support for basic multi-threaded primitives like mutexes, semaphores, and condition variables. Note, there is currently not async/await or any kind of concurrent execution.
5/6. It does have modern type features like sum types. It has Optional(T) and Result(Ok, Err) out of the box. That is the preferred method of error handling.
7. It is mostly statement oriented, but there are features like Elixir's pipe operator (|>) and if/else (value if condition else otherwise) expressions.
There are currently no large Onyx projects that can really test how this number scales, but I would guess the growth is not linear. So for an off the cuff estimated, I would say a million line project could compile in about 4-5 seconds.
Also worth noting that the entire compiler is currently single-threaded. So that number could get much better.
I started making Onyx because I simply wanted to create my own programming language for the fun of it. I choose WebAssembly as the target for my compiler because it was relatively easy to write my own full-stack solution, instead of relying on something like LLVM. That language evolved over the years until I had something I felt was worth sharing. When the Wasmer team reached out to see if I wanted to do a collaboration blog post, I said yes.
The main reason I didn't go with LLVM was simply because I wanted to write everything myself (for fun) and to have a super-fast compiler. LLVM is not the fastest backend, but it is understandable for all the optimizations that it does.
Even pointing to somewhere in the code that I could start poking around would be great.
You could definitely use Onyx to create a modern lisp interpreter that can run in the browser without any dependencies, but it might be worth porting an existing C project if that is your goal. I'm not gonna stop you from making something like that in Onyx tho; it would be very cool!
There is a however possibility for a REPL for Onyx to exist outside of the browser. This is because Onyx has another runtime it can use alongside WASM called OVMwasm. This runtime is entirely custom (you can see the source code in the 'interpreter' folder on the Onyx GitHub), so it could be modified to do non-standard things with WASM, like add new functions dynamically. That would take some work to achieve, but in theory should be possible.
From the "why webassembly" section :
> When making a programming language, you need to decide how your programs will actually be executed.
In abstract maybe, and obviously some language features might make an interpreter or a compiler harder/easier to make. But in general and in practice the language semantic and the "execution mode" can be treated independently. Some of the languages mentioned even have multiple style of execution (AOT, JIT, interpretation , compilation etc...)
> Onyx does not rely on libraries like LLVM to generate its code, because WASM is so simple to target.
In term of correctness maybe. It will be interesting to see what happens as performance and more complex code pattern need to be supported. Most language evolve some type of high level representation between the AST and they target of choices for language specific transformation and error reporting (SIL,clangIR and HIR? not sure about that last one).
>My strategy was to wrap libwasmer.a, the standalone library version of Wasmer, into my own custom WASM loader, to allow imported functions to be linked against native libraries
Doesn't this both limit the portability and introduce potential security risk (and thus negating the whole point of wasm) ?
I think the inter reaction with the outside world is what WASI and the other is trying to address.
(Great work on that front btw Onyx Team)
But! When I see powered by WebAssembly. I can't see a good reason to choosing a unique language when any language in theory can target WASM/WASI.
In other words Javascript/Python is looking at your lunch and is very hungry.
While Onyx does have core library support for doing standard POSIX things like stdin/stdout/stderr, file operations, and networking, Onyx can be used in an entirely different environment without any of these things. In fact, the standard libraries are (almost) entirely separate from the compiler (i.e. the compiler makes no assumptions about what the standard library is), so if the standard library does not suite your use case for one reason or another, you can write your own. It will be a bit of work, but there would be nothing in your way.
Isn't that true for most native languages as well ? C++ have multiple stdlibs.Rust,C and c++ can be used against bare metals.
The orthogonality of the std lib and the compiler and language semantic (maybe at the exception of the types) seems pretty common these days.
On MacOS there is only one C standard library, and it is the interface to the operating system.
On Windows, your choice of standard library is tightly coupled to your tool chain.
On Linux, you can kind of mix and match, but it also impacts your loader.
I suspect that the terminology we are using might be the culprit. Thinking of it as compilers, distribution/package and target might be helpful. Both LLVM and GCC are quite flexible in how they can be packaged and distributed to target various platform. A compiler distribution typically would include a (default) runtime, default options, std libraries and supporting tools (linker,debugger etc... etc...) for each supported target in the case of X-compilation.
A distribution is definitely strongly coupled to a host os/configuration. But the compiler proper not really.
https://sourceware.org/newlib/
And related to that again is https://keithp.com/picolibc/
This might seem strange to those who are used to today's dominant WASM paradigm, where the programmer usually controls both the VM and the .wasm program — such as website developers who create a .wasm file and then also write the Javascript to load it into the browser. But one of the most exciting things about WASM, to me, is the potential for domain-specific execution environments in which the code which loads and executes the WASM is written much earlier and by a different entity than whoever writes the WASM.
In Perl I'd probably do something like having a .pm file that does
package MyBindings;
use v5.38;
use Exporter 'import';
our @EXPORT_OK = qw(function names to export go here);
use Wasm::Bindings;
my_function u32 [u32]
other_function ...
...
You can register custom loaders in https://bun.sh/ that I think would let me provide similar functionality there (maybe via a .wasmbind file extension or similar) and vague memory says python has ways and means to do such things but I've not looked in some years.
(I suspect you're still right that a language designed for WASM will end up being more ergonomic in that regard even so, and also that over time we'll likely find other ways it's more ergonomic that I haven't currently thought of, but I still don't think the situation for other languages is -quite- as bad as you suggest)
If i follow you correctly, you are simply saying that in python it's possible to refer to some code in a way that the compiler can't anticipate and thus can't make sure the relevant code is present at runtime.
First this a problem in pretty much every language. Even in very static languages like c++, one can dlopen his way into this exact situation. JVM/.Net also have missing classes situation when someone screw up the packaging
This is solved by having a sensible code laoding API. Java classloader or .NET equivalent manage that.
Any time I see a language that compiles directly to WASM my instinct is that it will compete with Zig, Rust or C/C++/emscripten.
That makes sense, the mention of JS got me thinking about browsers which is where I spend most of my time.
From that angle i quite like it. Personally I also like the syntax.
The "nice" thing about WebAssembly is that the job of generating optimized machine-instructions is up to the WASM embedder, which Wasmer is doing very well right now. Onyx does have a second runtime that is currently only implemented for x86_64 Linux, that does have some more optimizations, but it does not compile to native code; just to an interpreted byte-code.
It's niche, but I've thought it'd be cool if there were a mini Wasm-based game building tool in the browser, but that requires a Wasm language with a compiler that can run in the browser.
This is similar to what the Onyx Playground currently does. To do that I just POST your project files to a server, save everything to disk, run the compiler, delete everything, and send the resulting WASM file back to the browser, to be run in a WebWorker.
So the main reasons for using WASM and all of it's advantages are lost. It's neither sandboxed nor self contained any more, but slower than using "native" compilation and still missing libraries (compared to JVM, CLR, Beam,...). Don't get mé wrong, I know about the advantages of WASM for the language implementor (although I'd use WASM GC), but as a user, but I don't know why I should use Onyx.
From wikipedia,
Onyx - A banded variety of chalcedony, a cryptocrystalline form of quartz.
Ruby - A clear, deep, red variety of corundum, valued as a precious stone.
Crystal - A solid whose atoms (or molecules) are arranged in a repeating pattern. Crystals are found naturally or can be made artificially.
Someone should use Diamond for the next programming language, really like the wiki definition.
Diamond - is a carbon crystal formed under heat and pressure.
Ok, that got my attention. Its refreshing to see a new language which doesn't takes 10 seconds to build a Hello world example.
Couple of questions;
1. The Windows installation is a bit tedious. Any plan to support Scoop for installing Onyx?
2. Given that gamedev seems to be a focus, with the raylib integration, how is the performance compared to something like c. Will a comparable onyx program get me to at least 50%+ of the performance of c? I haven't used webassembly much so I am curious how this works out.
I was just thinking of creating a new project to try out Raylib 5. So your post has perfect timing for me!
Best of luck to your language. Looks lovely in the online playground. Now going to set it up on my device
Like the syntax, mainly the pipe operator |>
A few questions,
1. Any plans to use binaryen to optimize the wasm output?
2. How is memory management handled? Are you going to use wasm-gc?
3. How does the C FFI work? Do you convert the WASM types to C types?
>
> x := 10
I...
Why is this a feature in every new language? Can't we have a language that is more verbose and explicit, not less? I'd love it if named parameters were mandatory, not optional.
(Named parameters is when you name parameters/arguments you pass on to functions, like you can do in python and groovy: `foobar(arg1: 123, arg2: 'hello')`)
Most of my problems I run into is due to implicit behaviour that no one bothers to explain. In Onyx here, having type-inference means I now how to remember that x := 10 means x will be a signed 32-bit integer instead of, you know, the code that I'm writing remembering it for me.
And I'm just guessing here. Maybe x is a double. Or unsigned since its initial value isn't negative. Or maybe it was signed 32-bit integer but the Onyx developers changed it to 128-bit long long for version 666.0.0. The point is I have to look this stuff up or remember it instead of, you know, it being right there.
I don't even know what you gain by doing this. Less code is less messy but also hides a lot of information from you. Hiding information should be something an IDE does, not the language itself.
Thank you for reading my rant.
HashMap<String, List<int>> hashmap = new HashMap<String, List<int>>();
var hashmap = new HashMap<String, List<int>>();
var hashmap = new HashMap();
My preference would be to do some type inference, but maintain the property that you can tell the type of every expression without looking outside of it (except perhaps for an immediate enclosing function call). This requires, for example:
- The third option isn't allowed, you need to write the second option instead.
- You must annotate function argument types.
- In Rust, you couldn't write `.collect()`, you'd instead write `.collect::<Vec<_>>()`.
- The `x := 10` example is actually somewhat ambiguous. If the language fixes the type of `10` then `x := 10` is legal. If it's an unspecified type (as is typically the case), you'd have to write the type down.
For that case I like a type signifier as part of the number literal expression, like this: `x := 10f32` or `x := 10i32`.
public static <E> ArrayList<E> newArrayList() { return new ArrayList<E>(); }
public static <K,T> HashMap<K,T> newHashMap() { return new HashMap<K,T>(); }
import static GenericConstructors.*;
...
ArrayList<String> names = newArrayList();
HashMap<String, List<int>> hashmap = newHashMap()
--
What would resolve this whole issue is standardized editor/IDE visualization support for showing all inferred types, just one toggle button/key away.
Inferred types simplify writing code. But when reading code, why should we have to mentally emulate the language's inference algorithm? It is, by definition, supposed to be automating that for us.
Your static utility functions save only 2 characters, but will add massive confusion for other developers.
List<String> names = new ArrayList<>();
var names = new ArrayList<String>();
This is untrue. Inferring diamond types has been built into the language for over a decade.
Map<String, List<Integer>> map = new HashMap<>();Take your Java example. If the code in the .java file looked like
HashMap<String, List<int>> hashmap = new HashMap<String, List<int>>();
hashmap = new HashMap();An extra advantage I see is that non-default types stand out. At a glance at the code, `x:=10` is the most common int type. It’s plain, unadorned. It’s the usual, boring thing. Things that are less common stand out: oh, this is unsigned for some reason. Guess I should see why. I like the pattern where unusual things are easier to visually scan for.
But at the end of the day, darn it, compiler, you know very well that I mean 3.14 to be a float. Stop making me say so each time!
Is Onyx like that? I don't know. It would take me maybe two minutes to look up. The point is I have to look it up to be certain when that certainty could be part of the language.
I love complete inferred type systems, using one has completely changed how I think about types and what I expect from a programming language. I feel ill every time I have to use go or typescript. Like I am teaching the compiler how to compile. This is not my job, it's a waste of time and energy, it should already know. Things like rescript and ocaml are where my attention is going for typed languages in the future.
/* 'a' : int
* 'a' is truncated to char (self-evidently safe here(
*/
char c = 'a';
/* 1 : int
* c : char
* c is promoted to int
* 1 + c : int
* 1 + c is truncated to char
*/
char d = c + 1;
/* d : char
* c : char
* d and c are promoted to int
* d - c : int
* d - c is truncated to char
*/
char e = d - c;
// Zero. 32-bit integer left shifted by 32 bits is always 0.
1 << 32;
// 2^32. "U" makes it unsigned.
// "LL" makes it "long long", 64-bits on Windows and Linux.
1ULL << 32;
// -2147483648 (signed 32-bit)
// i.e. 0x10000000
1 << 31;
// 2147483648 (unsigned 32-bit)
// i.e. 0x10000000
1U << 31;
I prefer the type inference approach, e.g. in Rust, where they're i32 if the type cannot be inferred and the literal has no type suffix. And I like that no two integers can be used by the same binary operator unless they have the same type, so you need to explicitly cast them to the right type.
That is not type inference. 1 has the type int. 2.0 has the type double. Then, 1 gets converted to double. Then the whole expression has the type double. This isn't type inference; this is like saying JavaScript has type inference because it deduces that the expression ('4' - true) has the "number" type (i.e. double precision floating point).
Compare with Haskell, where a numeric literal like 32 has the type (Num a) => a, i.e., it's polymorphic, and the type is actually inferred based on the context it's used in (it could be Int, Integer, Double, Rational, whatever). If you ask it the type at a REPL, it just tells you "32 :: Num a => a", whereas C would tell you that 32 has the type int (if there were a REPL for C).
This is just a difference in terminology. To me, what you’re describing is exactly how a type inference algorithm works. This is also the traditional academic definition of type inference, but it sounds like you’re just using it to mean “inference of the types of variables” (which makes sense as the programmer-facing definition, to be fair, because basically all languages have type inference in other places)
> This is like saying JavaScript has type inference
JS doesn’t have type inference because it’s not statically typed, but Typescript does, and it works exactly how you described it.
The difference is, with compiled languages, the compiler needs to know the type of every expression and subexpression ahead of time, to know what code to emit.
Like, in Rust, the type of a literal depends on the context of the variable it's later used as.
// Due to usage below first line,
// 5 is retroactively reanalyzed as if it were 5u8
let a = 5;
let b: u8 = a + 1;
C++, which has a form of type inference, works differently: the type of a variable is always the same as the type of expression initializing it. The closest equivalent in C++:
// "a" is inferred to be an "int" because 5 is always of type "int"
auto a = 5;
uint8_t b = a + 1; // Implicit truncation from "int" to "uint8_t"
let a = 5i32;
let b: u8 = (a + 1i32) as u8; int(int(int(2) +::<int> int(2)) /::<int> int(8))
That's not why you don't have to do that.
Many (most statically typed?) Algol-likes have strict types for literals (e.g., 2 is always a signed int, if you if you want specifically unsigned you might say 2u and if you want a double you say 2.0, and if you want a single-precision float you say 2.0f, or something) and strict rules at how math between them works and what types it produces, this has been true since long before tyoe inference became common, and is why you don't have to say 2int+2int — 2 is syntactically defined as int.
There is no inference
Also reasonably, each source file should start with declaring the version of a language it was written in, so that e.g. Onyx 666.0 changing its default integer width wouldn't affect the code in the file with "///OnyxVer=665.0" at the top of it. Now that is a feature I would like every new language to have.
That is, as long as the core tools for the language provide that info somewhere that’s not more than barely more hidden than explicitly typing it out, I think that’s Ok.
Because superfluous type checker incantations are, aside from breaking up flow of thought when writing code, annoying visual noise when reading it.
> I'd love it if named parameters were mandatory, not optional.
Named parameters are a different thing than type incantations, and I agree that it would be good for any non-operator function that takes more than 1 argument to require named parameters.
I disagree. Implicit typing for the most part makes things harder read. The less I have guess about what type something is the better. I never use stuff like "var" or "auto", unless forced to for this very reason.
I would include functions which implement (especially unary/binary) mathematical operators where the operands either are interchangeable or have a clear conventional order, in languages where you cannot make them actual operators, within the exception for operators.
So if you want the type to be always explicit, the type specifier should be coupled with the literal like `10[i32]` or similar, so you can write `foo(10[i32])`.
Assigning variables or calling methods (when you discard the return value) would look the same as is typical in many languages:
int x = foo(); // foo() returns int but we can see that in what type x is
foo(); // we don't care what foo() returns
CoolObject cool_obj = bar(int arg1: foo())What's wrong with the old verbose languages?
If you have a type system, then force the user to specify exactly what type they want something to be. It's completely reasonable in my mind to write var x float32 = 10, the compiler can then deal with adding the .0 if it wants.
This is really cool. I wonder if there is any overhead using native libraries from wasm like this. My intuition says it does similar to node-ffi and JNI but I hope it's not as bad.
Currently, for WASM to call any imported function, its arguments have to be formatted in a particular way that is specified by the wasm-c-api. Then it can invoke the imported function. Only after that can the arguments be translated back into the native calling convention in order to call the real native function. I don't know how that compares to node-ffi and JNI, but I imagine its not as slow.
I have a piece of documentation I need to write that will explain how calling native libraries works from Onyx, so that should be able to go into more detail.
Thanks!
https://onyxlang.io/playground
The second one is a 404 but I wasn't even seeing that earlier (this is linked from the first page of the docs).
Currently, I don't have any issues but the problems I was having lasted at least a few hours which is why I mentioned it.
To me that's the piece that's missing in all these new languages
Being able to hot patch code saves you from having to recompile everything and reload the state of your program, saves lot of time
I was excited when Zig announced planning it, but it's been like 4 years now and they went radio silence unfortunatly
This arbitrary combination of :: { () : core std f fmt -> is just annoying.
Seems cool but I was trying to read one of the examples on <https://onyxlang.io/> and the carousel changed pages, throwing me off.
Would you consider not using a(n auto-advancing) carousel with code examples?
Not unrelated: <https://shouldiuseacarousel.com> (short answer: "no")
New languages mean you start all over and wipe away decades of tools, knowledge, libraries, development environments, package managers, workflows and now people are in the wild west again. To justify that you have to have some enormous benefits.
https://cloudblogs.microsoft.com/opensource/2021/09/02/onnx-...
Actually TIL seemless is actually a word but it's a synonym for unseemly.
Interesting if anyone else have such stories regarding language syntax and features.
Popular counter-argument is that it can be fixed with syntax highlighting, but for me it's a flaw in the language syntax.
May you wear it with pride.
To assign in Onyx, you only have to use '='.