zyedidia

When I've measured this in the past I've seen Wasmtime's Cranelift compiler generating code that runs roughly 1.5x-1.9x slower than native (LLVM) on SPEC 2017 workloads, with x86-64 being closer to 1.5x and AArch64 closer to 1.9x. From what I recall, interpreter performance was generally more like 10x slower than native, with some workloads that involved heavy cryptography/SIMD being 30x slower.

I think the only "C replacement" that is comparable in complexity to C is [Hare](https://harelang.org/), but several shortcomings make it unsuitable as an actual C replacement in many cases (little/no multithreading, no support for macOS/Windows, no LLVM or GCC support, etc.).

I don't think there are any plans for Ladybird to have a JIT compiler (they used to have one but decided to remove it) [1, 2], so it's not clear to me that this performance gap will be improved anytime soon (if ever).

[1]: https://youtu.be/dKHopzDtElY?si=jc3ho2NT4vPTbXBz&t=14

[2]: See FAQ here https://awesomekling.github.io/Ladybird-a-new-cross-platform...

What is the recommended way to use defer to free values only on an error path (rather than all paths)? Currently I use goto for this:

    void* p1 = malloc(...);
    if (!p1) goto err1;
    void* p2 = malloc(...);
    if (!p2) goto err2;
    void* p3 = malloc(...);
    if (!p3) goto err3;

    return {p1, p2, p3};

    err3: free(p2);
    err2: free(p1);
    err1: return NULL;

With defer I think I would have to use a "success" boolean like this:

    bool success = false;

    void* p1 = malloc(...);
    if (!p1) return NULL;
    defer { if (!success) free(p1) }

    void* p2 = malloc(...);
    if (!p2) return NULL;
    defer { if (!success) free(p2) }

    void* p3 = malloc(...);
    if (!p3) return NULL;
    defer { if (!success) free(p3) }

    success = true;
    return {p1, p2, p3};

I'm not sure if this has really improved things. I do see the use-case for locks and functions that allocate/free together though.

I think data races can cause undefined behavior in Go, which can cause memory safety to break down. See https://research.swtch.com/gorace for details.

It sounds like what you're describing is one-time allocation, and I think it's a good idea. There is some work on making practical allocators that work this way [1]. For long-running programs, the allocator will run out of virtual address space and then you need something to resolve that -- either you do some form of garbage collection or you compromise on safety and just start reusing memory. This also doesn't address spatial safety.

[1]: https://www.usenix.org/system/files/sec21summer_wickman.pdf

The bug used by that repository [1] isn't the only one that can be used to escape the Safe Rust type system. There are a couple others I've tried [2] [3], and the Rust issue tracker currently lists 92 unsoundness bugs (though only some of them are general-purpose escapes), and that's only the ones we know about.

These bugs are not really a problem in practice though as long as the developer is not malicious. However, they are a problem for supply chain security or any case where the Rust source is fully untrusted.

[1]: https://github.com/rust-lang/rust/issues/25860

[2]: https://github.com/rust-lang/rust/issues/57893

[3]: https://github.com/rust-lang/rust/issues/133361

This is very cool! I'm always on the lookout for extensible assemblers. I especially want one that can handle a normalized subset of GNU assembly so that it can be used on the output of LLVM or GCC (using existing assembly languages, but assembling them in non-standard ways or with extensions).

I think one of the interesting aspects of WebAssembly compared to JavaScript is that it can be efficiently AOT-compiled. I've been interested in investigating AOT compilation for a browser (perhaps there is a distant/alternative future where web browsing does not require a JIT), but maybe Wasm AOT compilers aren't really there yet.

Fair enough. I think it would be unfortunate if the WebAssembly language in browsers were a significantly different language than WebAssembly outside of browsers (just referring to language itself, not the overall runtime system). I don't think that has quite happened, and the outer ecosystem can probably catch up, but it worries me.

Is there any AOT WebAssembly compiler that can compile Wasm used by websites? I tried locally compiling the Photoshop Wasm module mentioned in the article but the compilers I tried (Wasmtime, wasm2c, WAMR) all complained about some unsupported Wasm extension/proposal being required (exceptions seems like the blocker on wasmtime, and the others gave cryptic error messages).

Is it really the case that browsers have default-enabled all sorts of extensions that are not yet widely supported by the rest of the ecosystem?

I am excitedly awaiting the full release of ASL1 from Arm. I wonder if anyone with more knowledge might be able to comment on how it compares with Sail and/or when we might expect to see a full Arm specification in ASL1 (as opposed to the current spec which is normal ASL and appears to be incompatible with the upcoming version). Perhaps in the future there might also be a RISC-V specification written in ASL1.

I think it's a difference between ARMv8 and ARMv6/7 (I believe BKPT on ARMv6/7 sets the exception return address to `addr(instruction)+4`).

The reason is because the ARM `brk #0x1` instruction doesn't set the exception return address to the next instruction, so the debugger will just return to the breakpoint when it tries to resume, instead of running the next instruction. Recent versions of LLDB will recognize `brk #0xf000` and automatically step over it when returning, but I don't think GDB does this. With GDB you would have to run something manually like `jump *($pc + 4)` to resume at the next instruction. Clang's `__builtin_debugtrap()` will generate `brk #0xf000` automatically.

The Rust team did a deep dive on the bug in 2020, which has some more details that might be helpful to understanding what's going on: https://github.com/rust-lang/lang-team/blob/master/design-me....

Thanks for the shout-out -- in case someone wants to check it out, the code for Lightweight Fault Isolation is available here: https://github.com/zyedidia/lfi.

Does anyone know why LSP uses UTF-16 for encoding columns? It seems like everyone agrees it is a bad choice, so I'm curious about the original reasoning. Are there any benefits at all to using UTF-16, or was it something to do with Microsoft legacy code?

I think the Cortex-X series cores are the ones starting to make their way into laptops and the like (Cortex-X4 is the latest). These are Arm's "flagship" cores.

I didn't fully read the paper you linked, but I think at a high level the difference is this: the Exynos approach only encrypts the data stored in the branch predictor, while the Arm approach additionally encrypts the index.

The Exynos approach means that if the attacker can find a branch of its own that has a hash collision with the desired victim branch during the course of the victim's lifetime, it can still perform a cross-training attack (however, if the victim exits and is relaunched then the mapping will change and the attacker must start over). This is perhaps unlikely, but the Exynos paper only suggests as a mitigation that the OS periodically change the key, at the expense of mispredictions (see the last paragraph of Section V).

The Arm approach solves this by using a "light-weight set update mechanism" that allows the hardware to automatically change the key periodically without incurring as much overhead. I'd have to read the paper more carefully to understand exactly how it works though.

A big reason is because I started a working towards a PhD recently, and so I've been more focused on that. I think micro has also reached a relatively stable spot, where it would only be significantly improved with some large new features. It is serving its purpose well as a simple/familiar terminal-based editor. I have plans to give it more love and release a version 3, but no timeframe.

I agree -- relying on Safe Rust's "guarantees" for security purposes is very likely to be problematic. To make the reasons concrete: for the last 4 years rustc has had a bug that allows writing transmute (arbitrary type conversion) without the use of unsafe: https://zyedidia.github.io/blog/posts/5-safe-transmute/. This is one of the 77 current open unsoundness bugs on the Rust issue tracker. To make this tenable you would probably have to use a separate language -- maybe some formally-verified minimal Rust-like language, and with different priorities from a people perspective.

I came across this issue in Rust recently: https://github.com/rust-lang/rust/issues/57893. I thought it was interesting since it lets you easily write transmute in safe Rust: https://zyedidia.github.io/blog/posts/5-safe-transmute/.

I think Hare (https://harelang.org/) might fit the bill: it retains the minimalism and simplicity of C, but fixes issues like this (and others). Unfortunately I don't think it's ready for real use yet, but I am keeping an eye on it.

You need to provide implementations of the sanitizer callbacks yourself. For example, to use the address sanitizer you should use `-fsanitize=kernel-address`, and then the compiler will generate calls to `__asan_load8_noabort`, `__asan_load4_noabort`, `__asan_store8_noabort`... when loads/stores happen.

In those functions, you need to implement the sanitizer yourself, and have it panic if it detects an invalid access (address sanitizers are usually implemented with shadow memory). You'll have to give your sanitizer enough shadow memory during initialization, and also add your own alloc/free sanitizer callbacks to your allocator (so the sanitizer knows when things are allocated/freed).

I have an example [1] that implements a basic ASAN and UBSAN in a kernel (it's written in D, but could be easily adapted to C or C++). Hopefully it's helpful!

[1]: https://github.com/zyedidia/multiplix/blob/master/kernel/san...

Depending on how much language support you want, you may want to compile without the D runtime (in which case you only have access to the C standard library, and various features are disabled, such as classes/interfaces, garbage collection, exceptions, and most of the D standard library). You can disable the D runtime in GDC with -fno-druntime and in LDC with -betterC. With those flags, the basic hello world program looks like this:

    import core.stdc.stdio;
    extern (C) void main() {
        printf("Hello world\n");
    }

I've been using D for OS development and have found it very good for controlling low-level details. GDC is the GCC frontend for D, and has most/all of the same features for controlling this stuff as GCC. For example, you can use `@register("edx") ulong x;` to specify that a variable should be in a particular register, `@section("t1")` on functions to place them in certain sections, and the inline assembler is the same as with GCC. Note: the @register feature is new with GDC 13. See here for the docs: https://gcc.gnu.org/onlinedocs/gdc/.

And of course LDC supports all the LLVM custom attributes, plus GCC-compatible inline assembler (not sure about the "g" constraint though) and LLVM-style inline assembler.

Thanks! Perhaps I shouldn't have characterized phony rules as a hack, but just that I think it is cleaner to express that information directly in the rule rather than in a separate rule. The paper you link is great, and I should read it in more depth -- are there any particular lessons or insights from it that you think Knit or other build systems could benefit from?

I haven't run into these pain points much with Make, but Knit does support spaces in targets/prerequisites (use single or double quotes to surround the name). For the second point, Knit has regex rules (a Plan9 Mk feature) that can allow multiple grouped matches. The example from the article is:

    $ ($name-(.*)-(.*)):RB:
        GOOS=$match2 GOARCH=$match3 go build -o $match1

which is a metarule that matches targets like `$name-linux-amd64` and can be used for easy cross compilation with Go. Of course since it allows full regular expressions, even more complicated rules are allowed.

This will unfortunately be a pain point for any build system that isn't already entrenched in the ecosystem. I think Knit does as well as it can to mitigate this:

* There are fully static prebuilt binaries for a wide variety of platforms (including Windows, where I think it is actually more complicated to install Make than Knit).

* Knit can generate a shell script or Makefile for the build, so these can be used for users who don't want to do development but just want to build from source.

* It is easy to build Knit from source (thanks to Go). Yes you need a Go toolchain, but once you have that it is just `go install github.com/zyedidia/knit/cmd/knit@latest`. Of all the languages that could have been used, I think Go is one that makes the "build from source" experience very straightforward.

I've been working on a tool called Knit (https://github.com/zyedidia/knit) that I think is similar to what you are looking for. Essentially, a Knitfile is a Lua program with Make's declarative rule syntax baked in. Or in other words, it is like Make (with some additional changes inspired by Plan9 mk), but where Make's custom scripting language is replaced with Lua (but it still keeps the declarative rules language). It's still in progress (I'm currently using it in some projects, and then will likely make some more changes based on my experiences), but I hope to release a stable version in the next few months. If you or others also have feedback, please let me know!