rockwotj

Would be super interested in this if you explore this more! Feel free to reach out (contact info linked from my bio) if you’re interested in collaborating.

I think the wasm C API is not going to live up to the promise. To actually use high performance features you need many additions to that API that most implementations provide, thus making it less portable.

Source: I contribute a lot to wasmtime’s C API and build Redpanda’s Wasm Data Transforms using it

I was researching this the other day, here is a non trivial example:

https://github.com/floooh/sokol-tools/blob/master/build.zig

Here is a tutorial: https://www.zvm.app/tutorials/zig-build-cpp.html

As I have been learning about stack unwinding and how it works, I really feel like this is quite the wizardary that is amazing it all works.

Generators are a thing in multiple languages that are essentially this (javascript, C# to name a couple). It’s nice for creating lazily iterators ergonomically

Seriously. Shortwave before it pivoted to AI, before it pivoted to Google Inbox replacement, was attempting to be slack for email. You can still see parts of this alive in the UI today. But it was a dream to just have a really good mailing list UX over a slack channel. I don’t like Slack, but it has a serious moat and a lot of folks were nervous about blending internal chat and external formal email. Which once you get to a certain scale you realize Slack doesn’t work and all large notification streams eventually turn into a crappy version of email without labels, filters, search, or etc

If your tsconfig settings are strict enough this is a type error :)

Agreed, runtime checks can be implemented in typescript - that’s the beauty of it

I feel like this is what code review is for. Isn’t this also possible with your application code? How do you prevent that?

As for I/O concerns, run it in CI with deno and ensure there is no I/O

On the reverse side of things, trying to find a C++ dev is usually harder than finding people who want to write Rust.

Ah just give me typescript, I do not need to learn a new thing for configuration languages and at the end of the day the output is compatible with JSON. Typescript has all the stuff I would want: types, first class json support, an ecosystem of libraries (if you want it, I would probably not for config generation). And the tooling is amazing. Does pkl have LSP, syntax highlighting in every editor, debugger or repl?

Yes RTTI

Presumably they just use -fno-exceptions, and not everyone has to have portable code. I appreciate the concept but pragmatically there are plenty of C++ server side projects that only use Clang and GCC

No stdlib or no exceptions/RAII. Because everyone at Google is on the no exceptions/RAII train (according to the Google Style Guide), and I happen to like exceptionless C++

Thanks for the correction! It's hard to track all of this :)

I have compiled JQ to wasi (in terms of moderately complex ( and useful )).

Took me a couple of hours, mostly because of some pthread stuff I had to noop. And that's being fixed with wasi-threads

https://github.com/rockwotj/jq-wasi

Honestly I find all these different config languages either too much to learn or they get too unwieldy quickly.

I have arrived that I think using typescript to generate JSON as being the ultimate solution.

Easy JSON support, optional typing, you already know it, and adding reusable functions and libraries is understandable. Just prevent external node modules, and have a tool that takes a typescript file with a default export of some JSON and renders the JSON to a string on stdout.

I recently wrote a blog post about how I think WasmCloud is very close to this with the new component model. They use IDL instead of language constructs to specify boundaries. Pros and cons of either

https://blog.rockwotj.com/musings-on-kubecon-wasmday-2023

Look at Netty.io, that's the defecto java networking library. They have some optional JNI stuff

Here is an example project that even downloads the right JDK automatically for you: https://github.com/rockwotj/SyllabusDB

Honestly Bazel and Java is actually a really nice experience. Pull in third party stuff from https://github.com/bazelbuild/rules_jvm_external. There is built in support for uber jar builds by adding a _deploy.jar suffix to binary jars. Protobuf and other codegen is super easy!

The benchmark is run 5 times without flushing file system caches. If you mmap it, it will still be in memory between process runs. The author intended this FWIW

A few things:

Disk access can certainly be parallelized, and NVMe is blazing fast, so the bottleneck is more the CPU than disk. There are systems that are built around around modern hardware that realize this (redpanda.com, where I work is one such example)

Parsing is a lot of the compute time and some SIMD tricks like SWAR for finding delimiters can be helpful. Stringzilla is a cool library if you want to see a clean implementation of these algorithms: https://github.com/ashvardanian/StringZilla

See my reply here about the file being cached completely in memory after the first run: https://news.ycombinator.com/item?id=38864034

Yeah I missed this originally, and stuff could be faster with this assumption without a full double parser. The fastest java solution dies some near branchless decoding for these

Why?

The author does explicitly want all the files to be cached in memory for the later runs: https://x.com/gunnarmorling/status/1742181941409882151?s=20

Yeah so I had a discussion on Twitter about this, turns out 12GB is small enough to fit into memory, and the author runs submissions by running a solution 5 times in a row, so using direct IO actually hurts because having the kernel cache is a way to enforce the file is in memory for the 4 runs after. I have a direct IO solution with SIMD string search and double parsing, just in C++ (using libraries). It runs in 6 seconds on my 24 core linux box (NVMe).

Code: https://github.com/rockwotj/1brc

Discussion on Filesystem cache: https://x.com/rockwotj/status/1742168024776430041?s=20

> Thanks to Llama, Meta's open-source AI development tool, almost anyone can build their very own social media platform.

I don’t believe this in a second. Even if an AI could cobble code together to make an entire social media platform, there is a lot that goes into building something like this. Even if you could make a Twitter clone in an hour with Firebase, you have to figure out how to get people to use your platform, which usually seems to happen because of network effects.

I see you don’t like threads, you just want each message to be it’s own entity. I think Apple Mail has a setting for this but Shortwave is very much a more sophisticated version of threads compared to other clients. They detect side conversations in threads and split them off into separate threads with links between them.

You should try shortwave.com, it’s cheaper, has a similar feature set and in some ways it’s better (the dark mode is much better, for example)

It is a balance between simplicity, speed and size. They choose size, honesty leb128 isn’t *too* complex (you can see an implementation in my repo I pointed too). But yes they don’t take all the simplicity tradeoffs.