lor_louis

I wrote a summarizing article on type safe container types a while back, but with some C23 specific changes and a few tweaks to work better for complex types.

https://louissven.xyz/article/how_I_do_container_types_in_C....

Feel free to flag/delete if this isn't the place.

Do no give me hope like that.

A couple of months back my boss asked me why I didn't use AI all that much. I told him that I didn't think it made me more productive in the tasks at was doing at the time (having to wrangle undocumented really custom legacy infra stuff).

He told me he found AI to make him really productive and said something along the lines of: "It's really good at summarizing long reports and it saves me time when I have to write end of quarter status updates".

I'm convinced about 50% of management decisions come from Claude now.

I've implemented a similar system based on the original 2020 paper, but we applied it to the text log to try to "extract" similar features from free-form text. It looked promising and even supported full regex search, but the work was ultimately abandoned when we got acquired.

Handling text is a minefield. UTF-8 is great but when you get into graphemes, there's basically no way to handle them properly unless you write some code to generate graphene recognition based off the spec which is rather large and continuously updated.

Same for font rendering, there is a reason why harfbuzz is used everywhere. Getting an 80% working renderer is easy but the remaining 20% can take years.

It really "handling text correctly"should be a masters, and I'd sign up in a heartbeat.

I do something similar, but I don't implement the logic in a macro, instead I have a Vec struct which looks like

    struct Vec {
        void *data;
        size_t len;
        size_t cap;
        size_t sizeof_ty;
    }

I then use a macro to define a new type

    IntVec {
        struct Vec inner;
        int ty[0];
    }

Using the zero sized filed I can do typeof(*ty) to get some type safety back.

All of the methods are implemented on the base Vec type and have a small wrapper which casts/assets the type of the things you are trying to push.

I write a lot of C and Rust, and my personal experience is that for smaller C programs, Rust tends to have a slightly higher line count, but it's mostly due to forcing the user to handle every error possible.

A truly robust C program will generally be much larger than the equivalent Rust program.

Even in Linux, I'd say the tooling is a bit rough, dune and the new lsp are going in the right direction though.

Oh by "more complex" operations I referred to multiple cursors and multi line regex searches. I've noticed some performance problems in my own editor but it's mostly because "lines" become fragmented, if you allocate all the lines with their own allocation, they might be far away from each other in memory. It's especially true when programming where lines are relatively short.

Regex searches and code highlight might introduce some hitches due to all of the seeking.

Kilo is a fun weekend project, but I learned the hard way that it's not a good base uppon which you should build your own text editor.

The core data structure (array of lines) just isn't that well suited to more complex operations.

Anyway here's what I built: https://github.com/lorlouis/cedit

If I were to do it again I'd use a piece table[1]. The VS code folks wrote a fantastic blog post about it some time ago[2].

[1] https://en.m.wikipedia.org/wiki/Piece_table [2] https://code.visualstudio.com/blogs/2018/03/23/text-buffer-r...

As someone who stopped using windows about 7 years ago, and only recently used it last weekend, my eyes probably glossed over the fact that some buttons were laid out horizontally.

It also makes way more sense.

To be fair, win11 is a nightmare in terms of usability. I can only assume a committee of eldritch beings and accountants designed it.

It blows my mind that when right-clicking on a file in file explorer, the 'delete' option is hidden in a sub-menu under 'more options'.

That's the exact same circumstances that lead to the development of guerilla warfare. I don't know how you'd go around creating a "highly visible gimmick" that has any lasting impact though.

As the author of the repo I was surprised how close to my mental model the graph was. (I haven't worked on this repo in a while so my mental model is a bit fuzzy)

And yeah it seems to be missing a few connections but the ones that are there are correct line.h depends on str.h which links str.c which depends on xmalloc.c/h.

But had I been a new contributor to the project I would have found the missing links between modules pretty frustrating.

My guess is that the graph is only good enough to give and overview of what things are dependent on what if I were working on a monorepo and I had to justify to my manager that team B needs to do something for us.

I like the idea though, a lot of direct code to graph tools are too noisy and that tends scares non technical people away.

I tried it on a personal repo and it never ended up generating a diagram.

Might be a bug, so here's the repo. https://github.com/lorlouis/cedit

To be fair, I'd rather type BEGIN instead of <<? Or whatever the trigraph is supposed to be. We tend to forget that a lot of computers didn't have the keys to type "mathematical" symbols.

Luck and money.

I've worked on a database that considered FSST as a way to query over compressed log lines. We found that the compression ratio was highly dependent on how repetitive the data was. In the end segmenting by service (Apache, our go stuff, our rust stuff, etc) yielded pretty good results and log lengths of ~200 bytes were pretty well compressed.

We ended up not using it in production because the worst cases were absolutely terrible compared to our dumber skippable zstd.

Cargo check won't report linker errors, that the only difference I ever ran into.

A bump allocator can be implemented using atomics, but it does not support freeing individuals elements. I have seen some CAS heaps (linked list like) but I never had to implement them.

I did not know about Eytzing-sorting, I'll look into it after my vacation, thanks! And yeah our current system is column oriented, and I already tried most of your recommendations (including tries) but the biggest limitation we face is that different kinds of queries will be better served by different kinds of data structures but you can only store a few of them on disk before the cost to index becomes too big.

Yeah I tend to work with strings data around the 15 to 30 ish char count so I'm also sceptical of german strings when it comes to raw memory usage. What really interests me, is that in theory, an SSTable built from German Strings that point into a larger text buffer further down could result in less pagefaults during a binary search? Maybe?

It looks a lot like cedarDB's "german strings".

<https://cedardb.com/blog/german_strings/>

You could probably write a C++ implementation based on the article.

Note that it's not all that useful if you don't plan on searching for text based on their prefix. From my understanding, this is mostly a better way to store SStables in RAM/partially on disk if you mmap.

Used to work with telecom equipment in situations where high availability was required and equipment failure was expected.

Redundancy was needed because individual nodes/machines were more prone to failure. As machines got more and more reliable, having highly redundant infrastructure was seen as an extra cost.

I personally believe that Swift's strings where graphemes are the smallest indexable unit are the gold standard for writing logic that might truncate multilingual text. It's still not perfect though, they add overhead and updates to Unicode might change behaviour so there's that but it should handle most cases gracefully.

In utf-8, bytes (uint8_t) may not represent a whole "code point". A code point being an individually meaningful element in utf-8 like a space an 'e' or a modifier code point like an accent or a ZWJ. Most utf-8 libraries will let you address individual code points but it might still garble the text if you split between an 'e' and a '`'. To prevent this, splitting should be done in between graphemes (sequences of code points that render like a single unit*). And even graphemes have their problems.

Very interesting blog post about graphemes that parallels my experience writing a terminal text editor: https://mitchellh.com/writing/grapheme-clusters-in-terminals

* Event that is not a proper description of a grapheme

And don't even get me started on regexes

I would argue that strings as an array of individually addressable 'characters' is a bad fit for modern multilingual text handling.

It would be interesting to replicate the study in a financial district...

> old computers

> not updated since 2021