msichert

Vector instructions usually don't have overflow flags, so a compiler can't easily vectorize loops containing overflow checks. However, detecting overflows in integer operations requires only a bit of bitwise arithmetic. In my experiments, this lead to an overhead of only 7% for vectorized additions with overflow checks: https://cedardb.com/blog/vectorized_overflows/

That's very interesting, I'm only familiar with the C++ standard where bit shifts are defined in terms of multiplications and divisions by powers of 2: https://eel.is/c++draft/expr.shift

So it seems in regard to bit shifts, C++ behaves slightly differently (it seems to have less UB) than C.

That's a good point. We just use pointer tagging in many different places (e.g. for pointer swizzling in our buffer pages), so including a few bits of information in a pointer just seemed obvious.

If you want to improve equality matching for longer strings, you could even store a 4B hash of the entire string instead of the prefix. I guess that should work well if you equality match on URLs since their prefix is always "http".

We chose 4B because that was the maximum number of bytes that would be unused otherwise (4B for the length, 8B for the pointer leaves 4B), the UTF8 encoding doesn't really matter.

Also, for UTF8 specifically, cutting code points in half is fine as long as all strings are valid UTF8. The UTF8 encoding is prefix free, i.e., no valid code point is a prefix of another valid code point, so for prefix matching we can usually just compare bytes.

It only gets more complicated if you add collations or want to match case-insensitively. But at that point you need to take into account all edge cases of the Unicode spec anyway.

I would tend to refuse such an offer but not because of an irrational fear of crypto. Rather, I would have very real concerns about the value of the tokens:

How do I know they will be worth anything when the company exits? I probably won't get a contract with the company that tells me exactly how I'll be paid in cash if the company is acquired. Instead, I will get a token that will be bought back by the company at a price set by the company at a time set by the company.

Another similar issue I see is dilution: How do I make sure the company doesn't just issue a billion new tokens so that my tokens are diluted to nothing? How does the token price change when new real shares of the company are issued (presumably investors want to get real shares instead of tokens)?

Hey, I'm the author of the blog post!

I had planned to write about the Kahan summation, but I didn't want to overload the blog post with too many different concepts. I'm definitely considering writing about it in a follow-up post.