marginalia_nu

If you submit 16 contiguous read requests the system will just merge them into one large read request.

Modern SSDs tolerate moderate queue depths very well, but piling on the I/O queue also incurs tail latency jitter unless you're able to ensure the queue depth stays in the moderate range and never goes higher. All else being equal, fewer larger requests is better for I/O latency (though read amplification for the sake of reading more data obviously doesn't help anyone). Though in this scenario, we're mostly comparing the syscall overhead of a single preadv against io_uring bookkeeping for multiple preads, regardless of how you submit the reads they end up being the same operation.

Though at that point you can just use FADV_WILLNEED.

Depends a lot on the memory pressure. If you can be fairly certain the data is (or will be) resident in memory, mmap is basically unbeatable. If you can't (because the data is larger than RAM or there's other stuff competing for RAM), mmap can have gnarly system-wide performance implications[1].

[1] Mandatory mmap=poop-emoji link: https://db.cs.cmu.edu/mmap-cidr2022/

So I have a buffer pool with O_DIRECT reads.

I implement read-ahead in the application by (optionally) preadv:ing a single read into multiple destination buffers in the pool, leaving them unpinned, since as long as you aren't up against the bandwidth limit of the drive, a larger read is generally as fast as multiple smaller one on modern hardware.

I've tried doing this with io_uring as well, but found just eating the preadv syscall cost was faster.

If you're doing contiguous readahead in userspace, why not just use preadv? It'll limit you to doing readahead up until the next resident page, but at least in my experiments in Marginalia's index, preadv beats io_uring in all cases you can use a single preadv call to do the full read.

We've laid claim to one of Jupiter's moons though.

No I think this is the same case. It seems to just be following local links on the SERP. From a search for jack parsons, you can find hyperlinks to the sorts of requests it's making.

It's very likely the last few years of bot behavior is the consequence of the residential proxy business booming. This is indirectly due to AI company crawling, but the fact that they are as cheap and available as they are changes the incentives for anyone using them toward reckless and unsustainable request behavior, as there is no risk of burning your IPs, and very small chances of seeing any consequences of essentially DDoS:ing a website.

FWIW, git hosts have always interacted very poorly with crawlers, to the point where you have to actively code in git host detection to avoid getting stuck in an accidental crawler trap if you want to run a well behaved crawler. Easiest is just to look for anything that looks like a commit hash in a path and drop those URLs from the crawl frontier.

Reason they interact so poorly is that is that git hosts generate a lot of links. One for each file in each commit, and a diff for each file appearing in a pair of commits. Even a small repo can have millions of viable links, and most of these are stupidly expensive to render for the git host. On top of this crawlers generally don't have a very deep understanding of what they are crawling, and can't meaningfully distinguish computationally expensive requests from cheap ones.

I've had a fair bit of success with increasing the bot mitigation based on a global rate limit. During periods of high request rates, I throw progressively more hurdles at the bots, and during periods of low request rates I disable them all.

Yeah my search engine saw traffic of up to 160 queries per second the other day from some bot that was ostensibly searching for information on Jack Parsons. Just variations on the same query in different permutations of filters and site:-terms.

I think the biggest claudism is adjectiveing nouns.

"Foo is byte-identical to bar" is weird.

"Foo has an identical byte representation to bar" is I think closer to what most humans would write.

AI in general, both visual and textual models, has this horror vaccui where it likes to cram in as much stuff as possible everywhere. This seems to show up both in redundant sentences that pad a text with irrelevant information, and sentences overflowing with information.

Isn't that just programming terminology? Here it's used in 2016:

https://internals.rust-lang.org/t/random-thought-of-the-day-...

It's always been pretty common in programming discussions.

Here's a blog post I wrote in 2021 that uses the term shape to describe an algorithm:

https://www.marginalia.nu/log/40-wasted-resources/

And again in 2023, I use it twice in the same blog post both for data and code, and it definitely had no AI input:

https://www.marginalia.nu/log/87_absurd_success/

If you're doing high performance Java you're definitely looking at the compiler output. At least you should be.

It's arguably more important with Java than with the compiler output for something like C++, as C2 is much more unpredictable and dependent on runtime circumstances. You also want to be real certain that bounds and null checks are omitted as those come at a pretty big performance premium.

Anyone who is involved in any sort of performance critical work looks at compiler output on a regular, if not daily basis.

A lot of this is due the size of the corpus.

Grep falls apart for severely underspecified queries, which is the difficult part of web search. For any given query in web search there can be several millions of candidate results. You can get good results with FTS as well, but just finding phrase matches is inadequate, you need more ranking signals to find relevant results.

When Claude is looking for a function in your code base, it needs to sift through dozens of matches. This is not hard, and anything beyond grep is likely not worth the effort.

Also hard to overstate the sheer impact of a tool like Excel, which in many ways is an ad-hoc programming tool so intuitive that anyone can learn to use it. Even though it sees more jeer than celebration, and is often something techies seek to liberate users from, it's likely one of the most economically impactful inventions of the 20th century, already delivering many of the things we hype AI for.

Entire businesses run off Excel spreadsheets.

This was pretty much always wrong. Nobody writing performance critical code in Java trusts the compiler to magically figure things out, and the sort of code you arrive at if you want Java to go fast is generally unidiomatic.

Printing presses are not typewriters.

The printing press did displace the manual copying of manuscripts, which had insanely low bandwidth and generally kinda sucked for everyone involved. But for several hundred years after, until the industrial revolution and the mass production of mechanical typewriters, pretty much everything printed on a printing press was originally authored by hand. Even with typewriters handwriting lingered for quite a while, since they were very demanding in terms of skill, and all mistakes were permanent.

Arguably it's not until digital writing tools emerged handwriting was truly displaced. That made typing on a keyboard accessible enough that anyone with working hands can do it fairly easily.

Making a staggering $800/mo off Marginalia Search nowadays, off donos and API deals.

Seems to be a pretty decent interest in API access now that everyone and their mother is building a personal AI assistant.

Workflows will randomly break and things will move around and change names even more often than before!

It's also chemistry.

As long as we're using natural gas, there be a surplus of cheap ethane and we'll indirectly subsidize plastic, since making plastic out of the ethane puts that waste product to use (by creating plastic waste instead).

Yeah it'll be great when plastic starts to rot like wood. Thank god we haven't spent the last 60 years using plastic as a water proof electrical insulator.

> How many creative pursuits do you think the average person had throughout history when close to 50% of people died before age 5, fewer than 10% of people could read, women and minorities had close to zero rights, the average worker was impoverished and farmed sunup to sundown to barely eat and avoid famine, there was little treatment for diseases (infectious or otherwise), common people could be conscripted for war at a moment’s notice…I could go on.

That's an industrial age work ethic. Before industrialization made everything cheap, you couldn't just go to the big store and buy stuff, and as a peasant you definitely didn't have enough income to hire a craftsman to make everything you needed.

People have historically spent a lot more time than we do now making things they need themselves, or things for other people to trade for their craft. Shoes, clothes, basic tools, food, and so on.

So yeah, quite a lot it turns out.

So you're a peasant in England in the 1300s.

You caught the end of the medieval warm period, a period of unusually warm climate in Europe. It's been really easy to grow food, and the population of Europe has boomed. That ends in a fucking hurry though in the late 1200s and early 1300s, the climate rapidly cools (you're now in the "little ice age"), coming to a culmination in the great famine of 1315. Recovery took longer, but basically two years of starvation, disease, rampant crime, cannibalism. 10% of the population succumbs to disease and starvation.

Because this period of repeated crop failures and famines of varying magnitudes lead to increased imports of food to relieve the starving population, the black death arrives in 1348 as a cherry on top, claiming about half the population. That's a lot of dead people. Like think of the logistics of somehow burying or ... dealing with the bodies of half the population for a bit.

But yeah, we in 2026 have it pretty rough too I guess. RAM is too expensive now.

Eh, this is true prior to an IPO, but listed companies (like every single one doing this shit) are very much beholden to shareholders and quarterly reports.

Thing with investor money is that after a while the investors start asking where their dividends are at. If you put all that money toward building features nobody asked for, you're in for a rough time.

As long as you're scheduled by the kernel and not something like Kubernetes with a CPU limit, you can generally oversubscribe I/O threads without much of a problem. They're mostly parked waiting for syscalls anyway. Heck, even if they're mostly doing CPU work, the scheduler generally deals with it pretty gracefully.

Even so you could use cgroups to cap the memory for the process (including its caches).