knuckleheads

I will email you when I have something up for sure! First cargo check has it at about 28% reduced on codex rs, from 383 seconds down to 277 seconds on a 16 core machine. No -Zthreads yet though, and that typically has a similar effect and reduces the speed up the early meta data gives.

Yes I think I read a post of his from like 2019 were something was tried like this before. My guess is that this will be an interesting and hopefully informative PR for you all to see, but will likely not be merged as is. I had to add the ability for rustc to pause after early meta data emission and that was hacked together I feel, not well designed per se.

I view this as speculative execution/compilation and just throw out any errors from a crate that depended on another crate that ultimately errors out. It has the same outcome (same errors are printed in both cases) you just have a chance of having totally wasted some cpu time (that was otherwise just sitting around though).

This thing is a monster, blows out the disk space on a Claude code web session. Will run it on my test server later and let you know.

Funny to see this now. I’ve got a private branch I am in the process of shaping up this weekend to show the compiler team. For deep nested projects like rust analyzer, if you emit the meta data about function types earlier for downstream slots to use, before successful type checking, you can start other crates earlier and use all the slots you have instead of sitting around waiting for the full type checking of the bodies (which other crates largely don’t care about). Something like 40% wall time speed up, maybe 10% to 15% if you have the parallel frontend on.

Better for who? If I don’t care about the welfare of mathematicians and just the advancement of mathematics, would AI doing the math not be better for me?

As well, if the open models were as good as the proprietary ones, do you think they wouldn’t still have complaints?

Maintaining the existing standards is in fact a form of protection from AI disruption. They are asking the AI companies to follow their norms, instead of them having to conform to new norms created by AI. They don’t want to have to change the way they do things, understandably!, and are asking the companies to accommodate their way of life.

Mother of Learning is pretty good, similar vein, does some interesting things. The Wandering Inn is also pretty good, I'd say more challenging than Dungeon Crawler Carl while still being entertaining.

I don't see why mathematicians should be protected from AI anymore than any other profession. It's either everybody or nobody, not fair on the face of it otherwise.

> One of the foremost tasks of art has always been the creation of a demand which could be fully satisfied only later. The history of every art form shows critical epochs in which a certain art form aspires to effects which could be fully obtained only with a changed technical standard, that is to say, in a new art form. The extravagances and crudities of art which thus appear, particularly in the so-called decadent epochs, actually arise from the nucleus of its richest historical energies. In recent years, such barbarisms were abundant in Dadaism. It is only now that its impulse becomes discernible: Dadaism attempted to create by pictorial—and literary—means the effects which the public today seeks in the film.

> Every fundamentally new, pioneering creation of demands will carry beyond its goal.

Walter Benjamin, The Work of Art in the Age of Mechanical Reproduction, 1935 https://web.mit.edu/allanmc/www/benjamin.pdf

Edit: to put a finer point on it, AI art like this is, through it's rigorous application in almost every previous medium, showing that there is a demand and energy for a new art form that depends on the changing technical standard and capabilities that AI has introduced. I don't know what it will be, but you can see some of the edges of it with how it is making waves in all the other mediums.

Agents get a little upset when you ask them to do this and then check their work with a hook. They do not enjoy the process and create some weird looking text!

I couldn't get past all the network errors on OpenCode. Seemed smart enough, and was useful when I was low on usage on Claude, but beyond that, really hard for me to say whether it was Good or Bad.

Handcrafted software will find it's own market the same way that handcrafted clothing, furniture and other previously industrialized goods that became commodities did. They'll be more expensive generally and it'll be a smaller market overall, you'll always have some people who will love them all the same. Not my choice, but I can understand why they would want it.

The scheduler in the Linux kernel jumps out to me as a place where it would be cool to find something of this nature.

RIP to a real one. Have you considered using one billion Chinese residential ips to address your problem? I’ve heard one billion Chinese residential ips really does the trick here.

"Software [ought to be] about people, not code"

Related but not quite the same as https://en.wikipedia.org/wiki/Is%E2%80%93ought_problem

Previously, I had done a fair amount of research into how Google's monopoly on web crawling further entrenches their monopoly in the search engine market. You can read more about this here, https://knuckleheads.club, there is a long report from ~2020 or so that explains how it worked at the time. The club is mothballed, I am doing other things with my life, and I'm happy to say that we played a very small role in the DOJ ordering Google to share their crawl data with qualified competitors (a work in progress, but it's progressing).

Chatbots have super charged this dynamic though, to the point that it is showing up in the robots.txt data. The last few weeks I've been having Claude rerun some old analysis of Common Crawl from back then, when I have spare usage and time. What I've found is that you can see pretty clearly the rise in people outright blocking AI chatbot related crawlers likely because of how aggressive they have become.

  Quarter  Crawl    GPT     Claude  CC      G-Ext   Byte    Bing    Google
  2023 Q1  2023-06  0.00%   0.00%   0.16%   0.00%   0.06%   0.47%   0.39%
  2023 Q2  2023-14  0.00%   0.00%   0.18%   0.00%   0.06%   0.45%   0.38%
  2023 Q3  none       —       —       —       —       —       —       —
  2023 Q4  2023-40  2.21%   0.00%   2.12%   0.04%   0.11%   0.39%   0.27%
  2024 Q1  2024-10  0.53%   0.05%   0.31%   0.09%   0.18%   0.34%   0.31%
  2024 Q2  2024-18  0.55%   0.09%   0.32%   0.11%   0.24%   0.32%   0.31%
  2024 Q3  2024-30  0.68%   0.22%   0.36%   0.20%   0.38%   0.24%   0.33%
  2024 Q4  2024-42  1.10%   0.50%   0.44%   0.32%   0.50%   0.25%   0.40%
  2025 Q1  2025-05  1.14%   0.66%   0.54%   0.42%   0.66%   0.25%   0.44%
  2025 Q2  2025-18  1.37%   0.93%   0.63%   0.70%   0.92%   0.29%   0.19%
  2025 Q3  2025-30  1.42%   1.07%   0.74%   0.62%   1.01%   0.31%   0.27%
  2025 Q4  2025-43  1.92%   1.51%   1.23%   1.15%   1.52%   0.27%   0.19%
  2026 Q1  2026-04  2.13%   1.76%   1.68%   1.58%   1.77%   0.22%   0.15%
  2026 Q2  2026-17  2.80%   2.38%   2.26%   2.13%   2.50%   0.22%   0.14%
  2026 Q3  2026-30  3.45%   3.01%   2.89%   2.71%   3.16%   0.21%   0.14%

GPTBot is OpenAI, ClaudeBot is Anthropic, CCBot is Common Crawl, Google-Ext is a way for website owners to indicate they don't want their content to be used for AI, Bytespider is Bytedance, Bing and Google are the last two. Take these numbers with a truck of salt, haven't had time to verify them.

It's very clear that website owners do not like getting their content scraped and are indicating to GPTBot et al. that they are not welcome. It's a shame that CCBot is caught in the cross fire, but that's life. Bing and Google are doing just fine though, almost like having significant power in the search engine market gives you an advantage in other markets too. Who knew!

Long shot, but if there is anybody reading this from Salzburg, lmk if you want to meet up!

Reminds me of the idea of Radical Monopolies from Ivan Illich in a way. If a technology or service becomes so wide spread within society, even though many different versions of the technology or service may exist, a Radical Monopoly means that non users will suffer for their non use. Cars and non drivers in cities are the typical example. And I wonder, whether mathematicians who don't user theorem provers will soon suffer under the tyranny of the theorem provers, whether it be Lean or one of the others.

Looking forward to seeing how you handle it! I understand the complications of it as an engineer, I hope you figure out something soon!

You should talk with dang, as the email I received about this from him, to my eyes, does not agree with your stance here in the long term. I don’t want to get into the interminable blood quantum debate over Llm authorship either, however, I see substack doing something about it, and, as a long time hn reader, it makes me want to spend more time over there than here.

Yes, I am asking if that particular policy could be changed. A little AI here and there is fine, to each their own, but I would prefer not to read posts that are overwhelmingly so.

I ran this through an AI checker and it flagged half of it immediately. @dang, I know Substack just enabled Pangram integration, is there anyway you could get Y Combinator to spring for a Pangram subscription for the front page or something ?

Congrats! I very much enjoyed my time at Recurse Center back when it was Hacker School and idly daydream about returning some time. Thanks for making something so great!

Did you read the article? It provided a pretty good argument against exactly this attitude.

Would you say the same about something like, say, Spacemacs?

Great post! Thanks for the write up, enjoyed reading it and the visuals.

Yeah I was surprised a while back that ChatGPT was pulling them up (I was doing some research on origins of sudoku and it was pulled up very old threads on Usenet). So now I specifically ask for it and it consistently finds me some gold. Might take a few rounds of saying do deeper research but it often works.

Something that I have started doing lately is asking ChatGPT et al to check usenet for reactions from users about events (if it is the right 80's/90's time period). Sure enough, aol.sucks on usenet had some choice words about the outage:

>What does Cisco stand for?? Case's Internet System Crapped Out. That's right, Steve Case and his AOL pig fell victim to some mickey mouse networking equipment. Unfortunatly for AOL, they were the first ISP to feel real pain from using equipment made by Cisco Systems.

https://groups.google.com/g/alt.aol-sucks/c/iqjd7crtPs4 https://groups.google.com/g/alt.aol-sucks/c/K75nltM31Bw https://groups.google.com/g/alt.aol-sucks/c/vVup-HvlPWM

Here's a reporter asking for comments and getting laughed at and trolled: https://groups.google.com/g/alt.aol-sucks/c/mStonlu_H8E

Some more serious reactions over on comp.risks: https://catless.ncl.ac.uk/Risks/18/30#subj2 https://catless.ncl.ac.uk/Risks/18/31#subj3 https://catless.ncl.ac.uk/Risks/18/41#subj3

>Yesterday morning, I got a call because their mail system was backing up heavily. It took a while to discover the cause, but it turned out to be AOL. Because AOL's incoming mail from the Internet runs on relatively slow systems, and because they receive hundreds of thousands of Internet messages a day, they have 30 systems to receive incoming mail, all pointed at from the AOL.COM name. That means that any mail system trying to send mail to AOL would have to individually try all 30 addresses before giving up. Translate that to a 60 second (typical) wait for a connection timeout, and you've got a 30 minute time-in-queue for an AOL message.

nanog on seclists was an interesting read too https://seclists.org/nanog/1996/Aug/51

Flamewar over sendmail not handling outage well > Remember the AOL outage? One host built up a backlog of 2000 messages for AOL---but, because it was running qmail, it didn't even slow down. Meanwhile, sendmail users were choking on much smaller queues. https://groups.google.com/g/comp.mail.sendmail/c/TeNdv2laT94

Shouldn't we know a better answer to these questions once Anthropic's IPO materials surface publicly? I understand, and maybe even expect, SpaceX's materials to be all over the place and skate on by any discussion of unit economics, but the nerds over at Anthropic might just be forthright enough to just tell us what their margin is on tokens as part of their IPO.

Not affiliated with them, I just found it after casting around for a while trying to find something more or less exactly like this. Looks very neat, and like they made some good choices!