The AI Whale Fall and Open Source

Idea that AI is very expensive and will not be viable without "subsidization" for personal use is very shortsighted:

1. We measure "subsidization" in relation to API pricing. API pricing is very overpriced, to compensate for this "marketing subsidization", R&D, and everything else. A few months ago, Anthropic was close to being profitable

2. In general, OpenAI and Anthropic are US companies, and the US is known to have price disease, with all those huge salaries, multibillion note-taking apps, and fighter jets that cost too much to fly. China, and other countries in the future, could produce roughly the same output for way less money, as they did with everything else

3. Compute progress still happens. We can already run models better than GPT-3.5 locally on an iPhone

I don’t get this idea that AI will go away at some point. We’ll still have all the self-hostable models even if there are no newly trained models for a while. There will be companies that make money hosting those and selling tokens at a profit. We already have a pretty good idea what that looks like.

We’ll also have companies like Google and Meta where AI is not their main product, who get competitive benefit out of offering AI.

We’ll have companies trying to figure out how to update and train models more cheaply.

People aren’t just going to walk away from it all.

The author doesn't even agree with what the author says!

> Now, personally, the limited-time thing I think is bullshit–but, I’ll allow it for rhetorical reasons here.

So the main point was just a canard (a decoy duck) being used to urge open source devs to use LLMs. Like a phony time-limited offer. I guess it's nice to clearly and openly admit to bullshitting the readers, but in that case why do it at all? Honesty would be simpler: the idea is simply that LLMs could help open source, and this metaphor about whales falling into sunny hayfields doesn't pertain. It's more like "make hay whenever you're ready".

Then we might ask whether the LLMs can indeed be used to make hay or just to make a mess. In principle, when faced with an overwhelming backlog of trivial work, an LLM ought to be a great solution, but much depends on the work genuinely being trivial.

I use LLMs, and I use them for my Open Source projects, and I get the enthusiasm some people have for them.

But, I don't find straw man arguments interesting. They don't engage at all with why many Open Source developers don't want to use LLMs and won't allow LLM contributions to their projects (which are two different things; I regularly use LLMs, but I generally don't want or accept LLM-generated code from other people in my projects).

e.g. "Most-all of the people I’ve talked to that are strongly anti-LLM and anti-AI are oddly silent on the subject of models like GLM 5.2 and other modern open-weight models that are “good enough” for assistance now, and in another year will likely be as good as Fable or 5.6 today. I suspect this is because they are larpers and aren’t actually keeping track of the technology."

The people who are strongly anti-LLM may have many varied reasons for that position, and the models being open doesn't address most of those reasons. e.g. the ethics of ingesting everyone's writing and code, environmental costs, costs to communities where data centers are built, ending the developer pipeline by lopping off all junior roles, some of the worst people in the world becoming wealthy and powerful beyond human comprehension, the list isn't short and it doesn't end there..."open" models covers maybe one of those.

In short, the anti-LLM people are silent on the subject because it has very little, possibly nothing, to do with their position.

So, I even agree that we might as well enjoy the subsidized tokens the big guys want to sell us. My position is that I'm going to try to do some good work with them, use them with care to produce good code that survives without the best models, should it ever come to the point where very good models are prohibitively expensive. Fix security bugs in my projects where that matters. Fill out APIs and docs and translations. Just generally do the janitorial work that never seems to get done when it's just a couple of volunteers.

But, I'm not trying to convince the anti-LLM folks they're wrong. Some days, I agree with them.

There is only one 'falling whale reference' and it is this:

https://www.youtube.com/watch?v=THSY7-CxKnQ

    For mechanical work–things like bumping versions, fixing failing tests, checking documentation for inconsistencies–the clankers have proven their worth (even before LLMs, anybody remember dependabot?)!

    The way to fix that, of course, is automated processes and mechanical guardrails. Automatic enforcement of style, automatic testing and linting and formatting–same as ever. 

    We can use the whale fall tokens to build that machinery.

This seems to be the main contention of the post, however it leaves me confused as to what the author specifically thinks should happen. “We should have the AIs generate more dependabots”?

I agree completely.

>> Most-all of the people I’ve talked to that are strongly anti-LLM and anti-AI are oddly silent on the subject of models like GLM 5.2 and other modern open-weight models that are “good enough” for assistance now, and in another year will likely be as good as Fable or 5.6 today. I suspect this is because they are larpers and aren’t actually keeping track of the technology.

It's really pointless to try to engage with that sort of attitude: I'm right and everyone who disagrees with me is just larpers but I'll go ahead and strawman them so I can have something to write about online. Why should I read that? To waste my time and feel insulted?

Btw, I'm not anti-AI, but one doesn't have to be, to be critical of the current LLM market, but hey, I'm just larping eh?