plaidfuji

God I love Yann. All of the AI fear-mongering is perpetuated by the two companies that stand the most to gain from it: OpenAI and Anthropic. It builds an aura of mystique around their products to juice their valuation and stay relevant in the news cycle, and simultaneously builds a case to regulate their competitors out of the market. Even the people who have quit the companies over their “concerns” probably still have RSUs and stand to gain from the publicity, especially if they’ve pivoted into AI safety research. Easy to delude yourself when it happens to benefit you financially.

People need to stop the absurdity of imagining AI as some out of control independent entity. Every job is kicked off by someone’s prompt. Every job runs on models and compute owned by people. Assign accountability where it’s due: GPT didn’t hack huggingface - OpenAI did. They wrote the prompt, built the sandbox and ran the compute. When you write a program that hacks another company, you are responsible. This doesn’t magically change with LLMs. Also, if their model is so smart, why didn’t they use it to design the sandbox? Or was it incapable? Or were the humans too lazy?

If you build the world’s fastest train, start it up with no driver and don’t finish the tracks, when it crashes, it’s just your fault. Not the train’s. So OpenAI saying “we’re worried AI will wipe out humanity” is basically equivalent to them saying “we’re worried we will wipe out humanity”. Like, seriously? Don’t worry, we’ll take care of it if you even come close.

If there is a risk of that catastrophic of a fire, why would I rush into strapping one of these to my house?

Murakami has a very different style than Butler… unlike western novels there is less focus on plot and pace, but as you’re noticing there tends to be an ever-present mood that builds throughout. I couldn’t tell you a single plot point from 1Q84, but I remember the feeling of reading it very clearly.

I liked The Windup Bird Chronicle slightly more, and it’s an easier read than 1Q84 to get into him. He also does short stories that are phenomenal.

I worked in this industry for almost five years. Even the “fermented” alternative meats (bacterial, fungal cell culture) which were basically a drop-in fit with existing large scale fermentation plants were still not cost-competitive with chicken. Some were on par with beef if you made some hand-wavy assumptions about scale.

The killer was usually the downstream processing required to clean out off-flavors and colors, which was required to get even a passable product (these are just protein powder fillers at this point, not whole cuts), but ballooned CapEx and cut yield such that cost went back up to like, A5 Wagyu levels.

I’ve sat in the awkward tasting sessions the author describes, and they’re all the same. The attendees muster up a “that’s pretty good” - but we all know nobody would ever choose to buy this unless they were forced to.

Now that I’ve been out of it for a while, it’s pretty obvious to me what the problems are. Cost, obviously. Taste and texture simply aren’t there. And “the knowing” is real. I never got over imagining the whole factory process involved in the production. I’m sure vegans would say the same about animal farming, though. So my conclusion was, the only way this technology becomes relevant is if people are forced to use it. And that’s either because you’re cruelty-free by choice (small fraction of the population), or because the cost of meat suddenly skyrockets, which today, is not looking that likely. Keep in mind as well that meat costs would likely be driven by feed costs in such a scenario, which also impacts cultivated meat. So it’s ultimately a process efficiency play, and you’re competing against eons of evolution.

“… since we called for a slowdown in AI research”

But stating it plainly like this would make the contradiction too obvious.

It is 2026. How are we still publishing articles on medical diagnostics data science and using area under the ROC curve as the primary metric of success. ROC-AUC of 0.9 under severe class imbalance (almost always the case in diagnostics) could still mean something like 4/5 predicted diagnoses are wrong (false positives). Precision-Recall curve + mAP or GTFO.

Science article in question: https://www.science.org/doi/abs/10.1126/science.aec6129

Also, the most interesting result here is that the CNN-based feature encoder significantly outperformed a vision transformer encoder backbone…

Occam’s Razor could also simply be that OpenAI and Anthropic are massively overvalued.

If OpenAI or Anthropic go away tomorrow, people find a new model and move on. If Google goes away tomorrow, people’s lives would be severely disrupted, and in the case of Gmail / Drive / auth access, even temporarily collapse.

This is the most grounded and coherent take I’ve seen on the actual realizable value of LLMs.. pretty much since they came out.

> the classes of firms that can accept the use of fully autonomous LLMs are few, by my count just three:

1. those who can accept failure cheaply: firms that would otherwise hire interns, firms involved in rapid prototyping work, etc.

2. those who need done a small set of narrowly defined tasks with existing clear guardrails: repetitive physical labor in a controlled environment, call center and customer service chat work, etc.

3. those that can accept or already do by nature the costs of rigorous specification and validation: chip design, drug discovery, and other domains where failure on deployment is an existential concern.

the first two classes are price sensitive and arguably don't need the jump in reasoning quality you see going from cheap to frontier models. most of these firms will be best served by open models running on cheap hardware, perhaps even locally at the site of use. for the first and third classes, the type of fuzzy combinatorial search that has produced headline results in mathematics and security research seems more sensitive to agentic swarm width than reasoning capacity

…

This is just so on point. And for the third class (which I would extend to things like materials research as well), specification and validation are already by FAR the larger costs, so automating search and simulation is really not a massive game changer for the broader business.

I’m wondering if they even see a coding agent as a valuable prize. It’s a competitive market in a race to the bottom economically, hard to establish consistent differentiation and virtually zero switching cost for customers.

I think they’ve made a shrewd move in focusing on search integration and everyday users (Gemini app) vs software power users. They have their corner and nobody is really competing with them, plus it feeds directly into their existing revenue stream.

> For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them. Super intelligence (the ability to have many smarter minds than your own reporting to you) has always been available to the wealthy and powerful.

This is an interesting framing, but when you put it this way, it makes it quite clear that intelligence without actionability is not so valuable. And capital is what creates action.

I can have the smartest doctor in my pocket, but if they can’t administer treatment, I’ll still die. I can have a world class scientist to talk to as long as I want, but they can’t build me a breakeven fusion reactor. A pocket lawyer… slightly more useful, but they still can’t take a case to court for me.

Talk and ideas are cheap. Always have been. Now they’re even cheaper. Implementation takes resources, and the more you have, the more useful intelligence is to you. Capital will be just fine.

> Here’s the thing: most of our habits at work still assume that producing something means you have to understand it.

This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level. You trust that those layers work. At some point in the past you couldn’t just trust that. At some point in the near future, you will just trust that the software code works. Instead you’ll just debate the spec (the prompt). Your thinking can move one level up the system hierarchy. Everyone’s can. We’ve done it before, we’ll do it again, and this won’t even be the last time it happens. Let go of the anxiety and let it happen.

1. They are massively unprofitable. It is a statement of fact. Nowhere did I say “can’t ever”.

2. Even their pursuit of this problem was itself unprofitable - $15M in compute to solve a problem with a $1M prize. Not that that was the point, but still.

The whole thing reeks of the desperation of an unprofitable venture-backed startup looking for its next PR win to keep the wind in the sails.

But I think what’s being overlooked in the race to claim absolute credit is that both sides ultimately relied on a LLM (and one of OpenAI’s at that). Either a human researcher made a breakthrough discovery with the help of Codex, or the latest GPT model made a breakthrough with the help of human training data, or a little of both… either way it is undeniable that LLMs have quickly become an integral part of R&D workflows and are accelerating research.

This would be a major win for any normal company. You could even build a bigger collaboration with this guy, give him a big budget and push for extensions to this preliminary result, and in return do a write up on how he uses your model in his workflow. Huge PR win. What this says to me is that their valuation is so astronomical that they feel the only way to justify it is to demonstrate a fully autonomous discovery bot… which it simply is not.

To me it’s morally ambiguous… if you hand parts of your thinking over to a tool like this (knowing full well the terms of service), of course the tool makers will want to claim some credit, and they do deserve it. But the bigger question to me is the scientific one: did their new model arrive at this result because it had closely-related training data from a human, or did it extrapolate to this line of thought on its own? The answer says a lot about how valid their claims of “AGI” are vs. a very fortuitously cherry-picked example.

It would actually be a really interesting study, if they would ever be willing to be transparent about this, how the result differs with and without his conversations in the training set. How quickly it arrives at the result, whether it takes the same approach, etc.

More likely ten years from now you’ll think, “man, we were pretty naive to think we were on the cusp of AGI back when LLMs automated low-level software engineering.”

Bingo

I believe Gemini Flash is smart enough to know when to ground with web search. Their app has been saying it’s running a web search on almost all of my queries since 3.6. And given that Google … is Google, I trust them with web search grounding more than anyone else.

I think the sentiment is misplaced here (there is a legitimate concern for IP protection), but this is my absolute favorite line from Silicon Valley - small correction though: “… makes the world a better place better than we do”

> The week before Christmas 2025, five mathematicians were holed up in a classroom at ETH Zurich.

So happy this didn’t end with, “tweaking their prompt for ChatGPT Sol” or whatever.

> He, Diskin, and Radhakrishnan made some progress and brought their results to Sudakov and Tassion. As Tassion took in their work, an idea — perhaps an outrageous one — formed in his mind. … Over those weeks, the collaboration became frenzied. The mathematicians traded ideas constantly, often texting late at night.

It will be a truly sad world if we automate this away.

This seems like a pretty straightforward and easily winnable situation for GitHub. The demand for their services just doubled, apparently. They have no real competitor operating at the scale they’re at. They have pretty substantial network effects.

They are under no obligation to continue functioning as a bottomless free repository for text file hosting, especially now that text file creation has multiplied exponentially. They could make a few almost purely commercial changes and solve this without any major re-engineering while maintaining their status as the go-to public / open source code hosting platform.

1. Immediately increase pricing of all enterprise licenses and add super-committer overage fees.

2. Rate limit or cap commit size / frequency for public accounts.

Their service is more valuable than ever and switching is much harder if people have automation built up on their platform. Now is the time to cash in their chips.

And the positive externality of increasing commit cost would be forcing people to have some semblance of restraint for the AI content they generate.

This is so random. I had just pulled this exact page up this morning to settle a debate with coworkers over whether the product names were real Swedish words or not. I thought they were all made up!

Was very surprised to find things like “bookshelves are all men’s names”, “all names must have ä, å, or ö”, etc.

Complete coincidence that this is #2 today. Had to check that ’NaOH’ wasn’t one of my coworkers

In chemicals and materials, 50 rows of good data is a really solid study. That’s e.g. a 3x4x4 experimental design (assuming replicates for each condition get averaged into a single row). If you managed to prep that many samples correctly and obtain consistent characterization data across all properties of interest, you’ve easily got a paper. It’s also kind of malpractice to jam this type of data (few samples, wide rows) into modern ML models. There are plenty of simpler statistical methods that will tell you what’s going on, and even then a well-made plot might be good enough. The difficulty is not in drawing insight from the final numbers, it’s almost always in how those numbers came to be in the first place.

Thus the reticence of science-oriented companies to invest heavily in these mass data-gathering exercises to feed ML. It’s damn expensive, and almost always leads you back to raw data issues, not breakthrough discovery. Doing it without a set purpose in mind is even more likely to yield garbage.

> “…the focus of AI in drug discovery must shift from doing what can be done - such as modelling data that is readily available, but that is unlikely to move the needle - to doing what should be done, even if this requires, for example, substantial data generation…” It’s a worthy goal, but I think that many involved in this work might be thinking, even unconsciously, “You first”.

This is the problem with AI for all of science - not just drug discovery. Applied ML has spread like wildfire through academia over the past decade - this started well before the LLM hype. It’s the perfect honey trap: research is painstaking and slow, ML offered a shortcut, and best of all, it just needs data. Research produces lots and lots of data! Surely this will be a match made in heaven.

I’ve watched the same pattern play out at least four or five times now in various roles.

(1) Propose an ML-guided approach to material/chemistry discovery/optimization.

(2) Gather existing data (real, experimental data).

(3) Realize there’s less than about 50 true rows of data on the outputs of interest.

At this point, you either: (4a) revert to traditional methods but keep the veneer of using ML to save face, or (4b) pivot to computational/simulation work or a high-throughput system that’s very far removed from your original problem, but allows you to keep playing with ML toys

It’s really bad. I left the industry. I don’t know how long it will take for people doing real science to take back the reins (and the funding).

That means they’ve learned how to say “no” more diplomatically :)

This assumes that writing standalone apps (what is now implied by “realizing an idea” in the AI hype-o-sphere) still has defensible value

Poison can act on more than just your stomach

If this were true, food companies wouldn’t have a Regulatory team. Except every single one of them does, and they function similarly to how you describe, but without dissolving. They’re treated as a constraint that must be checked before major projects can advance. Sometimes they’re the longest-lead item on a new project and if you don’t get them involved early, your project can sink at a late stage after great expense.

The only difference is AI isn’t regulated, so they just have a vague “ethics” department with no real teeth because it’s essentially PR and has no legal consequences to back up their stance.

I would be genuinely very impressed - but still not scared - if the Riemann hypothesis were solved. I suspect that we may require “new math” to make progress on that. If a new operator / symbol is required, is that fundamentally not doable by an LLM because it’s outside of current tokenization space?

Any computable problem will eventually fall to computers.

LLMs have made math proofs more computable, in the sense that a computer can both generate potential solutions and check the validity of its solutions on its own, with a reasonable chance of converging on something correct. I assume this was already doable to some extent, but it seems like it’s now exponentially easier. That still doesn’t mean that all math is automatically solved.

This is somewhat similar to things like molecular dynamics or protein folding or finite element simulations, etc. Some problems that were previously intractable via computation became tractable. Others - the vast majority of other problems - remain unsolvable by these computational techniques, because the scale of compute required is beyond imagination. These are simple things like simulating the dynamics of a cubic millimeter of water molecules for 1 second. Unfathomably beyond current capabilities (and LLMs aren’t going to change that).

I think LLMs are great, I use them every day and I think they have a ton of value. But if these things were as revolutionary as people promote/fear them to be, you should immediately point them at the highest value math problems and see progress. Like the Millenium Prize problems. Haven’t seen a solution to those.

So there are limits - but we’re about to learn a lot about the new normal of what constitutes a layup math proof vs the truly difficult.

I think the manufacturing analogy is apt. If you consider both the need for minimal COGS and the large R&D expense, it’s kind of like… biomanufacturing. You spend a lot on R&D to develop your bug, which at the end of the day is just a very complicated sequence (DNA vs model parameters), and then you expend a ton of capex (or rent someone else’s capital equipment) to produce a relatively commoditized product (a drug, a protein, food…) at as large scale and low cost as you can.

But that also makes the case for government subsidization + bailout, if you accept it as a capability critical to national security, but one that may not be profitable to run within the US on its own merits.