plaidfuji

This seems like a pretty straightforward and easily winnable situation for GitHub. The demand for their services just doubled, apparently. They have no real competitor operating at the scale they’re at. They have pretty substantial network effects.

They are under no obligation to continue functioning as a bottomless free repository for text file hosting, especially now that text file creation has multiplied exponentially. They could make a few almost purely commercial changes and solve this without any major re-engineering while maintaining their status as the go-to public / open source code hosting platform.

1. Immediately increase pricing of all enterprise licenses and add super-committer overage fees.

2. Rate limit or cap commit size / frequency for public accounts.

Their service is more valuable than ever and switching is much harder if people have automation built up on their platform. Now is the time to cash in their chips.

And the positive externality of increasing commit cost would be forcing people to have some semblance of restraint for the AI content they generate.

This is so random. I had just pulled this exact page up this morning to settle a debate with coworkers over whether the product names were real Swedish words or not. I thought they were all made up!

Was very surprised to find things like “bookshelves are all men’s names”, “all names must have ä, å, or ö”, etc.

Complete coincidence that this is #2 today. Had to check that ’NaOH’ wasn’t one of my coworkers

In chemicals and materials, 50 rows of good data is a really solid study. That’s e.g. a 3x4x4 experimental design (assuming replicates for each condition get averaged into a single row). If you managed to prep that many samples correctly and obtain consistent characterization data across all properties of interest, you’ve easily got a paper. It’s also kind of malpractice to jam this type of data (few samples, wide rows) into modern ML models. There are plenty of simpler statistical methods that will tell you what’s going on, and even then a well-made plot might be good enough. The difficulty is not in drawing insight from the final numbers, it’s almost always in how those numbers came to be in the first place.

Thus the reticence of science-oriented companies to invest heavily in these mass data-gathering exercises to feed ML. It’s damn expensive, and almost always leads you back to raw data issues, not breakthrough discovery. Doing it without a set purpose in mind is even more likely to yield garbage.

> “…the focus of AI in drug discovery must shift from doing what can be done - such as modelling data that is readily available, but that is unlikely to move the needle - to doing what should be done, even if this requires, for example, substantial data generation…” It’s a worthy goal, but I think that many involved in this work might be thinking, even unconsciously, “You first”.

This is the problem with AI for all of science - not just drug discovery. Applied ML has spread like wildfire through academia over the past decade - this started well before the LLM hype. It’s the perfect honey trap: research is painstaking and slow, ML offered a shortcut, and best of all, it just needs data. Research produces lots and lots of data! Surely this will be a match made in heaven.

I’ve watched the same pattern play out at least four or five times now in various roles.

(1) Propose an ML-guided approach to material/chemistry discovery/optimization.

(2) Gather existing data (real, experimental data).

(3) Realize there’s less than about 50 true rows of data on the outputs of interest.

At this point, you either: (4a) revert to traditional methods but keep the veneer of using ML to save face, or (4b) pivot to computational/simulation work or a high-throughput system that’s very far removed from your original problem, but allows you to keep playing with ML toys

It’s really bad. I left the industry. I don’t know how long it will take for people doing real science to take back the reins (and the funding).

That means they’ve learned how to say “no” more diplomatically :)

This assumes that writing standalone apps (what is now implied by “realizing an idea” in the AI hype-o-sphere) still has defensible value

Poison can act on more than just your stomach

If this were true, food companies wouldn’t have a Regulatory team. Except every single one of them does, and they function similarly to how you describe, but without dissolving. They’re treated as a constraint that must be checked before major projects can advance. Sometimes they’re the longest-lead item on a new project and if you don’t get them involved early, your project can sink at a late stage after great expense.

The only difference is AI isn’t regulated, so they just have a vague “ethics” department with no real teeth because it’s essentially PR and has no legal consequences to back up their stance.

I would be genuinely very impressed - but still not scared - if the Riemann hypothesis were solved. I suspect that we may require “new math” to make progress on that. If a new operator / symbol is required, is that fundamentally not doable by an LLM because it’s outside of current tokenization space?

Any computable problem will eventually fall to computers.

LLMs have made math proofs more computable, in the sense that a computer can both generate potential solutions and check the validity of its solutions on its own, with a reasonable chance of converging on something correct. I assume this was already doable to some extent, but it seems like it’s now exponentially easier. That still doesn’t mean that all math is automatically solved.

This is somewhat similar to things like molecular dynamics or protein folding or finite element simulations, etc. Some problems that were previously intractable via computation became tractable. Others - the vast majority of other problems - remain unsolvable by these computational techniques, because the scale of compute required is beyond imagination. These are simple things like simulating the dynamics of a cubic millimeter of water molecules for 1 second. Unfathomably beyond current capabilities (and LLMs aren’t going to change that).

I think LLMs are great, I use them every day and I think they have a ton of value. But if these things were as revolutionary as people promote/fear them to be, you should immediately point them at the highest value math problems and see progress. Like the Millenium Prize problems. Haven’t seen a solution to those.

So there are limits - but we’re about to learn a lot about the new normal of what constitutes a layup math proof vs the truly difficult.

I think the manufacturing analogy is apt. If you consider both the need for minimal COGS and the large R&D expense, it’s kind of like… biomanufacturing. You spend a lot on R&D to develop your bug, which at the end of the day is just a very complicated sequence (DNA vs model parameters), and then you expend a ton of capex (or rent someone else’s capital equipment) to produce a relatively commoditized product (a drug, a protein, food…) at as large scale and low cost as you can.

But that also makes the case for government subsidization + bailout, if you accept it as a capability critical to national security, but one that may not be profitable to run within the US on its own merits.

Right. It also makes it sound like they’re not using the Haber Bosch process… but then:

> Electrolyzers are located on-site. They use wind energy to split water into hydrogen and oxygen gas.

> Additionally, nitrogen gas is directly harnessed from the atmosphere using an air-separation unit.

> Hydrogen and nitrogen are combined under pressure, resulting in carbon-free ammonia fertilizer.

That sounds a lot like the Haber Bosch process to me. So they’re just saying, rather than divert excess generation to a battery, you could instead build a micro H-B plant on site to produce ammonia.

I doubt this makes any sense in the grand scheme of things as you achieve major efficiencies at scale for chemical plants. Makes more sense to just electrify existing ammonia plants and ensure they use renewable electricity sources.

The green chemistry industry is full of penny-wise pound-foolish concepts like this and it drives me absolutely nuts.

> I don't think most software is like solving a math problem or series of math problems.

I agree with you when talking about high level software design. As you say it ultimately boils down to building something people will pay for, which is a fuzzy correctness function that is hard to measure within an agentic sandbox.

But unlike other professions, there are a lot of sub-problems within software development that are able to be fully specified and tested via text generation. And I think the developers of AI overestimate how many such problems exist for other professions. What I’m saying is most other professions tend to be “fuzzy all the way down”… which incidentally is why they select for people with fuzzier skillsets. Or in other cases, like physical engineering, the correctness is quantitative, but the necessary I/O integrations and physical automation lower the ROI of agentic workflows considerably.

It seems like a solid set of criteria for how easily a task can be automated by AI agents is:

- extent to which correctness of solution be easily specified and checked

- extent to which new potential solutions can be implemented as text

- extent to which prior art exists online

This basically maps to software engineering and math. I think a fair bit of AI hype comes from the fact that the very architects of AI are the people whose jobs are most easily automated by AI. They think, “if my job receives this much of a boost from AI, surely every job will be the same”. Ironically it couldn’t be further from the truth… and likewise the predictions of widespread labor obsolescence

Does anyone really believe he’s doing all of this just to sell out and live on a beach somewhere? This dude sleeps at his own factories. He literally works like 24/7 and has no personal life. If this was all a cash grab he’s had dozens of opportunities to cut and run with well beyond F U money. If you wanted to scam people there are a lot easier ways to do it than repeatedly founding revolutionary technology companies.

I’m not denying that his companies are awash in zany financials, but I don’t think that’s ever been the point

As much as it’s tempting to read some kind of ulterior motive into this, I think the most reasonable explanation is that AWS, as perhaps the single biggest point of failure in the backbone of US IT infrastructure, has legitimate concerns about its ability to fend off attacks from bad actors armed with the most advanced models.

> The first requirement is that the computer program has a body (either physical or virtual) and sense organs

Ok, deploy a local model on a lightweight edge compute device and strap it to a chassis with wheels, and attach a cheap webcam

> Then I’d want to see an embodied agent that could navigate its environment in order to survive as well as, say, a lizard can

Give the robot appendages that enable it to plug itself into a standard wall outlet, guided by a vision model plugged into its webcam. As long as it can feed itself, it can survive long enough.

> Next I would want to see an embodied agent with the same capacity to deal with novel situations as a mouse.

I think if you fed frames from the webcam into a local VLM every 5s you’d be able to assess a situation and respond with simple actions (turn, advance, retreat).

> After that I’d want to see agents whose social dynamics are as complex as those of wolves, and then agents with the tool-making abilities of chimpanzees.

Social dynamics could be implemented in many ways, maybe by transmitting tokens over RF? Idk. Then you have a scanner that picks them up, feeds them into some LLM frontend and decides whether to add them to a global context file that guides the VLM action-taker. A new action could be to broadcast a token message. Tool-making would have to be code-based. Physical tools are hard. Still unsolved.

> At that point I would want to see people successfully teaching such embodied agents how to communicate their desires

This part is relatively straightforward except for the “via nonlinguistic modality”.

Anyway. These are all engineering problems. Personally I would demand to see the AI reproduce its body under its own power and volition. That’s a pretty neat trick we’ve got going for us.

To put that in perspective, if you assume a token is 4 bytes, that’s about 164 MB/s of traffic, which sounds a bit less staggering.

> Altman’s realization was born partially out of an experiment of his own. He tried delegating his Slack and email responses to AI, then began responding to some again manually.

Well I’ll be..

There may be additional major leaps forward, and there may not. I kind of struggle to imagine what the next step actually is. Certainly there will be improvements in performance (speed) and cost. But at a point you reach a barrier where the limiting factor is the specificity of the human prompt and our ability to manage all the code we’re generating.

Somewhat oversimplifying; writing software and building apps was a bottleneck - now it is not. What is the next bottleneck that LLMs can solve? Is there one? And is there enough publicly available data to solve it repeatably at scale? Or did we just automate stack overflow searches and now we’re stuck again?

Or is the endgame of this innovation cycle the complete removal of interaction with machines through code? Will we simply interact with machine coworkers purely through natural language? Can an LLM make PowerPoint slides and run a meeting? So far not seeing much progress on that.

Our shop is forced to use Copilot on gov cloud, and it’s so useless I usually stick to manually coding. Its syntax is messy, it randomly combines lines together, flips order, or drops a couple tokens worth of output in the middle of a line, and for some reason it consistently drops the last line of every code block. I assume we’re getting a few versions back of GPT under the hood. But it does make me appreciate how the models of the past year or so crossed the threshold from interesting to truly productivity-enhancing.

Between Copilot, Claude, and Gemini, I still actually prefer Gemini. I do a lot of scientific writing in addition to coding and Gemini is the only model I can trust to “just be right”. This trust then transfers over to its code output.

kind of sobering to realize that whether your job can be profitably automated away comes down to what $/token some hyperscale AI provider can deliver… I suppose it’s nice that this article highlights some upward pressure on that number.

It’s the exact same mental gymnastics that cause people to accuse model providers of large-scale plagiarism.

That is to say, not that much gymnastics. Like a cartwheel at most.

This comment couldn’t be more ironic given the content of the post

If I’m understanding this correctly, it starts by raising this question, then argues that it’s kind of a cheap and meaningless punch-down.

I’m not sure I completely agree.. I think these types of questions are more a response to the level of hype around LLMs and less about knocking people down. You see a lot of people excited about the personal productivity app they vibe-coded (which IMO is totally legit - it’s cool that run-of-the-mill apps that used to require a professional developer are now available more on demand), and yet it’s hard to think of a new piece of high-impact traditional software that has come out since the release of ChatGPT.

But it’s also hard to think of the most recent piece of important traditional software that came out… at all. I couldn’t even name the most recent Photoshop-like release. Ableton / Fruityloops? Tableau? Big pro-sumer apps kind of plateaued in the early 00s.

LLMs have made it easier to develop software, but at the same time they’ve also raised the bar of what’s worth writing software to do. Many things that used to be apps are now just prompts. Maybe ChatGPT was the next Photoshop - it turned writing basic apps from a profession into a hobby.

Anyway. Good post - definitely not written by an LLM, and that’s a good thing.

They offered him two free hour-long rides to their facility and back just to pick up the suitcase? Waymo trouble than it’s worth

It’s all moral relativism

> Teams is where most enterprise work happens: decisions get made, customers get answered, and projects move forward there.

… and Teams is where it will stay! Microsoft’s piss-poor integrability across its productivity suite is the single biggest reason agentic AI will fail to deliver its promised productivity benefits. You probably could build an agent to do a lot of people’s jobs, provided it had unfettered access to Teams, Outlook, Word, Excel, and PPT. But I can’t think of an organization who would want to grant that access today, and even if they wanted to, the solution here is to expose a self-managed HTTP endpoint for all your Teams traffic? Seriously?

No, you’d remark that your house has appreciated in value over the past 20 years. But you wouldn’t have realized any of that gain until you sold the house - the point being that the realization is the actual taxable event, which is why it matters from the pedantic technical accounting POV. The fact that you turned around and bought another house just means you’re doing something new with your realized gains. Now you have a new cost basis. Maybe that’s what you’re saying with “unrealized gain” though.

> SpaceX's valuation only makes sense if

It’s funny, I hear the exact same phrasing used when justifying Tesla’s valuation. “It only makes sense if…” … if you ignore what the actual, physical business does today, and picture it doing something entirely different, beyond its current capabilities (robotaxis, androids, etc)

The difference with this pie-in-the-sky ambition (Mars Colony) is that I don’t even understand how it would be profitable if achieved. What do you get from a Mars colony? What on earth (no pun intended) could you extract from it that would command that amount of value? This isn’t like colonization of the americas, where there was a trove of readily available natural resources to extract and sell back to the mainland markets - nothing is going to get shipped back from Mars any time soon. A Mars colony could only be supported through significant public investment - so is the valuation justified via the expectation that SpaceX will be the primary vehicle for public investment in Mars exploration, or through the centuries-long payback period of founding a self-sustaining civilization? Or both?