Without a thriving mathematical community to point out these things it would have stayed broken. With automated math that community as tao pointed out is at risk.
What about software developer community? AI has eliminated the need for junior software engineers. Almost no one is hiring junior software engineers. But companies still need senior software engineers. Without junior engineers how will there be senior software engineers in the future?
What is the solution? I don't think the solution is to say AI progress in software, mathematics etc. should be halted.
I have a data pipeline with 6 steps, A -> B -> C -> D -> E -> F. I asked Codex to make some specific optimizations to step B and benchmark them. It did what I asked. Then it decided to also benchmark the entire pipeline, and after noticing that step E was slow it decided to make some optimizations that I had not asked for on step E. It was at this point that I wondered why it was taking so long, saw what it was doing, and stopped it.
This is GPT-6.1 Sol High.
Right, companies won't need software engineers. They'll just need someone who can use tools to produce source code and maintain the generated artifacts, plus make domain-specific technical decisions like "what should the system do when two users update the same record as the same time" or "how should the system behave when a message in the queue cannot be processed".
We really oughta come up with a job title for these people.
Who do you think the will be choosing whether the database uses a pessimistic or optimistic concurrency strategy? The CEO?
“ Claude! what should the system do when two users update the same record as the same time, explain to me with full clarity”
or "Claude! how should the system behave when a message in the queue cannot be processed? Give me all the possible ways ranked from best to worst, also explain to me all these concepts so I can understand as I don’t have a cs degree".
If you think that there is no future where software engineers don’t matter then you are delusional. While the future is not set in stone the pace and trendline of AI point to this future as a MORE realistic future then the alternative.
Your example btw is ALREADY a solved problem. AI can answer it and design around it. Agents at my company already handle our infra.
This isn't even the right question to ask, I think you've basically proved my point. You are in charge of deciding what the system should do when two users update a record at the same time. It's extremely dependent on what you're trying to do.
> Your example btw is ALREADY a solved problem. AI can answer it and design around it.
What's the one-size-fit-all solution for concurrency management that works for every single domain and application? I'm curious.
> Claude! how should the system behave when a message in the queue cannot be processed? Give me all the possible ways ranked from best to worst, also explain to me all these concepts so I can understand as I don’t have a cs degree".
Who's going to make this decision? The CEO?
It’s your example. I simply took your example and asked Claude. If it’s not the right question then don’t give it out as an example.
> What's the one-size-fit-all solution for concurrency management that works for every single domain and application? I'm curious.
I’m curious how your brain concocted I said that. Examine the context of our conversation. What I mean there is that AI can solve those questions for every possible domain application.
> Who's going to make this decision? The CEO?
Armed with Claude any non technical person can make this decision.
Knowing the right question to ask is what makes a person an engineer.
> What I mean there is that AI can solve those questions for every possible domain application.
Yes, if you know what to ask. You're doing an excellent job of demonstrating my point!
> Armed with Claude any non technical person can make this decision.
You just disproved that by asking the wrong question. Much like you, the CEO won't even know what to ask an AI.
Yes, but this is orthogonal to the point and that is: AI can do it too.
>Yes, if you know what to ask. You're doing an excellent job of demonstrating my point!
No it's your comprehension that needs work. You are missing MY point while being repeatedly getting enamored with your own point. My point is that AI KNOWS the questions.
>You just disproved that by asking the wrong question. Much like you, the CEO won't even know what to ask an AI.
I didn't ask a single question bro. I only regurgitated your examples. Much like AI, half your statements are based off of hallucinations.
I do not know or care what my if statement turned into in x86 assembly unless it becomes a performance problem and even then, I'm not profiling or debugging in machine language. Neither do most developers these days. A message in a queue becomes something akin to that in this era.
I see that I got downvoted there. This is not something I advocate or look forward to but I feel this is where it is going.
Likewise, you don't care exactly how an if statement gets converted into machine code, but you do know precisely what an if statement is and how it should behave, and could identify if it was buggy, and that that part of the codebase contains a bug. If you can't do that, then there is an impossible-to-estimate probability that at some point you get stuck and no progress will ever be possible. I don't see that as a winning strategy, in the long run (but it may work very well in the short term).
No it isn't lol. Have you worked on any real systems with customers? Good luck telling your boss at AWS that a poison pill message stopped the payment queue from processing so they lost $100 million in sales but hey, it's an implementation detail, no big deal.
I have. Those systems already fail in spectacular ways and people tell their bosses that some worker process stopped working because its transaction IDs overflowed.
I bet that sounds like `the flux capacitor stopped reticulating splines` which is already an implementation detail for the boss anyway. Nothing changes.
Or maaaaaybe.... an engineer?
agent herder!
If that comes to pass, I will have to re-evaluate my career options
I think that person does not need to know about locks anymore.
Some people hope AI will get good enough in a few years that it can innovate without human experts. Maybe? But that remains to be seen.
By the progress of AI from ChatGPT to now is horrifyingly fast.
Trendlines and basic reasoning point to a most probable future where the AI is superior. We can’t just say “that remains to be seen” because the alternative is the least probable future.
Anticipate the change and act prior.
In the age of extraction capitalism where building sustainable, profitable companies is not the goal, no one will care.
Human language is famously terrible at being unambiguous.
But that's the thing, you and the AI might be "on the same page" for one prompt, and then you aren't for the next.
> and yet judges and lawyers agree on how to interpret most of them
Um, no? Lawyers and judges frequently disagree on how laws should be interpreted. And it is often not written in a way that a layperson can easily understand.
> defined arbitration process to resolve any new ambiguities.
That is famously slow and expensive, and can resul in something very different from what the lawmakers originally intended.
LLMs eat away at all of these requirements.
The developers who knew how to write efficient low-level code found that there were no jobs for that anynmore, so today the developers who can work at that level are very few.
The same will happen with AI being the new abstraction. In another decade or two, very few people will be able to write code by hand. We'll need a rack of compute in a data center and multiple KW of power to do what we used to do on a desktop PC drawing a couple of hundred Watts.
AI progress in software
Your code used to be a masterpiece, so well crafted it's easy for AI to tweak and modify because you've got everything so logically organised and scoped... and now this is what you're producing? Hard to debug monstrosities that only an LLM can realistically bolt new features or tweaks onto, because it can do the kinds of refactoring necessary each time.
We use to talk about the fact that code should be readable because you spend more time reading it than writing it, but I think that misses the key part that readable code is also typically easier to debug. If you can read and understand the code, you can follow the logic when things are wrong in production, and you can more easily reason about the emergent properties of interactions between the complex systems that are involved.
I jest, while in agreement with this whole comment. We used to care about fostering informed developers and maintaining high standards and good quality software.
I literally compared it to being an expert woodworker. Beautiful ornate decoration. Rich, sturdy mahogany, one of a kind, beveled edges and a fantastic stained hardwood.
Now it’s the 30$ Ikea cardboard stuff.
My biggest question is how many tables does the world need, and how many woodworkers will be required to build + maintain those table factories.
If you buy a house or rent an apartment in most of the US, any furniture will be stripped out, even if the only thing you plan to do with it afterwards is throw it away. Then the new tenant provides their own furniture, which they either moved from elsewhere at great expense or had to purchase on the spot. No part of this makes any sense. We don't strip countertops when selling a house even if they're unfashionable, we don't replace white goods, but somehow it's expected for the furniture.
If you buy a house in Hawaii, it's understood that you're buying the furniture that's already in the house. I assume the reason is that it's more difficult to obtain new furniture in Hawaii.
If you rent an apartment in China, it will come with furniture, because how else are you supposed to live in it? And if you're not happy with the furniture it's shown with, you negotiate with the landlord for the furniture you need.
If Americans sold their furniture when they moved instead of throwing it away, you'd see everyone using much higher-quality furniture. It would have come with their house. And providing it to houses that didn't have it yet would be an investment, just like the countertops.
> If you rent an apartment in China, it will come with furniture, because how else are you supposed to live in it?
I think with AI, most people are getting lazy to do it properly.
The other day, I deleted 65K LOC that were dead code or stupid explanations over very obvious code from a vibe coded repository which had 95K LOC (but should have 10k imo)
Do you even see anything underneath that rose tint ?
I've waded through unfamiliar code at 3am trying to figure out just why the hell things broke this time more times than I can care to remember, particularly trying to tease apart the ways that code has organically grown compared to the original intent. If I'm subjected to one more piece of "spooky at a distance" injected behaviour code I'll probably scream loud enough to be heard half way across the country.
It's still just leaps and bounds more readable than what people are slopping together (I do have some co-workers that have been extremely tightly focused on avoiding "slop" with their AI and doing a lot of tuning, and it's certainly preferable to the ones that haven't)
And this is the pattern I've seen on most (semi)successful projects I've worked on in the past - I'd say correlation between financial success and code quality is 0 (up to a point where the whole thing doesn't fall apart). Once scale (both in load and in code size/features) starts mattering you're stuck building on a foundation of shit. LLMs are very good at identifying and cleaning up said shit layers, as long as you're steering them towards a desireable outcome.
Not saying it doesn’t exist, just saying this sounds dangerously close to a boomer talking about the 1950s
Luckier than me. I've worked at a few places where the code was simultaneously brilliantly written[0] and also unmaintainable nonsense that caused endless problems. One place had its own object model and ORM that absolutely no-one currently at the place understood and literally every bug filed (whilst I was there) could be traced back to that code.
(Probably just a coincidence that most of those places where Perl shops but I've seen it with Go too...)
[0] In terms of "cleverness", not in terms of "maintainability" or "readability".
Only once?
Oh god, we're gonna do the four Yorkshiremen, aren't we?
there are several features over the last few months that were obviously made and deployed and no one even launched the dev server and tried a single thing to verify if it was right. just "pull my ticket, do my ticket, push my ticket. i am a developer."
I’ve seen monthly and quarterly executing results AI generated with plainly wrong factual information.
Time unfortunately that you are simply not given because the whole point is to work faster so expectations of velocity have of course increased.
So you end up having to get a fuzzy idea of a given changeset, look at it a bit and evaluate quickly if it's a good change or a bad one and click it through, and on to the next thing.
At the size of PRs being whole features and doing this 5x faster than before, all you've got left as a senior is your Spidey sense. As a junior.. no idea how they cope.
We'd all love to still spend that 90% thinking time, but it used to be justified by necessity of getting the job done. Now, it's just not a luxury that is afforded.
What can ya do.. I'm slowly learning to be the best bot herder I can be, but it certainly feels like a change in skillsets. I definitely only feel like I'm able to do a good job due to enough experience and building and growing software projects over time to have an idea of what kind of decisions might bite us down the road, as well as to know what maybe can't be known but is just worth a risk. Without that kind of intuition I think I'd feel like I was really shooting blind.
IME a lot of people are now subject to output-rate expectations that preclude doing much else, honestly.
Some jobs will stick around in vastly diminished numbers with tasks that are completely different than what they used to be to produce the same output (e.g. farmer). Other jobs will be eliminated entirely (e.g. switchboard operator). I'm guessing things like software engineering will go the way of the farmer, with the main unknown being just how much demand for software there is.
In this case with AI that power will shift to the companies that run the AIs
The entire valuation of the AI industry is predicated on people not just losing individual jobs, but being taken out of the workforce entirely on an economic level.
They are talking about the workforce of the entire economy shrinking. People will lose their livelihoods for good.
This has the potential to be even worse than the second agricultural revolution to industrial revolution phase, which made ordinary workers lives absolutely miserable for maybe a hundred and fifty years.
This time, there will be no jobs. If you are displaced from one industry by AI, you will end up in another industry also being decimated by AI; if you get a job at all, you will do so by working lower pay than other workers, who will in turn be pushed down the ladder.
And that is if you are lucky: if you have only IT skills, why should you be the first to get a fruit picking or plumbing job?
Easy. Because you have the skills to increase productivity by automating it. Oh wait...
Ahh that's OK then. Everyone's in this same boat simultaneously in multiple industries! Cool!
> What is the solution? I don't think the solution is to say AI progress in software, mathematics etc. should be halted.
I think the solution from the maths world is to not grant these AI papers (or their human sponsors) the normal courtesies of "regular order", just as you would not with an AI lawyer or someone who was just pressing enter at a law firm.
But in the software world, nobody gives a shit, apparently. We are collectively morally bankrupt and should not be granted the regular order to help other people to decide what to do with us.
In fact, the world is always filled with curious people who like to go one level below.
This hysteria about losing "Junior Software Engineers" -- most of them in it for money, promotion rather than craftmanship, is over-rated.
People who love solving puzzles will always find ways to sharpen their mind.
People who love understanding things, will always find ways (AI will help them tremendously).
People who love taking shortcuts will always find ways for it (AI or not)
Eh? Apart from it not being what they are paid to do on their 9/9/6 jobs, when will they have the time to make it happen?
What is going to happen is that the remnants of the open source community will do the job of educating juniors for free, when the university degree system collapses. Just like it currently keeps a bunch of systems going with inadequate compensation.
The corporate world gets the problem off its balance sheet. Again.
The actual argument is "there will be no jobs for Junior Engineers, so there will be far fewer, and as a result there will be a huge shortage of Senior Software Engineers".
You are responding to the problem as if it some kind of extinction event, like a rare bird, where if we can find a breeding population we save the day. A few curious people, self training for the love of the game, and as a result we still have a few Software Engineers so everything is fine. It is not like that, and I haven't heard anyone suggest that is the issue. The potential problem is a massive shortage of workers with skills that are currently essential to the functioning of a large fraction of the economy, whom we might still need in the future.
The continued existence of talented enthusiasts does not establish an adequate workforce pipeline. If paid entry level experience contracts, what replaces it, and why should we expect that replacement to operate at sufficient scale?
If you have made it to the point of being a Junior Developer, I can assure you food and shelter is not a problem for them. You just have to adjust to a standard of living like the other 7 Billion people on this world.
Also, if a Junior Developer can show me(or anyone) they have built an entire system on their own and explain key concepts, there is no dearth of jobs for them
It's just a search problem now.
Trying to actually get the framing to be more reasonable is just hard because so many people have oversold it and large swaths of the public are sick of hearing about it.
My take is: the bar to what counts to HR as "senior" will go down as businesses everywhere try to adapt and hire more seniors - "senior" now just a name, as it becomes the new "junior". Then everyone will pat each other on the back till it all goes down in the flames of bankruptcy.
In other words, they are increasingly devauing their own senior position and discarding their hard earned skills that make them seniors in the first place.
There will be no more seniors. Tech companies will hire junior llm agent wranglers who took a class in undergrad doing this. That is probably the nearterm.
You have to do the hard thing eventually, or you never get anywhere.
Disclaimer: I am not a fan of AI, and I am currently writing software without any help from AI, as I prefer.
Would we need doctors, accountants or analysts? Probably not. At some point farming will be fully automated too, and so will grocery distribution and food preparation. At that point we will have arrived in the post-scarcity world. This is the promise of AI.
The problem is that the post-scarcity world will arrive gradually, not suddenly. Some jobs will be automated sooner than others. The ones that are not yet automated will expect payment for services. People who just lost jobs to automation won't have income to pay for those not-yet-automated services. But that's only until all jobs are automated.
There will be tremendous social upheaval and unrest during the transition to post-scarcity world.
Just for a thought experiment lets say junior hiring actually goes to 0% starting today and there is no other route into software dev for example maybe it is illegal for anyone under 22 today going forward to work in dev. Maybe something like ~2-2.5% of the workforce retires each year? And lets just define senior as 10+ YOE so 75% of the current batch of ~40 year working timeline. For simplicity lets just say the other 25% don't ever become senior devs and in 10 years the total number of senior devs has reduced 33% due to retirements. ChatGPT public launch was less then 4 years ago! Look at the ludicrous progress in the timespan. Even if progress suddenly massively slows or hits a wall it is currently hard to fathom it not improving at a rate of 3% a year.
More realistically I think we just have no ability to predict wtf things will look like 10+ years out at this point which is the point in this artificially constrained timeline where a reduction of senior devs just due to time would even start to be noticeable I think.
And that's a bad thing. If the math community didn't exist or was weak, OpenAI would still benefit from the prestige of these results they were forced to withdraw. Withdrawing these papers has harmed OpenAI's investors, and that's totally unacceptable.
> With automated math that community as tao pointed out is at risk.
Good to hear. The problem they represent needs to be eliminated.
This concerns the Hodge conjecture (millennium prize related) paper. Seems to me like PhD nerds weren't confident bosses pushed ahead anyway.
This is... logically equivalent to the claim it is offered to contradict.
Because this is what they say all the time. It's like a badge they have to wear and tell everyone they are wearing, even though we see it.
You can see the same thing with ANT. Had they looked at Mythos output, they would have realized there were only 76 items, not 79 like the bot claimed. Or the ones that were just a "it crashed" and nothing else (not a cve imo).
https://www.youtube.com/watch?v=NnV_cWeoo5Q (Linux Kernel team sharing their side of the Mythos "hacking" story)
Presumably the tip would be from someone who’s familiar with the area but doesn’t want attention. Which is unlikely to be someone in OAI.
Perhaps with AI.
A manual human check of each one would take a few month at least. In peer review, there are horror stories in math about more than 1 year before the journal accept the paper. So 3 reviewers x 700 pdf = 2000 mathematicians, that is 10%-20% of the community according to an unreliable count printed by Gemini after scrapping r/math.
Also, in most cases the only people that can understand the proof in a so short time (let's say a few months!) is the small group of people working in similar problems, i.e. the same group of 20-100 guys/gals that you meet in every conference.
Scientific progress used to be people debating and correcting other people. Now it's going to be people with AI assistance debating and correcting other people with AI assistance.
but do these 'people' need to belong to a thriving community or not to be able to do those things?
1. This is expected if you only use a single model family like Claude, eg. we use a different model family for code review than authoring, OAI could have done this too for their math dump
2. Ai needs a good human driver beyond the trivial or mundane, they are expert enhancing machines, not expert creating machines. This is where the community comes in. Reading Tao's ChatGPT session reveals this: https://news.ycombinator.com/item?id=49010345
3. OAI is not trying to be a member of the/any community, this is not the first story to shows this, nor do I expect it to be the last. Perhaps this is them being effective altruists today? /s
I agree LLM review is also fallible (as is human review) but the interesting part to me is that finding this sign error before publication should have been table stakes for OpenAI, it’s their own model that found the sign error.
I’m curious what was in the original prompt and what was in the prompt that led to finding the sign error, I think it matters a lot for understanding the dynamics here
As much as anything can be infallible.
If they can be automated, they are not necessary. If they are necessary, they won't be fully automated. It's a pretty simple experiment to run, the math "community" should bear with us. Darwin would be proud.
It's great that we're starting to see the light at the end of the tunnel, and will some day achieve a perfect market without humans. If you think about it, all the market really needs is a people to own everything, everything else can be automated, and all those annoying human workers can be eliminated.
To witness an arson and rejoice reveals an ugly kind of sadism.
I see why you might call it sisyphean, but I don't see what's ironic about it.
Even if your stated assumption was baked into the original comment, which is doubtful: the historical record shows that we will keep relearning The Bitter Lesson and each community will pretend what they do for a living is exceptional and immune because of xyz. The screams will get louder when the "greedy" and "dumb" automation comes knocking and it turns out nothing was truly immune or "nuanced ".
Getting some new hobbies may be in order, it's a Brave New World.
Bringing it back to math explicitly: you are essentially betting that the singularity is here, today, and that there are absolutely no downsides to breaking the pipeline which trains mathematicians (meaning that in 5-10 years at most there will be zero humans capable of assessing AI math output or independently advancing the state of the art).
The CS101 lesson in the first paragraph is appreciated, you should do it more often for us simpletons.
It is pointless to call out the little wins humans still have because again, those wins are temporary.
We need real concerted effort into asking: what is the point? For me the only answer I came up with is: fun.
An absolutely ridiculous statement. There is a vast amount of mathematical knowledge that hasn’t even been written down, much less formalized.
"If those grapes exist they are probably sour."
The same happens all the time in mathematics.
At the minimum you ought recognize that world class experts would disagree with your dismissiveness. Trying to explain the differing positions should not earn such immediate dismissal.
You're also failing to comprehend my passing comment about professors as a minimum criterion for the diversity of what reasonable opinions on the issue ought to like. I said it order to increase an open minded discussion, whereas you then used your familiarity with academics to validate your specific views. There's a world of difference there already, and metacognitively yours is the problematic one.
You’re distinguishing “knowledge” from “idea” in a particular way that doesn’t correspond to common usage (see my counter examples). Without you being explicit about your definitions, I can’t tell whether what you’re saying is meaningful. It feels tautological.
Given that an executive assistant has unwritten knowledge that is necessary to do their job, where your evidence that no mathematician has analogous knowledge (using the word in the common way, not whatever way you mean it)?
It’s possible, but it’s not as obvious as you seem to think.
We can dig into the philosophy of these definitions, but I think the far more interesting point is that even if we grant the existence of this kind of knowledge in the minds of human mathematicians, we have passed the threshold where that "knowledge" can keep up with systems that do not have access to it. Moreover, to claim humans have a "vast amount of mathematical knowledge" that is apparently valuable and that AI systems don't have, you'd have to prove that this "knowledge" is not implied or cannot be reverse engineered from the entire corpus of mathematical writing on which AI systems are trained. You'd also have to demonstrate that this "knowledge" leads to actual results that AI systems cannot generate without it. Given the results AI systems are producing, which are far beyond human ability at this point, it is more likely that AI systems have already internalized the entirety of this so-called "tacit knowledge" and then went much further, much faster, without humans in the loop at all.
On the question of a single word out of my entire position, that is a reasonably fair statement for mathematics (which is the topic under discussion). The fact that you think it's tautological supports both that you agree with its correctness and that this is a mostly irrelevant side conversation. And whatever you think the answer should be is not somehow not subject to the constraints of logic.
Not your contention that LLMs likely have something analogous to what you call “ideas” (you’re almost certainly right).
Mostly irrelevant? Dunno, maybe according to your rigid ontology ;-p
Cheers!