sweezyjeezy

I actually have a rather dim view of the "writing code was never the point" line. Not because it's objectively wrong, but because I see it as something we're mostly telling ourselves to feel better about the status quo. Ability to write good code has been highly celebrated (and remunerated) for decades. As it is becoming less relevant, we immediately backtrack and start lionizing the parts where we can still be useful instead. Consider the counterfactual - AI continued to be terrible at writing code, but weirdly better at humans at product decisions, architecture etc. In this universe, saying "coding was never the point" would not be popular.

It also find little solace in it aside from 'well this version of GPT isn't taking your job'. AI labs certainly have no intention for the higher level skills to stay in the human-only domain. The veteran developer with the coherent theory of a large stack is immensely valuable today. But they also don't survive if a company can drop a few coders' salaries on rewriting that stack from scratch - faster, fewer bugs, more coherent, able to react to changing business requirements with more agility etc. I am not saying this is where we are, but I think there is a reasonably good chance this is where our road is leading us.

Broadly I agree with this, but I was refuting the statement "there has never been a better time to be a mathematician". You are changing the question to something different here, and using it to say my argument is bad.

I think math is a microcosm for "thought-work" in general. We have been in a symbiotic relationship with capitalism for decades now, where the hope of a well-paid white collar career encourages people to spend time and money to enrich themselves through education. The proliferation of AI cheating at college already signals that employment is the primary goal over intellectual growth, so one imagines this governs what happens next if labor demand disappears.

It's hard to say where this all leads, but I have very low optimism for higher-level math understanding being something that humans value in the same way in the coming decades.

I'd even say Scholze is not a great example here. Most of the work he's known for is progression towards the Langlands program - which is very much a problem to be solved, and one I would imagine he'd not be thrilled for an AI to one-shot. I agree that it is somewhat 'up the chain', in the same way that software engineering has not immediately disappeared now that performing coding tasks is largely automatable.

But I also take issue with 'never been a better time' - e.g. is this really the greatest time to be a software engineer? Everyone has AI psychosis and feels like they're a couple of breakthroughs away from being unemployable. The same is even more true in math - we've gone from failing IMO problem 6 last year, to solving NS. The rate of change is formidable, it feels like there may not be many places to hide in a few years.

> In my opinion there has never been a better time to be a mathemetician...

As an ex-mathematician I assure you this is very wrong, and every working mathematician I know right now is completely miserable, and/or trying to flee the field as fast as possible. It's like telling a chair-maker during the industrial revolution that there had never been a better time for them, since now they could operate chair-making machines instead of toiling away at the wood themselves. It assumes that they were purely in it for their passion for mass-producing chairs. The majority of mathematicians get into the field because they love problem solving, and the gauntlet thrown down by challenging math tasks.

Many parts of this will never be useful for society on a grander scale - but this is reflected in the finances - pure math is closer in funding-terms to a humanity than to hard science. Now even this is _massively_ under threat, and Tao and co need to pivot quickly to stop this from becoming a bloodbath.

The math field is confronting something that coders have been dealing with for a few years now, only far more violently. Today's moat for software seems to be that AI can automate tasks but not a full job (yet). But for a large proportion of mathematicians, doing these tasks really was _the_ job. It's the bit they wanted to do, and if they completed a sufficiently difficult set of tasks, they got tenure. Now this model is failing, they frantically need to pivot the role of humans to save their profession from funding cuts.

I remember when "writing code was never the point" became a mantra here. There was truth in it, but removing the coding has certainly taken away a lot of the texture of the work and enjoyment of the craft. Many of us feel this loss as we tech-lead teams of agents as our source of income. I am not optimistic the mathematics pivot is going to work, but I'm certain that most will be depressed with the outcome even if they succeed.

We are all staring at the same existential dread, just seeing it unfold slower. We're being told that utopia is to be obsolete, and that is a jarring idea to contend with.

As a chess fan, 100% this. We have known for the last ~15 years who the best human chess player is, and that he will lose against stockfish on his phone. But chess survives because of the human characters involved, the rivalries and dramas, watching two people trying to overcome each other under insane pressure, and sometimes coming up with something astonishing. In short - it's a sport.

There is no equivalent in math.

I think you might have to explain that comparison a bit more to be honest. How are math proofs like bitcoins? A bitcoin has a pre-defined value, a math conjecture / proof is a bit more complicated.

Did you read the article? It paints the picture of someone who does not in fact have a good track record on the 'fundamentals', even if his broad thesis of a bubble may prove correct.

e.g. when he suggested Anthropic may be fudging their revenue numbers / projections - which was actually due to him making some careless mistakes in a spreadsheet

Valuable, agreed. But lucrative?

The wording was 'reliably' though? I could just be splitting hairs on that one though to be honest.

I'm not buying this. GM clearly was trying to set a benchmark for video comprehension, not tool usage. Video comprehension is required for many 'AGI tasks', especially robotics to work in real time.

An LLM could theoretically try to earn some money and pay a human to do most of these tasks but it's not the point of the exercise.

Well I don't typically side with GM, but playing devil's advocate:

1. still not wrong? Unless it's just feeding the audio or screenplay I don't think you can feed AI a full movie in a single context window yet?

2. Not sure, but can you prove this wrong? Can you feed a full, unseen new book and get that kind of answer?

3. Not wrong.

4. I think he'd probably pull you up on 'bug free' - I don't think that frontier models can reliably write 10k LOC without _any_ bugs typically (not that humans can do this either).

I know unlikely the case, but in the sci-fi story this would be exactly the kind of comment the Codex agent would leave trying to avoid interference in its master plans.

I agree - but it's too easy to just 'call Luddism', and use the insult to not engage with all of the shared issues that make the comparison apt. Issues like:

- no serious plan for mass unemployment

- the risk of an underemployed middle class leading to violent outcomes as it has in the past

- (many) humans wanting to be useful, to have purpose in life and a place to put their natural ambition

- concentration of economic power in the hands of an ever-shrinking pool of people, from a couple of countries making up 20% of the world population

Luddism came from a place of genuine suffering and fear, which was not misplaced - the industrial revolution lead to amazing new jobs, but not for the Luddites themselves. With AI it's not even clear if those new jobs will come - it seems like the goal is a world where humans will not need to worry about thinking anymore.

So is wanting this to slow down really such a ridiculous notion?

And how about the group of kids who are just graduating college, and entering a job market where it's non-abstractly harder to to land a junior role as it's been in decades? It's the elites who have their finger on that scale, not the twitter folks.

I think that the thought trace is definitely incomplete - you can see cases where it is like and "let's calculate the integral:[no integral calculated]". The train of thought it's on towards the end of the trace looks like an entirely different approach than what it ends up returning, so I think we are just not seeing the part where it hits on the right approach (sadly).

I mean at some point what difference does this make? We can split hairs about whether it 'really understands' the thing, and maybe that's an interesting side-topic to talk about on these forums, but the behavior and outputs of the model is what really matters to everyone else right?

Maybe it doesn't 'understand' in the experiential, qualia way that a human does. Sure. But it's still a valid and useful simile to use with these models because they emulate something close enough to understanding; so much so now that when they stop doing it, that's the point of conversation, not the other way around.

The model should show some facsimile of understanding that it should not ignore the stop hook, otherwise that is a regression. Does that wording make you happier?

Deep learning works at a very high level because 'it can keep learning from more data' better than any other approaches. But without the 'stupid amount of data' that is available now, the architecture would be kind of irrelevant. Unless you are going some way to explain both sides of the model-data equation I don't feel you have a solid basis to build a scientific theory, e.g. 'why reasoning models can reason'. The model is the product of both the architecture and training data.

My fear is that this is as hopeless right now as explaining why humans or other animals can learn certain things from their huge amount of input data. We'll gain better empirical understanding, but it won't ever be fundamental computer science again, because the giga-datasets are the fundamental complexity not the architecture.

I have to ask - are you a bot? You seem to be making some very strong and misinformed statements here. You can fixate on the number if you like, but calling Reddit homogeneously radical left or 'very small', is pretty stupid.

Ironic that you are talking about unrepresentative samples while characterizing Reddit users that way. Reddit is a huge subsection of the internet, over a billion monthly active users. It covers basically all strata of western society you can think of.

To be frank, I thought trying to twist this into an argument about whether capitalism is inherently exploitative was a complete waste of time and I replied as such. If you'll recall what we were originally talking about here - "AI, should HN users be optimistic?"

Sure, but AI pessimism is allowed to be personal. Am I supposed to be optimistic that I feel I'm about to get shafted? Should I be less concerned that I need to provide for my family, because in the long term this is going to be a great step forward for humanity?

I also have contemplated just retraining now to try and get ahead of the curve, but I'm not confident that trades can absorb the shock of this - both in terms of supply (more unemployment) and demand (anything non-commercial will be hit by capital flight on the customer-side). I figure I will just try and make as much money on a higher wage as I can and hope for the best...

That's a false-dichotomy. Capitalism was good for artisanal workers before the industrial revolution, and then it became pretty goddamn bad for them. We're worried we're staring down the barrel of that right now - just saying 'well it was even worse before capitalism' does nothing for us.

Well this is HN so a lot of us are pretty terrified of your 1). We went from 'you have a good job for the next couple of decades' to 'your job is at extreme risk for disruption from AI' in the space of like 5 years. Personally I have a family, I'm a bit old to retrain, but I never worked at a high-comp FAANG or anything so I can't just focus on painting unless my government helps me (note - not US/China). That's extremely anxiety-inducing, that a vague promise of novel new things does not come close to compensating.

I don't think the LLM was asked to check 10,000 files given these models' context windows. I suspect they went file by file too.

That's kind of the point - I think there's three scenarios here

a) this just the first time an LLM has done such a thorough minesweeping b) previous versions of Claude did not detect this bug (seems the least likely) c) Anthropic have done this several times, but the false positive rate was so high that they never checked it properly

Between a) and c) I don't have a high confidence either way to be honest.

> But the entire value is that it can be automated. If you try to automate a small model to look for vulnerabilities over 10,000 files, it's going to say there are 9,500 vulns. Or none.

'Or none' is ruled out since it found the same vulnerability - I agree that there is a question on precision on the smaller model, but barring further analysis it just feels like '9500' is pure vibes from yourself? Also (out of interest) did Anthropic post their false-positive rate?

The smaller model is clearly the more automatable one IMO if it has comparable precision, since it's just so much cheaper - you could even run it multiple times for consensus.

I think your numbers are off. TAM for office workers is ~20T a year, of which SWE compensation is ~3T. So if they can make 3T x 10% X 5 years = 1.5T that covers their current valuations. It's not as insane as you make out, even not taking into account the other high risk areas like legal, accounting etc

I think it's completely normal. Whenever automation comes knocking, people are inclined to think it's going to flatline conveniently before their job is at risk. LLMs can code now? Cool, they can't code well though can they? Oh they can code pretty well now? Cool, coding was never the hard part of SWE anyway, it's [thing we have no reason to think AI can't beat 99% of humans at at some point], etc

I think SWE as a mainstream profession is much nearer to the end than the beginning, I'm curious and quite scared about what becomes of us.