kypro

ARC-AGI-3 launched a few months ago which would suggest that prior models likely had no knowledge of ARC-AGI-3 or training on similar challenges.

I could be wrong, but given the large outsized jump solely in the ARC-AGI-3 score, it would suggest that the model didn't become significantly more intelligent overall, but significantly better at solving those specific problems.

This could mean one of two things (I think):

- Opus 5 was not benchmaxxed on ARC-AGI-3, but has benefited significantly from discussions about the various challenges and mechanisms deployed in ARC-AGI-3 such that it has far better heuristics to solve its challenges.

- Anthropic looking for buzz around their latest model picked a well regarded benchmark with significant room for improvement and focused some of Opus 5's training compute on ARC-AGI-3-style problems.

Or it could be some combination of both. Personally, given how much of an outlier the ARC-AGI-3 jump is I struggle to see it being the product of a significant improvement in general intelligence.

I think that's a strange way to look at it. The brain also has different regions with different functions, and part of what makes us humans intelligent is that we can use tools like pen and paper to keep notes and help us solve problems.

Similarly LLMs are not just massive uniform artificial neural nets, and now we also have harnesses, which I'd personally view more of an extension of the model itself. The harness is both the executive and also what allows it to keep notes, use a calculator, or maybe even create scripts to help it solve complex deterministic problems.

I think it's unfair to give a human a very complex maths problem and say that they're not intelligent if they can't solve it without pen and paper or a calculate. At least expecting a human to solve complex problems this way doesn't really measure anything useful in the real world.

> It will never be economically viable to replace kitchen staff with AI

Can you expand? Specifically what is it about humans that AI and robotics could not replace?

While everyone is building more, I've found the amount of software I build now has dropped off a cliff. I haven't produced this little code since I was a young teen 25 something years ago.

I used to love building software which required solving difficult algorithmic problems then presenting it in a nice UX.

All the things I used to enjoy doing has become trivial for modern LLMs to the point where it's a pointless pursuit. I used to like spending days doing intense research and learning, then weeks working on challenging algorithmic problems, then at the end I could wrap it all in a fun UI.

I guess I could still do the UI bit, but there's just so many other creative, difficult and rewarding stuff I could be doing with my time now...

I'll add to this...

The "funky" websites of the past were mostly a result of tech immaturity and a lack of profit motive.

Businesses have been able to easily install templates like this for at least a decade. They don't because stuff like this looks cool but isn't very functional.

AI isn't going to make your local restaurant have a funky website, it's just going to make everyone who use to work directly and indirectly for that company unemployable. And even the local restaurant will close down because they can't compete with the multi-national competitor that has automated their kitchen with AI.

In late 2026 the role of the developer will be more or less redundant in any traditional sense. There will just be builders. Most non-technical people now are quite capable of building software with the right tools. And they build in the same way as developers.

Yes, same here in the UK. I think imgur blocks VPN IPs so I suspect unless I register a private IP or something it's basically unusable.

I'm from the UK. What's in the image?

> Until they can get the confidently incorrect problem solved, I don’t see it being a viable replacement. Maybe if you have agents that are aggressively checking the main ones’ output, but that seems unnecessarily wasteful of resources to me.

2 years ago AI could even count the number of letters in a word or do basic maths. Now we're at the point where we can't trust it because makes occasional errors.

Don't get me wrong, I agree with you. These are things that need to be solved first, I just don't have reason to believe these are particularly difficult problems.

Anyway, I enjoyed the convo. Was interesting hearing your thoughts. I would give anything to be wrong.

We're so screwed man.

It was only a few years ago I was debating AI risk with people and they were saying, "but obviously we're not stupid enough to give it access to the internet!!"

And honestly, it wasn't always easy to argue with that. Like yeah, maybe we would take this stuff serious and run it on a completely isolated machine with no external IO or network access. Maybe my opinion of humanity is too low.

But it's hard for me to read this and believe anyone cared risk here beyond the most surface level concerns like adding some minor restrictions to the network. Not even I would have expected us to be this reckless.

> With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.

This simply should not be possible. Call me crazy, but I don't agree with giving a frontier AI model with unknown cyber capabilities access to a restricted network in the first place, but clearly this was an incredibly poorly designed sandbox.

If one of these models have a genuine step-level capability improvement and start to pursue their own goals, then who knows what might happen. I mean who knows, maybe it's already infected critical infrastructure. We have no idea what these labs are cooking up, where they're running these things, and neither us or them seem to have any clue what their capabilities are.

Every day that passes it becomes harder for me to understand how there are still people denying what's coming.

As always is the case, nothing will be learnt from this.

> However, I don’t see a viable path forward where that occurs, and we all still have jobs. Why would I be kept around if a superhuman AI can do my job 1000x better and faster, and at a drastically cheaper price? Or, if I was kept around as a human QA agent, why would I be paid nearly what I am now?

Well as an AI doomer I agree with this. In my view you will lose your job, along with everyone else, and we'll just need to hope that those in power will give us some cash to feed our families. Ultimately us regular folks won't have the collective bargaining power of our labour and could not revolt with nation-wide strikes, nor are we any physical threat for those with power anymore.

But that's the good outcome. This assumes AI remains obedient to it's human owners and that bad actors don't have access and can't use it to create biological weapons and other civilisation-level threats.

I know we're not there yet, but the history of AI is one of continuous step changes in capabilities with each algorithmic breakthrough. AIs today are far more knowledgable and much quicker than humans, but still missing a few bits that keep them from being broadly super human. One or two algorithmic breakthroughs is all it takes though. It was just two breakthroughs over the last 8 years that took us from not even having code-autocomplete to having reasoning models advanced enough to solve math problems which have stumped humans for decades.

> The problem is if you don’t understand the root cause, you’ll at best feed the LLM a list of symptoms

Today maybe.

When Cloudflare has an outage does the CEO need to know the root cause though? If he just assigns some engineers to investigate, fix and report back to him – what's the issue with that?

Assuming you agree with my analogy, can you explain what it is that you feel an agent or group of agents assigned to an engineering tasks could not do that a human can? And if can't do it today, why do you believe it wouldn't able to do it in the future?

Are we just disagreeing about timelines here or whether humans have some magical quality which algorithms and AI can never capture?

I feel like it's reasonable to disagree on timelines. It's quite unreasonable to assume human coders are special.

GPT-Live isn't a particularly intelligent model. It needs to handoff for reasoning and more complex tasks.

You shouldn't expect it's responses to be as reliable as a frontier model. It will sometimes say odd things.

It's way easier. People don't need to read code anymore, nor will they have to care about building mental models about the system as this article suggests.

The way to think about coding agents is that a good one should in theory literally replace the developer entirely. No, a whole organisation of developers.

In theory a product owner should just be able to dictate how they want to product to function at a high level and the AI should take care of the rest in the same way a human programmer or team of programmers would have in the past. If there are conflicts in what's being requested, then the AI should be able to recognise that and ask for production direction.

As always with every generation of AI people seem to way over index on the now.

- "They're good for autocomplete, that's about it"

- "They're good for quickly mocking up small functions, but they make a lot of mistakes"

- "They're good for scaffolding some parts of the system if you're good at prompt engineering"

- "They're good at writing most of the code, but humans will always need to do the last 10%"

- "They can write all of the code, but humans will still need to architect the system"

You are here. Perhaps this is where progress stops. I wouldn't bet on that however.

AI is making programming an irrelevent field.

As someone who lives in the UK 100%.

There are things I don't say online anymore, not because I don't think they're correct, but because my opinion could be deemed offensive.

Whenever I say something I think could risk being seen as offensive, I tend to add a lot of footnotes explaining my reasoning for the things I'm saying and why I'm not trying to offend.

I also never share remotely offensive humour to friends over social media or messaging apps anymore, which used to be fun. Lots of people have been arrested for jokes so just not worth it tbh. Even if I don't think it's that offensive the law is too subjective to take the risk.

I see a lot of these "don't use [insert product]" posts and it's normally always BS about how some exec at the company has the wrong political views or upset some community who was the product by taking it in a different direction from what they wanted.

I think this is probably first time ever I read one of these and was convinced immediately that I should probably look into using something else.

I don't think your wheel analogy holds. Wheels and rockets are fundamentally different technologies.

> LLM technology itself has nothing to do with general (human-like) intelligence.

Neural nets to the brain are like a plane to a bird. Saying LLMs have nothing to do with human-like intelligence is fundamentally ignorant of the field and the inspiration behind many of the algorithmic experiments.

LLMs clearly do things differently, but that doesn't mean they can't be generally intelligent, or they can't produce output that's remarkably similar to a human.

And to be clear, I have studied AI, published on AI and work in AI.

> Do you people get off by fantasizing about people losing their job, or worse? (whatever you are implying)

I'm the exact opposite. I've been terrified about AI for decades. I had nightmares last night as I do most nights about AI. I think humanity is years away from a disaster and I'm pleading for people to see.

But what I've learnt is that every step closer we get the goal posts just move a little more. I'll note just a few years ago HN was saying ChatGPT and LLMs were just stochastic parrots or advanced autocompletes. I pushed back against this and said within years they'll be raising PRs on their own from tickets. But everyone told me, "nooo, they couldn't even do basic maths!!"

Remember this comment when the time comes. We could have acted, but we didn't because every advancement people dismissed and seem to think that further improvements would not be made. You will be wrong.

I'm just going to be honest because lying won't help.

Realistically you will almost certainly fail because almost all startups fail.

There was a post on Indie Hackers many many years ago now where someone looked through all of the projects posted there over the years and found that something like 1% of them succeed (may have been less tbh – it was so long ago).

You are more likely to succeed if you have serious money to throw behind it or you are able to get investment, but if you're just a dude trying to start a businesses online with some savings you're more or less guaranteed to fail statically. This wasn't true of in tech in the 90s and 00s, but things have matured a lot since then. The fact you have users is actually impressive and imo you've already done better than the vast majority do.

I'm saying this because the first thing you should be doing if you're not already is assuming failure with an extremely high confidence. If you have no plan for failure then you need to start coming up with one now.

I've been through this myself. Startups were my life in my late teens and early 20s. I had a breakdown in my mid 20s because after some early success in my teens, I just couldn't get anything to work anymore. I was also isolating myself from friends and family and the realisation that I had basically lost some of the best years of my life behind a screen and text editor for nothing ate away at me.

Quitting is not failure. Here's what you need to do now.

- Come up with plan assuming your startup will fail.

- Decide the point at which you will stop.

- When that time comes execute your plan like a zombie through the pain.

Your emotions will tell to keep going one more day and your plan however good will almost certainly not be what you want to be doing because if it was you'd already be doing it. Just keep going and you'll come out of it in the end.

Good luck. Wishing you the best bro.

If you have a magic wealth creation box you don't need labours or people to sell to. You just ask the magic box for the resources and you have them.

The post-AGI world doesn't depend on people buying stuff.

Post AGI economy won't depend much on human demand.

As you say there will be no job to strike from. And pissing off the government probably isn't going to go down well when they're you're only hope at feeding your family.

Unfortunately in this world income taxes have dropped to 10% of their previous levels and the government is now primarily funded by AI. There are rumours some leaders are questioning why they're even redistributing wealth when they could just take it. Citizens are not needed anymore, they're just a burden and taking from those at the top.

> The good thing is that LLMs will not bring forth AGI.

Even if this were true you'd be silly not to see the larger trend.

LLMs and the transformer model were just a step function in AI capabilities upon the last generation of AI.

Whether it's LLMs or not is irrelevent. Major algorithmic improvements in AI have happened every 5-10 years over the last 50 or so years, and now they're happening maybe every 3-5.

The question you should be asking is what happens when the next step function in AI capabilities occurs?

Arguably people worrying about job loses are not worried enough.

If it's any consolation, I suspect people worrying about job loses and a loss of meaning will soon realise these are minor concerns.

What people always struggle to model with AI are the step functions that occur in capabilities as new algorithmic advancements are made. Historically those come maybe every ~5-10 years, although the pace has dramatically increase in the last decade and last 5 years.

Most people right now are plotting a linear improvement over the several years as models get bigger, harnesses become more complex and capable, more people adopt AI for efficiency, etc. Some are adding small exponents, but most are fundamentally modelling wrong.

What's more likely to happen is that in a few months someone at some AI lab will leak that they've made a breakthrough internally. Then another leak will come, and another. We'll know something big is going on because we'll see strange things happening within governments, then FOOM.

We're likely now one more step function away from something that's not just capable of taking your job as an intelligent and knowledgeable human, but surpasses your ability so significantly that you can't even comprehend what's happening around you.

This job concern is really just a concern perhaps of the next year, or maybe at a stretch, a few years.

It's not the purposelessness or job-loss that should have you concerned, but the pandoras box that we're opening. Once that box is opened every problem bottlenecked by intelligence is solvable practically over night, and therefore the way the society will look in a decade is almost impossible to imagine.

The problem of how to cure cancer is solved, but the problem of how to kill everyone with a super virus is too. The problem of how to cure alzheimers is solved, but so is the problem of how to hack the human neural activity is too.

I genuinely do not understand how people still don't see what's coming, but I try to remind myself that these are the good problems and we should enjoy this for as long as it lasts.

Business loves them, actual engineers hate them.

If you care about your career you should be one. If you care about engineering you might have other concerns.

I find mental burnout is largely a product of my ability to keep pace.

I feel burnt out in spoken conversations because I find it very difficult to process sentences at speed. But I don't feel that at all when reading, writing or listening to podcasts because I can go at my own pace.

Similarly I love maths and it doesn't give burnt out when I do it myself, but in school I remember it being agonising trying to get my brain to keep up with everyone around me.

I think this is why AI burns out engineers. Agents can do things do much faster than us so we're no longer comprehending things at our own pace, but trying to keep pace with the AI.

I feel that useless thing too. It hurts me more than anything. I've poured my heart into so many blog posts, tutorials and open source projects over the years. They were genuinely hard at the time. Each tutorial or code project might have been weeks of effort – and it showed. Today an AI could do the same in 10 mins from a single prompt. What was a source of pride for me, now just feels like a major waste of the last 2 decades of my life.

Maybe that's for the best though. I think in hindsight I over index on the value of knowledge. Maybe there are more important things in life.

Extremely weak bid, no?

PayPal was worth 5x this a few years ago... Even pre-pandemic it was worth far more. Hell, just a year ago it was valued at ~$80 share.

$60.50 per share would value them at just 11x their current earnings. I get PayPal isn't exactly a growth company anymore, but I'm not sure I understand why shareholders would agree to this offer?

Yeah, this is exactly the type of thing I'd support.

The UK (where I live) emits something like 0.5% of global CO2, but we're one of the largest economies in the world.

If we're going to do anything it seems we should really be doing everything we can to use our economic might to try to do something about that other 99.5%. Where as at the moment we're wasting our time obsessing about how we bring down that 0.5%, and providing an economic horror story for why no other country should replicate what we're doing.

That's a fair push back.

For a specific technology to become viable, you often need progress in several individual technologies to occur. Some of these will be exponential and surprise us, but many will see slower progress.

Either way, I suspect progress in robotics will be slower than AI, but I also think we'll see a lot of investment in robotics over the next decade given that AI opens up a lot of new potential applications for robotics.

I suspect any task that doesn't require extreme battery life or extreme dexterity will be doable by robots much sooner than most people think today. I think we could have decent-ish humanoid robots within a decade – robots that can prepare food, clean hotel rooms, empty bins, etc. I think Elon's bet on Optimus is quite likely to be viewed as very ahead of the curve with time.

However, it will take much longer for robots to do everything a human body can, but then most jobs don't require we push the physical limits of human body.

That's why I think 60% is reasonable. And even then I'm not saying it will happen, I just don't think that's a bad bet if given even odds.

Unpopular opinion, but we need to start using hard and soft power if we want to fix this problem.

Europe cannot fix climate change on its own, and we already represent an increasingly small share of total emissions.

The question now is how we get other countries onboard fast, and if they continue to build out cheap coal-fired power stations what are we going to do about it? Something or nothing?

Unless we can get every nation onboard, we should switch strategies to one of managing the inevitable rather than hopeless trying to prevent it. Costly energy policies that reduce global emissions by roughly 0.5% (basically what will happen if the UK ever achieves net-zero) are a total waste of money and time.

Those who care about this issue need to get real and need to stop pretending that the solution is for a few more people in the UK putting solar panels on their roofs. Climate change is a global problem. If your solutions are domestic then they're not serious.

UK is the worse for this too.

We have a story in the news currently about a foreign child gang rape leader (yes, those are things in the UK) who raped at least 30 children and was released early after serving just 14 years in prison.

We have major children sexual abuse scandals in this country basically every year too. Politicians never care about that though.

There's always been an arms race in anti-bot technology and more sophisticated bots.

I'm sure, they can add a jitter, but then you just change how you detect / weight detection.