Hacker News Reader

Comment by patcon | Hacker News Reader

I'm not sure this is even conceivable, but... Imagine the bootstrap problem of intelligence systems going down, when no one knows how to bring them back up without the system itself walking them through it... Like if power and intelligence were to go down in a Carrington-class event in 10 years...

Like what would happen to individuals ten years from now who are trying to live if AI suddenly became inaccessible

Of course we all know most humans are useless and unable to survive outside our culture, but the difference is that I'm trying to picture is the individual trying the reason through challenges. The version of this that has been obvious to me up to now is that the individual didn't have the knowledge to look after themselves. But I guess it feels different to conceive of them not having the ability to fully reason on their own...?

To not just lack the objects of intelligence, but the very process of it...

View on HN

The general drove on. "At the present time our chief bottleneck is the fact that missiles are limited in intelligence. The computer controlling them can only be so large, and for that reason they can meet the changing nature of anti-missile defenses in an unsatisfactory way. Few missiles, if any, accomplish their goal, and missile warfare is coming to a dead end, for the enemy, fortunately, as well as for ourselves.

"On the other hand, a missile with a man or two within, controlling flight by graphitics, would be lighter, more mobile, more intelligent. It would give us a lead that might well mean the margin of victory. Besides which, gentlemen, the exigencies of war compel us to remember one thing. A man is much more dispensable than a computer. Manned missiles could be launched in numbers and under circumstances that no good general would care to undertake as far as computer-directed missiles are concerned . . ."

I'm not sure what this quote is from, but I almost down voted you in disgust before realizing that was exactly the intended effect. In other words, "Thanks, i hate it"

asimov, "the feeling of power"

Thank you!

Asimov rocks! You should look into "Profession" and "It's Such a Beautiful Day", if you haven't.

We don’t even need to imagine that world, Russia is doing it today.

Missiles and ammo are too expensive, so they send people to run at the enemy, revealing their firing positions. Then they drone the people who revealed themselves.

If I was an AI, I would reason that a T-800 is far more valuable than a man soldier who I must feed and clothe and heat. Use the disposable soldiers first and save the robots for later.

I was unfamiliar with the short story “The feeling of power” but thought for sure that “graphitics” must’ve been an Asimovism

>A man is much more dispensable than a computer

sloppy (that is, negligent) writing. Something is either (in)dispensable, or is not. Either use something that can take degrees of comparison, like valuable (A man is less valuable etc) or reword it: A computer is indispensable, a man is not.

Wow you and Justice Scalia should have met!

> “There is no such adjective,” Scalia said. “I know we have used it, but there is no such adjective as choate. There is inchoate, but the opposite of inchoate is not choate.”

https://www.abajournal.com/news/article/justice_scalia_deliv...

Take it up with Isaac Asimov, I guess.

No - the computer is not indispensable. The computer is just harder to replace than the man (in this imagined situation).

Hence - the man is much more dispensable...

---

The water in my water bottle is much more dispensable than the toothpaste in my toothpaste tube. But both can be dispensed (in both meanings - distributed or replaced).

Some of us are here to help. $1000/hr.

A few of us old boys can still maintain Cobol

Right behind you with fortran.

Exactly what I was thinking.

Sounds like a bad novel. In the year 2050 no one knows how to actually program anymore. Then ClaudeGPT by GoogleSoft goes down. A 74 year old former software developer is on his farm tending to hens.

A self driving car crashes into his barn.

Out comes out a tech bro, “Yo OG programmer bro, it’s all busted bro, could you like make it unbusted bro.”

Then again with social security set to go bankrupt it’s not like we’ll be able to retire

Thinking more a mashup of The Matrix and Idiocracy with Doc Brown as the Hero retired Programmer. Or Rick.

Hero ?

What if the twist is that he’s writing spaghetti code that only he knows how to maintain.

Worse, it’s in a language that his llc charges licensing fees to use!

> Then again with social security set to go bankrupt it’s not like we’ll be able to retire

I don't know about you, but my notion that my social security check is all I will need to survive my retirement years died when my father started drafting his own social security in the 2010s, and it was literal peanuts after being a small business owner his entire life (restaurants) and paying more than his fair share in taxes. Do the average folk still have this notion that social security = retirement? Hopefully not, it's certainly misguided now, but it was misguided then, too.

Social Security is more than enough in Vietnam or another low cost of living country.

The issue then is getting a visa.

Social security has a low maximum, if your dad was used to clearing 15k a month and then social security only paid him 3500$, that’s rough.

For the typical person only making around 4500$ , it’s an adjustment, but doable

This is the type of movie I would be so excited about until I found out it was made by Seth Rogan.

How about Adam McKay, then?

If they turn it into a movie, they would send the greybeard to space to fix the AI satellite server in space to save humanity.

...where he decides to de-orbit the thing, thereby saving humanity.

> Then again with social security set to go bankrupt it’s not like we’ll be able to retire

That is why you should use current money to become less dependent on future money. That former developer on his farm had the right idea, he bought a farm. So did I, with many hectares of forest which I use to build, heat and can also use to hunt should that become necessary.

Joke's on you, after the AI economic collapse in a few years $1000 will have the purchasing power of 50 cents of today's money.

One meeeelion dollars then.

You've just given them a 50% discount

Talk about a #MeToo movement :)

Apologies. The downvotes remind me it had turned into a religion.-

A safer currency and rate would be $1MWh/hr

I think the reason that intelligence is so scary is how contextual, stateful, path-dependent, and coupled it is.

Which doesn't seem like it. The ideal is to keep removing couplings. But in practice, I find that focusing on the statefulness, the distribution-specificness, these "accidentals", is how you actually get leverage.

The obvious social consequence is now the tendency towards centralization, tyranny, "you people can't think for yourselves, not because you have some kind of 'inherent intelligence defect', but because the world is so complicated and interconnected that only coherent systems thinking (conveniently administered from one central point) is what can save you". In the Enlightenment world, the ideal was that every man could in principle, think for themselves.

A good ending might be that everybody benefits each other by distributing intelligence, but in practice centralization always seems to win.

This is why it will be more important than ever to have experienced, quality employees.

I dare companies to gamble on running their businesses on absolute skeleton crews long term

yeah this was the devops promise too. Enterprise evolved into dedicated teams to manage AWS and other infra, instead of reducing headcount it raised it. so it goes.

> Enterprise evolved into dedicated teams to manage AWS and other infra, instead of reducing headcount it raised it. so it goes.

I wager that's mostly due to the Jevons paradox, though; it's so much cheaper and easier to build out systems with cloud than on-prem that you end up building way more systems than you would have without it, resulting in an overall headcount increase.

Keep an offline copy of Qwen or something together with a knowledge base. Easy if you plan ahead.

Yeah seems unlikely we'll be without at least a baseline level of AI unless we entirely run out of energy and components

My fear was that people like Linus would die and then nobody would be able to build operating systems anymore, because zoomers and below can’t even use computers.

Now we know they won’t even be able to read and write. But maybe Claude sonnet 12817 can maintain the operating system. It’ll be 17,000 dependencies in and mostly written in JavaScript, but it’ll work.

Who needs Sonnet 12817 when 5 is maintaining is already maintaining Omarchy? And it's only 5% JS!

Hardly a unique problem to AI systems. You certainly can't bootstrap pretty much any industry to its current state without the existing items of its current state. Any sort of chip fab is a great example. Or most manufacturing. Or even more fundamental: mining (both metals like iron, and energy like coal or gas).

You go way too far - the leap you have made from some sort of AGI to "now individuals don't have the ability to fully reason on their own" is not clear to me at all.

If people are spending less time bent over monitors or at desks under the dictates of corporate imperative, they will probably be spending more time doing actually interesting things. Reasoning is innately human. What we call "knowledge work" is not and is often anything but. People who are naturally curious will still discover things, even if irrelevant (from a market perspective) when the AI is on, for their own satisfaction.

If such an event were to occur, they would be called upon. It's really little different from what would happen today if suddenly the power grid or supply lines were down for a sustained period of time.

The hordes of acolytes mumbling “Jevon’s paradox, Jevon’s paradox” every time AI gets cheaper beg to differ. Every minute saved using AI is just an opportunity to use more AI

It may sadden you to find that the majority of the population can go days, even weeks without ever seriously "reasoning" about anything major.

On the other hand, I get the sense that the misanthropes proclaiming these supposed facts are quite happy and self-satisfied about it.

> live if AI suddenly became inaccessible

some weeks ago, we changed the billed credit card - something went wrong on their end we were unable to use it for ~8-9 days.

Some pepople during this period said to certain tasks: "I will care about that if Claude is available again"

So, we are already there

Like navigating to the other side of the city without Maps(Google)?

People will have more things to worry about sooner. Where I live, if the power goes out so does my water. If AI is involved in either at any time in the future we will deserve what happens to us.

Also I disagree with the permise. People learned how to make technology that never existed once already, they can learn to do it again if needed. It will take time, which we'll have plenty of because your phone won't be working.

There are many people who have the weights of open source LLMs locally and just need 120v

I really hope there will be social change - flexing mind - like flexing body in a gym/sport - becomes a thing.

There always was. Like there's a gym crowd, there's an intellectual-hobby crowd.

Not even close to being as popular as sports or a gym stuff. There's basically common knowledge - to be a good <think profession> you need to hit a gym regularly. And it's going to be gym+something for a brain soon. At least i hope so

It's a real problem but your framing of it is itself very alarming.

Most humans aren't useless by any measure of most or usefulness. Our culture is our tools, dying due to a lack of antibiotics doesn't mean you didn't have anything to contribute, it means you didn't have access to the tool you needed. This is the problem we need to guard against.

No one is going to lose the ability to reason! This is accepting the most extreme version of the marketing presented by frontier labs, don't do it. People may lose fluency or familiarity or access to training on specific skills. We should put some care into making sure that can't be a catastrophe, for sure.

You've obviously never stood in a room of college freshmen and tried to deal with a young man having repeated issues being able to type his _own_ name into a box on a screen.

One kid is spectacularly stupid, therefore most of humanity is useless?

This is basically the premise of Bacigalupi's "Pump Six", from "Pump Six and Other Stories".

The titular Pump Six breaks down and basically there's nobody left in the world who knows how to even reason about the steps needed to fix it. There's just some log dump and nobody understands what it means. Humans are no longer capable of complex reasoning, they just live in the moment and hump.

Your dependence on AI is my margin of opportunity...

https://en.wikipedia.org/wiki/The_Machine_Stops

https://www.goodreads.com/en/book/show/2819368-pump-six-and-...

Both Pump 6 and The Machine Stops have examined precisely this in detail.

"The Machine Stops"

for those not aware of the reference, "The Machine Stops" was a 1909 novelette by E. M. Forster. It was very forward thinking, and so full of ideas!

On project gutenberg: first story in this anthology, https://www.gutenberg.org/ebooks/72890

PDF of just the story: https://www.cs.ucdavis.edu/~koehl/Teaching/ECS188/PDF_files/...

We hope an average guy who was stuck in an experimental cryopod gets accidentally woken up and through a course of events gets appointed to the presidential cabinet long enough to fix society.

> Of course we all know most humans are useless and unable to survive outside our culture, but the difference is that I'm trying to picture is the individual trying the reason through challenges. The version of this that has been obvious to me up to now is that the individual didn't have the knowledge to look after themselves. But I guess it feels different to conceive of them not having the ability to fully reason on their own...?

What this misanthropic/Anthropic-boosting bait is missing is that all of these pieces falling together doesn’t make sense.

1. We are so accustomed to not-even-thinking means that we are legitimately freed from labor and cared for. (The dystopic version of being freed from your labor means that you still have to constantly think to survive. But not in any “economically useful” ways. Surviving for yourself and your family outside of the robot economy)

2. At the same time as point 1, we voluntarily, with all of our leisure time, just don’t think at all. Where is the self-actualization? Where is the spontainety? The drive? Did this in all of these years not come back after being a rat racer/wage slave?

Curiously, the “hacker ethos” here is to create. Not to just “consume”. And that is taken as an intrinsic thing, not just a way to get rich. But that’s not how “people” are?

Other people are mostly like you. But maybe those here who spent their careers implementing Entertaining Ourselves To Death maws genuinely think that “people” (not them though) would genuinely, when left to their own devices and having beeng catered to every need, would just “consume” mindlessly onto death.

Sorry to Goomba anyone here.

> Of course we all know most humans are useless and unable to survive outside our culture,

All. Take away the “habitat” from an animal and they are helpless, useless, impotent.

There is a TV show called "Alone" where people have to live in a remote area using basic survival skills and very minimal tools. It's interesting, but I've always wanted to see a version of that show where small communities are competing against each other.

If we practiced for these Armageddon scenarios what would a highly prepared group of 20 be able to accomplish in a year. How fast could they get humanity back to the bronze age?

Reply on news.ycombinator.com