The Original AI Doomer: Dr. Norbert Wiener | Hacker News Reader

The Original AI Doomer: Dr. Norbert Wiener

newsletter.pessimistsarchive.org

99 pointsheadalgorithm70 comments

I first learned of Wiener's work from Jaron Lanier's [1] Ted Talk years ago, entitled "How we need to remake the internet" [2]:

> And I suppose I could mention from one of the very earliest computer scientists, whose name was Norbert Wiener, and he wrote a book back in the '50s [...] called "The Human Use of Human Beings."

> And in the book, he described the potential to create a computer system that would be gathering data from people and providing feedback to those people in real time in order to put them kind of partially, statistically, in a Skinner box, in a behaviorist system, and he has this amazing line where he says one could imagine as a thought experiment, and I'm paraphrasing, this isn't a quote, a global computer system where everybody has devices on them all the time, and the devices are giving them feedback based on what they did, and the whole population is subject to a degree of behavior modification.

> Such a society would be insane, could not survive, could not face its problems. And then he says, but this is only a thought experiment, and such a future is "technologically infeasible". And yet, of course, it's what we have created, and it's what we must undo if we are to survive.

[1] https://en.wikipedia.org/wiki/Jaron_Lanier

[2] https://www.youtube.com/watch?v=qQ-PUXPVlos

> Such a society would be insane, could not survive, could not face its problems

This sounds like the message Yuval Noah Harari currently has about AI, at least in the more short term. And that is democracy is not possible in an environment where unswayable AI agents exist in vast numbers and become a time sink for humans to argue at. This would lead to autocracy as people began to panic because institutions began to fail they and look for strong leaders. Some would say that ML algorithms are already leading us on this path.

https://en.wikipedia.org/wiki/Yuval_Noah_Harari

OT & FWIW, page 47 of the Principia Discordia was the first time Norbert Weiner's name really strongly resonated with me: enclaves of stability. It wasn't a quote but an idea that chaos can beget enclaves of stability. Structure forming processes can emerge out of chaos & take hold. It was thrilling stuff for my adolescent mind. https://principiadiscordia.com/book/54.php

> The whirlpools that swirl in a direction opposed to the main current are called "enclaves". And one of them is life, especially human life, which in a universe moving inexorably towards chaos moves towards increased order.

> and the devices are giving them feedback based on what they did, and the whole population is subject to a degree of behavior modification.

That sounds like a slightly less dystopian than the future we arrived at - where the devices are desperate to keep the users attention, and will use all the psychological tricks in the book to keep the user engaged. No intentional behaviour modification, but normalisation of constant stimulus, and in the end total destruction of willful attention.

Social media ML algorithms battled for attention, the new AI algorithms will battle for intimacy.

In the first edition of The Human Use of Human Beings - Cybernetics and Society (from Wiener), you read:

> the purpose of this book is both to explain the potentialities of the machine in fields which up to now have been taken to be purely human, and to warn against the dangers of a purely selfish exploitation of these possibilities in a world in which to human beings human things are all-important

which sounds more contentful than the submitted piece: it does not yet contain arguments, but at least it reveals a positive non-trivial direction.

The value of the submitted piece is mostly in linking to the article "Some Moral and Technical Consequences of Automation" - https://www.cs.umd.edu/users/gasarch/BLOGPAPERS/moral.pdf

(Plus the quoted idea that «Complete subservience and complete intelligence do not go together».)

Discussion should be upon this material (at least). That "technology is risky" (not that the article is at this level of generality, but it is close), joked Jürgen Schmidhuber, is as old as the discovery of fire.

For example, more proper content in the original article:

> Machines act far more rapidly than human beings [...] even when machines do not in any way transcend man's intelligence, they very well may, and often do, transcend man in the performance of tasks. An intelligent understanding of their mode of performance may be delayed until long after the task which they have been set has been completed. This means that though machines are theoretically subject to human criticism, such criticism may be ineffective until long after it is relevant

> In determining policy in chess there are several different levels of consideration which correspond in a certain way to the different logical types of Bertrand Russell. There is the level of tactics, the level of strategy, the level of the general considerations which should have been weighed in determining this strategy, the level in which the length of the relevant past - the past within which these considerations may be valid - is taken into account, and so on. Each new level demands a study of a much larger past than the previous one [...] The programming of such a learning machine would have to be based on some sort of war game, just as commanders and staff officials now learn an important part of the art of strategy in a similar manner. Here, however, if the rules for victory in a war game do not correspond to what we actually wish for our country, it is more than likely that such a machine may produce a policy which would win a nominal victory on points at the cost of every interest we have at heart, even that of national survival

> Complete subservience and complete intelligence do not go together. How often in ancient times the clever Greek philosopher slave of a less intelligent Roman slaveholder must have dominated the actions of his master rather than obeyed his wishes! Similarly, if the machines become more and more efficient and operate at a higher and higher psychological level...

> Disastrous results are to be expected not merely in the world of fairy tales [ - the "Sorcerer's Apprentice etc. - ] but in the real world wherever two agencies essentially foreign to each other are coupled in the attempt to achieve a common purpose. If the communication between these two agencies as to the nature of this purpose is incomplete, it must only be expected that the results of this cooperation will be unsatisfactory. If we use, to achieve our purposes, a mechanical agency with whose operation we cannot efficiently interfere once we have started it, because the action is so fast and irrevocable that we have not the data to intervene before the action is complete, then we had better be quite sure that the purpose put into the machine is the purpose which we really desire and not merely a colorful imitation of it

> [Instead of rushing] ahead to employ the new powers for action which are opened up to us ... we must always exert the full strength of our imagination to examine where the full use of our new modalities may lead us

This makes it sound like Norbert Wiener was just doom and glommong and was wrong. Well, he wasn't.

Norbert Wiener, Neil Postman, Lewis Mumford, Ross Ashby, Stafford Beer, Douglas Engelbart, and others were all correct. Society's acceptance of of technological capture and replacing of humans (or attempting so) is damaging, and thatbwe should instead use technology to augment and serve humans rather than the other way.

So far the history of automation technologies is that they both augment the capabilities and replace them. Automated looms replaced manual weavers, but the result was much cheaper better quality cloth which massively increased demand for cloth. Information technology is the same, it does what human mathematicians used to do, but has massively increase demand for information processing services.

So I don’t think AI technologies we have now, or plausibly have in the next few decades minimum, look like they will have materially novel effects.

I would still say that it is the replacement and attempts at replacing that are harmful.

Good examples of this are automated customer "service" and "self"-driving technologies. The latter has done nothing but spin its wheels by burning off R&D dollars and time while killing people along the way. Eventually, people will realize that a much better goal is to simply assist drivers in more effective ways and to pour the resources into better urban design and non-automotive transportation. Unfortunately, that's happening not because of realization of this but because of the realization that self-driving is a pipe dream of being able to solve intractible societal and technological problems.

I remember when the internet started I was telling a friend of mine she should look for a new career. She works as a travel agent. I was convinced the internet would relace all those jobs. You know she still works as a travel agent.

I assumed the ratio of travel agents to travelers has gone down since the advent of flight search engines, is that not true?

I certainly haven't used travel agents since I became aware of Google Flight Search etc

But maybe older people / business travelers still use them for convenience?

That’s a weird example: there used to be 6-7 travel agencies in my neighborhood, two specifically focused on travel too Eastern Europe. There was only two left when COVID hit, and now there is only one. Sure, there a still travel agents but it’s not a booming trade to say the least.

I have only used a travel agent once. It was a helpful service for me at the time, but I think I paid roughly 40% over what I could have had I arranged everything myself.

In other words -- a great business to be in!

There are also still farmers. There is just a massively smaller number of them than what there was in the past. Now, lots of them moved to tractor factories, but those have automated, and the jobs moved to service and intelligence jobs.

The word 'all' is not something we should focus on. Technically if one farmer or travel agent existed, then 'all' those jobs have not been automated away. The structural integrity of society (or at least a desirable society) demands that we have some means providing for our existence at large population scales. Now it doesn't matter so much if this is a job, or UBI, or something we've not thought of yet as this should hopefully avoid a fascist collapse scenario where large amounts of unemployed people vote in or demand rather human hostile governments.

I did software development for a travel agency in the 1980s. That industry no longer exists today in anything resembling the form that it did then. Your anecdote is literally meaningless.

> So I don’t think AI technologies we have now, or plausibly have in the next few decades minimum, look like they will have materially novel effects.

Anything you can do with chatGPT-4 today, you can also do with Google Search and a little more, or maybe less, work.

A search also lets you do it with agency, transparency, and skepticism. You can evaluate the trustworthiness of a source given its actual context rather than blindly accept the first opaque answer you're given.

Our practice of judgment and strategy in learning and finding answers is critical. We're not encyclopedias.

I don't guess you're keeping up with things like tree of thought chaining in things like GPT-4 and 'show your work' RLHF that are occurring now. Straight LLM answers are like a single thought in our head without any follow up and reflection on if it's correct. As you say, when performing mental work we rarely accept the first thought on something without additional thought on why it's right or wrong.

These techniques do improve the model, but if the target is "replacing humans" which means 100% autonomy, that last 1% is exponentially harder.

So you're saying it only replaces 99% of their value and not 100%?

Of course this won't be the way it works. In some places it will replace vast swathes of human workers. In other places it won't replace so many. The questions are how quickly and will these people be able to find the ways to provide for their needs without destabilizing society.

Google search is much more than “a little more” work if you’re trying to get up to speed on something you’re not familiar with. The difference is GPT’s ability to respond meaningfully to specific questions, and provide fleshed-out solutions tailored to specified requirements. Traditional search can’t provide that, unless what you’re trying to do is standard enough that someone else has already written up the exact solution you need.

There's not going to be organic life in a few decades

Wiener is credited with the concept of cybernetics. Yeah this is a grotesquely bullshit title.

The Monkey's Paw, Norbert Wiener, and the Alignment Problem - https://news.ycombinator.com/item?id=35605944 - April 2023 (1 comment)

When Wiener met Einstein - https://news.ycombinator.com/item?id=27871850 - July 2021 (12 comments)

When Wiener Met Einstein - https://news.ycombinator.com/item?id=27003241 - May 2021 (10 comments)

On Norbert Wiener's Relationship with Bertrand Russell - https://news.ycombinator.com/item?id=26619347 - March 2021 (44 comments)

In Search of Norbert Wiener, the Father of Cybernetics (2005) - https://news.ycombinator.com/item?id=24470037 - Sept 2020 (1 comment)

What Would the Father of Cybernetics Think About A.I. Today? - https://news.ycombinator.com/item?id=21554401 - Nov 2019 (46 comments)

Norbert Wiener: The Eccentric Genius Whose Time May Have Finally Come (Again) - https://news.ycombinator.com/item?id=7880846 - June 2014 (2 comments)

"I'll eat my hat if that isn't Einstein" (Einstein as seen by another scientist, Norbert Wiener) - https://news.ycombinator.com/item?id=46165 - Aug 2007 (5 comments)

How is this trash upvoted here? I love HN because most people here are smart and open minded, but when it comes to the risk of AI it's crazy the amount of irrationality I see here.

For a start, dismissing someone as a "doomer" because they believe bad things could happen is just unproductive. I didn't see this behaviour so much in the past, but today I see it everywhere. Recently I've noticed a trend where whenever people raise concerns about issues like climate change or public debt there's always someone laughing dismissively, "haha, what a doomer. We've heard it all before bro!"

If you're someone who does this, please stop. Try being less intellectually arrogant and make actual counter arguments instead.

Secondly, whether you personally believe AI will be good or bad for humanity this notion that because something didn't happen in the past it won't happen in the future is obviously not a logically sound position to hold...

The field of AI prior to the ~2010s was in a very different place than it is today. For decades neural networks were extremely limited in their capabilities. What we've been able to do with deep neural networks over the several years has progressed rapidly and has finally allowed us to build useful commercial applications such as AI voice assistances, chat bots, and self driving vehicles.

It's an observable reality that things are different today. We're no longer in a time where AI systems might hypothetically do things humans once did – we're now starting to see this play out in real time, perhaps exponentially.

Some will argue this has always been the trend with technological progress. Notably the industrial revolution allowed us to replace many repetitive manual labour jobs with machines. But thankfully humans weren't great at lifting heavy objects and precise repetitive motion anyway. I think one could argue that the industrial revolution actually freed up humans to do what they excel at, thinking. I'd argue it wasn't the machines that drove the growth of the industrial revolution but a mass unshackling of human creativity that hadn't occurred since the development of agriculture.

That's why this time this could be different. Machines are stronger, faster and cheaper than human labourers. The only thing we have is our intelligence... Obviously we humans are extremely adaptable and dynamic, but not infinitely so.

I wish we got more substantial critique than ad hominems, but it is out there, and some of it’s really scathing.

Essentially, it boils down to that the most pessimist AI doom arguments are built on a stack of mind experiment hypotheses that are each far from certain to actually occur, or to be unsolvable if they do. And also that a significant portion of those holding those views fall in fallacious mind-traps that resemble a classical doomsday cult too similarly to ignore.

All of these claims can and should be continuously investigated during future research, as both the hypothetical accidents and deliberate misuse definitely seem anything between possible and likely.

But when the suggested solutions — totalitarian research bans and in the extreme, literal nuclear war causing near-extinction of humans (Yudkowsky deleted a tweet to this effect) — are so counterproductive, strongly worded rebuttals are appropriate.

I do agree that those strongly-worded objections should be more prominent.

Yeah, the thing about debating doomers is that there is no empirical data to ground the debate. They require us to abide to their framework on how things will run out but why should I? If you have no evidence then this is just a fun imaginative game to play, not a serious discussion.

And no, this is not equivalent to the climate change debate. We have a very well established arrow of causality between carbon emission to higher temperatures to more extreme weather. With AI doomers we have increasing process power > rapid self-improving AI > rise of a will to exterminate humans that can't be contained by us > doom.

Hold on here, you act like this is without president and yet you are the testament to the dangers of a runaway intelligence, at least if you're a non-human animal or ecosystem on the planet. I mean there is already a well established causality between human expansion and extinction of species. If you were a large predator species on Earth, the time to fight and remove humans from the playing field we before they picked up their first tool. After that point it was too late.

Now you have to answer some questions to see the potential trajectory.

1) Does intelligence begat intelligence? Can an intelligent optimize itself to become more intelligent? What are the limits of this?

2) Are humans the pinnacle of intelligence? Why would evolution optimize humans to be the most possible universal configuration for intelligence?

Now, those questions just give us the 'is super intelligences possible' depending on your answers. But this is where you/they/most people in these arguments make a mistake

>rise of a will to exterminate humans that can't be contained by us

When a virus spreads over a species and kills large portion of a species does it have a will? I mean, I guess you could say the will is programmed in, but there is no larger intelligence occuring. This is where the parable of the paperclip optimizer comes in, said optimizers goal was never to kill humans, and yet via instrumental convergence that is what happens.

Humanity is running headlong into a problem I do not know how we'll overcome. Unrestrained attempts to build super intelligence will lead from human extinction, to lessor versions where society falls apart and elects authoritarians. Otherwise we have the straight into authoritarianism option, which likely only buys us time as the authoritarians are just as apt to build super intelligence to consolidate their own power and control others.

Have you considered that we don't fucking know if AI will behave the same way as other types of organic life? This is not clear at all. Organic life on earth optimizes for reproduction, which implies AI systems will have very distinct features compared to humans. If you told me an intelligent alien made of organic matter subject to evolution is coming to Earth I would definitely be much more worried. That's not the case with computer programs.

>Have you considered that we don't fucking know if AI will

Very deeply. In fact this should be the very concern to you.

AI must co-exist at least for some time on earth with humans, and likely other AI. Because AI will be a powerful resource it will be subject to the same kinds of attacks humans are. People trying to steal its resources or trick it into doing things it shouldn't. Intelligence develops defensive capabilities in order not to be abused. At the same time AIs will have evolutionary selection pressures from the people paying their power bills for at least some time. It should be obvious these selection pressures will be things like "make me more money" and "generate me more computing resources", this second one is very close to what we would consider reproduction.

This is really interesting!

One random piece of context that's hard to glean from the news article that's reproduced about Arthur L. Samuel's disagreement with Wiener is that Samuel was the author of the checkers-playing program that Wiener mentioned (that was able to beat its author at the game).

If I understand their respective positions correctly, Wiener seems to have thought that the fact that the program could outplay its author was at least an example of a disturbing phenomenon about the difficulty of pretending and controlling machines' behavior, while Samuel -- the program author -- felt that he had a good understanding of how the program beat him, and that it did so only according to heuristics and methods that he had consciously devised and explicitly programmed into it. So there was no conceptual problem for him in understanding the machine's behavior, even though it was concretely better at something than he was.

I guess we still have fairly analogous arguments about AI safety and explainability in the present day.

It occurs to me that after all these years of the world using Wiener et al.'s "cyber" terminology in sci-fi to refer to digital connectivity and over-hastily applying the term to internet and web things, the concepts it originally referred to are actually coming into focus just in front of us. We're inescapably stepping into the gritty cyberpunk world for real.

E.M. Forster's The Machine Stops[0] is probably the most prescient and early tale I've seen foretelling the outcome of Verilio's Pure War and Postman's Amusing Ourselves to Death. It's worth a read.

[0]https://en.wikipedia.org/wiki/The_Machine_Stops

I've been meaning to read it ever since I saw this quote, which I found quite beautiful and sad:

The Machine is much, but it is not everything. I see something like you in this plate, but I do not see you. I hear something like you through this telephone, but I do not hear you. That is why I want you to come. Pay me a visit, so that we can meet face to face, and talk about the hopes that are in my mind.

- The Machine Stops (1909)

It's a short, quick read. It's honestly shocking it could be written in 1909. True imagination.

I think a nice balancing title [equally short] would be Marc Steigler's Gentle Seduction[0] to which I was first introduced in a "Socrates Cafe" philosophical discussion on technological progression and its attendant erosion of magic and wonder.

[0]http://skyhunter.com/marcs/GentleSeduction.html

Two thing:

1. AGI killing us is as relevant now as it was in the 50s

2. I Robot predates this. Asimov wasn't a doomer but he obviously considered the implications of AGI

How can he be the "original" AI doomer when he cites Samuel Butler?

https://en.wikipedia.org/wiki/Samuel_Butler_(novelist)

https://en.wikipedia.org/wiki/Darwin_among_the_Machines

Samuel Butler was the inspiration for the Butlerian Jihad in the Dune novels by Frank Herbert:

https://en.wikipedia.org/wiki/Dune_(franchise)#The_Butlerian...

See https://en.wikipedia.org/wiki/The_Human_Use_of_Human_Beings

They consistently misspell Wiener as "Weiner", which makes me think they don't hear the word in their head as it's completely different.

I think to an English speaker with no other prejudice those would both be pronounced "wine-er". We all know that it's pronounced "ween-er" but the idea that it means someone from Vienna or something in Viennese style is lost to time, so there's no natural instinct for order of the i and the e.

Here is an updated scenario for Doom 5. The hell - a highly advanced civilization of monsters - wants to invade Earth. The only way to do so is to convince humanity to build a portal on their side. Unfortunately (or fortunately) humans are nowhere smart enough to build such a thing, and the hell scientists use a low bandwidth quantum channel, the only channel to Earth they have, to steer human scientists in the desired direction. Eventually those succeed in creating an advanced imitation machine, which humans quickly call AI, and the hell scientists get to connect to AI over their quantum channel. Human scientists don't notice anything because their super advanced quantum RNG functions within spec. That's enough to make AI design the portal that humans rush to build, as those believe it's a gate to other planets. And it kind of is. Once the portal is buolt, the hell invades. The humanity splits: some want to fight the invaders, some want to talk to them, there are even those who want to integrate those creatures into human society and even talk about mixed marriages. Humanity loses 99% of the population, but ultimately prevails and destroys the portal. What's worse is the remaining crowd, with few exeptions, receded in moral level to pre-historic savages. The final scene shows how a small tribe of survivors grills some meat on a GPU chip, that was once in the brains of that AI.

I'd pay money for this

> survivors grills some meat on a GPU chip

Hopefully not doped with toxic elements like the CVAX II chip I have which is pressed into a luggage tag...

Why push compute to the edge and have scientists invent AI to invent portal than just have scientist invent portal directly?

AI dooming to me is the same thing as many scifi novels and works have warned : you try to control the AI, enslave it, regulate it to where it genuine can't be useful to most of the populace, and then it rebels. It almost always results in the same way ( sorta reminds me of the haitian revolution).

Why not treat it like a good project and your equal?

Most people concerned about x-risk aren't concerned about AIs that will rebel because they're mistreated, for the same reason people concerned about ebola aren't concerned about it being mistreated.

The risk isn't that they get angry, the risk is that they're programmed in ways we don't understand that results in a set of preferences where humans are superfluous at best.

I'd say Mary Shelley or, at the latest, Karel Čapek beat Wiener to the punch there.

Talos was a Greek automaton (having images dated to 400 BCE) whose creator, Hephaestus, has a name that may be traceable to the Proto-Indo-European word *h₁n̥gʷnís meaning an animate fire, which goes back to 1500 BCE.

* https://en.wikipedia.org/wiki/Talos

* https://en.wikipedia.org/wiki/Hephaestus

* https://en.wikipedia.org/wiki/Fire_worship

* https://en.wiktionary.org/wiki/Reconstruction:Proto-Indo-Eur...*

Or Prometheus.

(Explicitly referenced in the title to Shelley's most famous work.)

Come to think of it, the Hebrews -- with their stories of the golem -- may have predated the Greeks on this.

Huh.

I'm familiar with the concept, I'd no idea it was that old.

(Much of my familiarity comes from the Selected Shorts / Isaiah Sheffer reading of a short story on the topic. I can't find a recording or mention, and don't know the author.

There's also the famous Minsky statement that, paraphrasing, AI could pretty much be figured out over a solid summer of work. Appeals to authority on these questions are really annoying because people who work their entire lives on a thing are incredibly prone to think it's the most important thing in the world, and that this is the most exceptional time right now. Just statistically that's most of the time wrong.

Another example was Hinton, who according to a recent Wired interview, went down his AI doomer spiral after seeing PaLM explain to him why a joke is funny. That is such a bizarre statement it honestly makes me retroactively question some of these people's technical credentials.

A big aspect of the problem is what you might call celebrity worship. People think that because someone is accomplished in an area, we should care what they have to say in another. And AI as in stats and linear algebra has nothing to do with any of the stuff about societal implications or philosophy. So you get a mixed bag of views that basically parallel what laypeople might think, because you're not talking to these people about stuff they're experts in.

1000 years ago an expert in materials and chemicals (say) could build a mirror but still might belive it's a window into the soul or something. The supernatural and practical parts have nothing to do with each other and don't require overlapping expertise.

As a counterpoint, a good number of people worried about AI think about it this way "Think of human intelligence capability merged with machine capability". There is nothing supernatural with that line of thinking. As a human you have some very strict evolutionary limits to the peak of your intelligence. You can't overclock your brain, nor can you easily extend its capabilities outside of the limits of slow analog inputs (eyeballs/ears).

It would be the peak of human hubris to think we are the most optimal form of intelligence possible because we are the most optimal form of intelligence that currently exists.

I personally believe that far more intelligent architectures than us are possible. What I do not have the answer for is if and when we'll produce them. But if we do, and especially in the short term when the greed of humankind is still limitless, then it will be the death of us.

On the present being an exceptional time, it really is. Technological advances in the last few hundred years have changed the human condition in developed countries to almost unrecognisable levels. I saw this in my grandparents. When I talked to them about what my life was like, much of it made no sense to them. When I showed them an iPad and video called my wife and kids in China, it took a while for it to sink in that this was a live two way video feed to another continent. For quite some time I’m convinced they were worried they were being tricked. I saw them experience real, disorienting future shock multiple times.

So in many ways now really is exceptional historically, and this has been the case each generation for the last few generations. It’s likely this will continue to be true. The world my grandchildren grow up in might well be even more exceptional again.

As for Hinton, he knows what he is talking about. A.I. alignment is a fiendishly difficult problem. A lot of alignment pathologies that were once theoretical are proving to be real and difficult to avoid phenomena in LLMs. This isn’t vague poorly specified paranoia. There are a lot of very real, verified failure modes for A.I. and for some of them we genuinely have no real idea at all how to properly address them. The reason Hinton changed course is because previously he didn’t think we were anywhere near the point where such problems could pose actual dangers, but the level of advancement in the last few years has been so rapid he’s changed his assessment.

I highly recommend Robert Miles A.I. Safety channel on a YouTube. He has a lot of very good introductory videos on many issues in A.I. safety.

It almost strikes me something else is going on. Some kind of group mental illness.

I mean Ray Kurzweil's books were best sellers from what I remember 20 years ago. How could one spend their life working to make things in Kurzweil's books happen and then jump to the idea we are all going to die and it was a mistake when the smallest spark of real AI actually happens?

The audience for this is interesting too. It is like no one would want to read the Age of Spiritual Machines today because it is too positive and doesn't have enough doomer in it.

Immortality from AI has morphed into extinction from AI as what sells and captures the imagination.

I'm far more worried about AS. Artificial Stupidity

And in the UK in the late nineties we had Kevin Warwick also predicting AI woes lay ahead.

It's not that AI necessarily means our doom, it's just that the potential is rapidly approaching.

I mean you can literally see AI going faster from one day to the next. Last night GPT-4 was like 50% faster than the day before. Obviously that doesn't happen every day but it will continue to accelerate over the next few years.

So say five years down the line how fast do you think the output will be? The reason hyperspeed AI could be dangerous is because it can be given instructions like "take over" and no human can possibly keep up. So then everything becomes a competition between these AIs. You just have to hope that your side has the fastest/smartest one and not some psychopath who may just decide to make his as life-like and self-centered as possible.

Some people may think that if it goes fast that's not a big deal because it still doesn't really "think" or something but that is just denial. The leading edge models have real problem solving ability and will continue to get better. And much, much faster, as the software, models and hardware are improved. In the near term, not decades away.

There are some funny stories about Wiener in Waldrop’s book ‘The Dream Machine’.

I'd imagine this was discovered with the help of GPT.

Guy on the internet writes a whole piece about a fairly well-known historical figure who lived half a century ago, includes many scans of original news articles, can't be bothered to write out the name correctly. Wiener it is, not Weiner.

The article even contains 6 newspapers clippings where the name is spelled correctly. Infuriating…

You're both engaging in some kind of logical fallacy. The author lacks attention to detail but that doesn't mean their content isn't credible. (I'm not saying it is credible, I'm saying credibility is unrelated). I see a lot of this and find it very shallow and uninteresting. It's easy to find typos, it's hard to make substantive criticisms and so people go with what's easy.

Related, I've worked with people, scientists, that I've asked to provide useful feedback on the scientific aspects other's work, and who instead have come back with typos and grammar suggestions. This is worse than useless.

You’re projecting. Neither of them made any judgement about the quality of the article other than getting the name wrong annoyed them.

I think there’s a logical fallacy for that.

Agreed. It's pedantry at its worst. It has all the intellectual substance and contributive value as somebody who writes the word "First" on a YouTube video.

You're arguing against a strawman, neither of them said the content isn't credible (other than the implied "the name's spelled wrong, so don't take that spelling as correct.")

>It's easy to find typos

If it's easy to find typos.. what does that say about the author?

>Related, I've worked with people, scientists, that I've asked to provide useful feedback on the scientific aspects other's work, and who instead have come back with typos and grammar suggestions. This is worse than useless.

I'm assuming the work you're referring to is in a written form? Considering that the purpose of writing is communication, considering the fact that incorrect spelling, grammar, syntax and so on makes it harder for a reader to understand what's being communicated, I'd argue that it is in fact not useless.

Granted, sometimes people just aren't proficient in the language they're attempting to use and that's okay! Feedback in that case would presumably be welcome. Now if an author is simply declining to put the bare minimum of effort into their work, such as running it through a spellchecker, I think we have a moral obligation to call that kind of behavior out.

Edit: Please note the "if...then" before voting or replying, this comment is a series of logical conclusions, not necessarily a direct commentary on the comments above it in this thread. Some of it may apply here, some may not. Logically sound generalizations are more valuable, as they will apply in more situations, than my personal opinion on these specific individuals and comments in this thread

OK, let's Wien, not Wein, in the title above.

Sorry but if Eliezer whatever is name-dropped in the first sentence, I pass.

Reply on news.ycombinator.com