Hacker News Reader

Comment by the_af | Hacker News Reader

As someone almost completely without knowledge of AI and ML, these are some signs why I'm skeptical of this kind of claims:

- Most of the imminent AGI / Singularity / Robot Apocalypse stuff seems to come, with few exceptions, not from practitioners or computer scientists specialized in AI, but from "visionaries" (in the best case), internet celebrities, people with unrelated areas of expertise, or downright cranks (who are self-proclaimed experts) such as Yudkowsky.

- The assertion that "a lot of effort/investment is going into this, so it will happen" begs the question that "this" is at all possible. If something is a dead end, no amount of investment and attention is going to bring it into existence. Quoting the article, "with this much distributed attention fixed on the problem, AGI will be solved" is not at all a given.

- Where are all the AI/ML practitioners, i.e. people who don't make a living out of predicting The End of the World, and with actual subject-matter achievements, predicting the Singularity and the Robot Apocalypse?

View on HN

> Where are all the AI/ML practitioners, i.e. people who don't make a living out of predicting The End of the World, and with actual subject-matter achievements, predicting the Singularity and the Robot Apocalypse?

The answer is in the question: they're spending most of their time doing AI research or ML work, whereas the internet celebrities who write most of what you read spend most of their time getting you to read what they write.

Vernor Vinge coined the term, and he was a computer scientist (though more famous as a science fiction writer, a profession which I guess TBF makes money from visions...).

An exponential looks the same whereever you are on it, so arguably we are in the singularity now, and have been for quite some time...

"Singularity" is a terrible term for an exponential. It's meant to convey that we can't predict what's next... which has always been the case.

The problem of predicting an exponentialling expanding search space of arbitrary conplexity is that it gets big fast. It also means each little 'bit' of information allows you to see a little further, sometimes revealing things you could never have imagined (because you couldn't see that far before).

But the above comment's entire point is there's zero reason for us to assume we're on an unbounded exponential vs a sigmoid.

> But the above comment's entire point is there's zero reason for us to assume we're on an unbounded exponential vs a sigmoid.

Something about that which is discussed at length in "The Singularity is Near" is the idea that the unbounded exponential is actually comprised of many smaller sigmoids. Technological progress looks like sigmoids locally-- for example you have the industrial revolution, the cotton gin, steam power, etc. all looking like a sigmoids of the exploitation of automation. At some point you get all those benefits and progress starts to level off like a sigmoids. Then another sigmoid comes along when you get electricity, and a new sigmoid starts. Then later we get computers, and that starts to level off, then networking comes along and we get a new sigmoid. Then deep learning... The AI winters were the levelling off of sigmoids by one way of thinking. And maybe we're tapering off on our current round of what we can do with existing architectures.

I’m no expert, just a humble biostatistics student, generally when you sum together a lot of random variables following a specific distribution you end up with basically the same distribution (scaled by the N random vars). So a lot of sigmoids put together (e.g. covid spread) will still eventually be a sigmoid. Biology seems to run on sigmoids that at first look like exponentials.

These ones aren't random, because they build on previous ones. In nature, you get a sigmoid because you run out of resource, I think? True at any scale.

Pragmatic people point will out one limit on GI is all the accessible matter in the universe.

But, theoretically, I think, there is no limit on complexity.

BTW you might like this paper on the exponential growth in complexity of life on earth, over long timescales. https://www.technologyreview.com/2013/04/15/113741/moores-la...

> generally when you sum together a lot of random variables following a specific distribution you end up with basically the same distribution (scaled by the N random vars). So a lot of sigmoids put together (e.g. covid spread) will still eventually be a sigmoid.

I haven't studied statistics very much, but I'm fairly sure the https://en.wikipedia.org/wiki/Central_limit_theorem says something a bit different from that!

Woops yes. If you sum them together you do get a normal dist. Come to think of it a cumulative normal distribution is a sigmoid.

That's the same argument. There's nothing that assures us automation itself is an unbounded exponential.

The point of the singularity isn't that technological growth will accelerate to cause some inevitable future, but that the rate of change will get so high, that 'minor' differences between how two technologies progress would lead to drastically different settings for your science fiction stories (which was Vinge's focus).

Singularity in the sense of a black hole refers to where spacetime becomes a single one-dimensional point. As far as I understand the usage in futurism, it is supposed to be similar, not in that growth is exponential, but asymptotic. The slope becomes infinite when progress is plotted against time, so all of time left to come effectively compresses to a single "state of technology" value. All possible future progress happens instantaneously.

This is, of course, not possible, but it's supposed to be an approximation. All progress is not literally instant, but in the "foom" or "hard takeoff" recursive self-improvement scenarios, developments that might have taken millennia in the past now happen in microseconds because the controlling force is just that much smarter than all of the collective powers of human science and engineering. To a human observer, it may as well be instantaneous.

To be clear, I still think this is ridiculous and am not endorsing the view, just explaining what I understand the usage to mean.

Indeed, Rich Sutton argued that we already have been through an exponential phase of self-improvement, we have been using computers to build better computers for decades, and have been using technology to improve our learning and cognition for a long time.

Piece this together with: 'Brain efficiency: Much more than you wanted to know' [1] which shows how our brains are incredibly efficient (near theoretical limits) at what they do, it's hard to think a bona fide intelligence singularity is anywhere likely. To quote Feynman, 'There is no miracle people'[2], and analogously 'There are no miracle beings' -- intelligence is built out of systems, and learning, and inference.

There's a possibility that the skills relevant to AI success are vastly different from our natural skills, such that although the human brain is highly efficient, it's efficient at the wrong things. That's clearly true in a few ways: we are not so good at arithmetic for example, a small CPU can literally be millions-billions times faster than a human at that. (that's addressed a little in the article as well) I wonder if AI could indeed be vastly better at something like computer programming, mathematics than we are. But there's no singularity (at most a Moore-like law will continue until AI intelligence saturates at a different skillset than our own).

[1] https://www.lesswrong.com/posts/xwBuoE9p8GE7RAuhd/brain-effi...

[2] https://www.youtube.com/watch?v=IIDLcaQVMqw

More importantly than what it’s capable of, the human brain is ultimately limited by our inability to increase its complexity. A human brain takes approximately 20 watts of power and uses 3 pounds of material to do its calculations. Even with less efficiency, we can make machines much larger than that. If AGI is possible at all, it should be possible to make an AGI that can have thoughts many times more complex than the human brain can. (Such a large machine might not have the reaction speed of a human, but the evolutionary pressures that required us to be able to quickly change our train of thought in response to danger aren’t a limiting factor to any designed intelligence)

Evolutionary pressure to adapt quickly is even more profound in a situation where the rate of change in the environment is proportional to the accumulated quantity of change: the claimed exponential curve of the singularity.

In my understanding (diffeq in college), the singularity in black holes and futurology are both special cases of the relatively old concept of a "singularity" in a function, which is (technically) something like a spot where the function stops being expressible as a series. The most interesting kind is where that happens because it went infinite, but IIRC it can also just not be defined there. Point is, neither Einstein nor Vinge nor especially Kurzweil invented it. :)

> An exponential looks the same whereever you are on it, so arguably we are in the singularity now, and have been for quite some time...

By that view we've been in the singularity since the first cell self-reproduced on primordial Earth.

Of course maybe that's true. As you say an exponential looks the same no matter where you are on it.

Hi! Author here. I think you raise some great points! I'll address them each:

-- I am a professional AI practitioner. I work in the field of medical deep learning and love the field. I am strongly considering starting a few experimental forays into some of the concepts I mentioned in my post as side projects, especially self-modifying model architectures.

-- Yes, you're totally right! I am making the fundamental assumption that it is possible. My reasoning is based on my belief that human behavior is simply a function of the many sensory and environmental inputs we experience. Since neural networks can approximate any function, I believe they can approximate general human behavior.

-- This is fair. The topic of the singularity is often used for traffic/fame (I mean, I'm guilty of this myself with this very post, though I hope I still managed to spark some productive discourse) and so there are always conflicts of interest to take into account. I can't name any examples off the top of my head that perfectly fit your criteria, but depending on how much you trust him, Elon Musk seems to be genuinely concerned about the potential for a malevolent singularity.

Thank you so much for your comment! I really appreciate your feedback. Have a wonderful day.

^ To be clear, mkaic is the original author of the article.

Oh, whoops, I'll edit the comment to make it more clear. Thanks for reminding me.

Cubic splines can approximate any function too, so the universality argument is a little weak IMO.

Even if one buys into the idea that human behaviour is a 'function' of sensory and environmental 'inputs', that's a long way from showing a neural net a million different texts and asking it to generalise.

I think the first AGI to pass a Turing test will probably be a simple language model. I don't think it will look like any of the GPTs, but I think text completion is a great starting point. I'm not sure how other inputs will be added into the mix, but I am sure that they will be -- heck, maybe once we train a general language model, it may very well just tell us how to incorporate things like video, audio, haptics, gyro data, etc into its architecture.

How can one turn the sinking, horrified feeling when one loses a love into a function? Or describe in terms of a function, the blissful wonder of being in the arms of a lover? An issue I have is the that there seem to be profound limitations to language, explored in the philosophy of language, that fail to capture much, if not most, of the world. Functionalist models of mind and behaviour seem extremely limited, as our subjective ontology doesn't seem to reduce to functional outputs.

You also say that an AI would rapidly consume the whole of human knowledge. For me, the totality of human knowledge would become a mass of contradictory statements, with little to choose between them on a linguistic level.

There are, for me, profound philosophical issues with creating a mind that is "conscious" in the sense that an AGI is implied to be, as a purely symbolic logical construction. Language is the only tool we have for programming a mind, and yet the mind cannot be completely described in language, nor can language seem to properly encompass whatever the fundamental ontology of reality involves. I don't feel there will be a "free lunch" where we advance computer science to the point where we get an explosion of the kind AI1 designs a better AI2, which designs an even better AI3, and so on. This seems to have a perpetual motion feeling to it, rather than one of evolution. It isn't to say AGI is impossible, but I believe that like everything else in computer science it will have to be solved pragmatically, and realising this could be an extremely long way off.

> How can one turn the sinking, horrified feeling when one loses a love into a function?"

The same way the brain does. Those complex feelings can eventually be resolved into dumb neurophysics. Love, fear, anxiety etc. al are just electrical impulses tickling chemicals. Is there anything in our brains that we could never approximate with technology?

This is my view as well. It's a little unnerving and it definitely starts to overlap with the whole "free will" debate, but yeah, I don't see any reason why we can't fundamentally replicate the behaviors exhibited by the brain. It doesn't violate any laws of physics.

We've had technology like poetry, art, and music for thousands of years, and yet no symbolic description of the feeling can contain the feeling of what it's uniquely like for me. Even though we could try and model the brain, say, functionalist models fail to capture qualia as it doesn't reduce to behaviour. To replicate the brain fully in a computer would need a full description of its chemistry and physics, along with that of the greater universe, which we don't have, and to describe it coherently to simulate it is a problem of the order of magnitude more difficult than ones we're going to be able to code for the foreseeable future.

I agree that it'll be while before we fully understand the brain but I don't have any doubts that we'll get there eventually. I am curious though, why would we need to understand the greater universe perfectly as well?

Where does the brain get its inputs from? These computational models are based on an ontology where the brain is an isolated box separate from the universe, which is only one of the many philosophical outlooks argued over. For example, most schools of Buddhist philosophy would regard this separation as entirely the wrong picture of the world.

Philosophers have debated this since at least ancient greece. It's hardly a solved question, and about a third do believe that there is something in our brain that we could never approximate with technology.

Have any of that third put up any testable theories about that something? Or are there leading theories as to what the something might be?

If a philosopher had a testable theory I don't think they'd be a philosopher anymore.

This assumes that there is no free will however :)

> How can one turn the sinking, horrified feeling when one loses a love into a function?

… or into electrical activity in the brain.

Can you point to a paper or website that contains this function, described in full? Yes, electromagnetism no doubt, but while we can postulate a function, we still don't have the function, and will somehow have to write it down for it to be a function.

> somehow have to write it down for it to be a function.

That’s not how AI is trained. You don’t need to know or even understand the resulting model to train it.

The training algorithms have little to do with the resulting model, and are certainly not themselves intelligent.

Why assume that the activity is purely electrical?

It's not purely electrical. It's also chemical, especially if you look at how synapses work. (But then again, chemical processes are driven by electric charges anyway.)

I don't think AGI is seeking to create a machine that can love, and I think it would be even less capable of it than "mere" logical intelligence.

Thank God too, the moral questions involved are truly terrifying (humanity as a Great Demiurge, birthing monstrosities).

> How can one turn the sinking, horrified feeling when one loses a love into a function? Or describe in terms of a function, the blissful wonder of being in the arms of a lover?

Evocative questions but I have to challenge the premise. First, that things like emotions and qualia are design ends in themselves for a successful AGI rather than potential emergent properties of same.

For that matter, are they really necessary to the brief?

> An issue I have is the that there seem to be profound limitations to language, explored in the philosophy of language, that fail to capture much, if not most, of the world.

And how much of the world does a human mind capture?

The piece already accounts for this claim. The theory is that all language has to do is describe a sophisticated enough network. After that it's black boxes all the way down.

> Language is the only tool we have for programming a mind

And a darn good one. Formal languages can express a great deal when you find the right abstractions.

> Most of the imminent AGI / Singularity / Robot Apocalypse stuff seems to come, with few exceptions, not from practitioners or computer scientists specialized in AI

I'm a practitioner at a top company for many years now, and I think we're getting close to a point of no return. I think best case scenario is some form of unrecognizably transformed "humanity", worst case is too horrible to even spell it out.

The reason why this is not the consensus among experts has to do with a combination of blind spots and biases. Humans are especially bad at (a) evaluating / reflecting on themselves, and (b) extrapolating, especially with non-linear processes.

That’s what experts in fusion have been saying for 50 years though… now that Moore’s law has ended, it might be a long while…

I have seldom seen predictions about technology come to fruition at the predicted date. If they come at all (they usually don't), it's usually much later. Sure, some naysayers might have said "we will never solve chess/go/etc." but I think they were overall fewer than the people who thought it was right around the corner (and it was more of a philosophical argument than a scientific one).

Totally unpredicted advances coming out of the blue, yeah, that happens often. But as far as AGI goes, it's been predicted pretty much every year since the 70s. Being too conservative about extrapolation doesn't seem to be our problem. If anything, I think we're overeager.

    In 2019, 32 AI experts participated in a survey on AGI timing: 

    45% of respondents predict a date before 2060
    34% of all participants predicted a date after 2060
    21% of participants predicted that singularity will never occur.

https://research.aimultiple.com/artificial-general-intellige...

Why 2060? I'd LOVE to see the ages of the respondents in each group.

(I have a phd in CS and am, by most reasonable definitions, an "AI Expert". Whatever the hell that means. I've been a respondent in very similar surveys run by PIs at fancy universities and so on. These responses are always wild ass guesses and should be totally ignored. I've even left a comment to this effect on one such survey.)

The prediction contest here has a peak at around 2042: https://www.metaculus.com/questions/3479/when-will-the-first...

Yudkowsky doesn't strike me as a crank. Why do you say that?

What has he ever done in AI except talk about it? At best he's an AI promoter, and hype-men are often cranks or scammers (see also: VR, web3, cryptocurrencies, MLM)

Yudkowsky is a philosopher, in the sense of someone who thinks a lot about things that haven't been achieved yet. Lots of otherwise smart people (wrongly!) discount the value of philosophy, but it's close by every time there's a paradigm shift in humanity's knowledge. Philosophers can be scientists and vice versa.

If anything, I'm surprised that this philosophy isn't mentioned more in a thread where the author gleefully talks about ML being used to create better AI, layer by layer until the thing is even more opaque than what we're currently working with.

This is terrifying, as we currently have only very loose ideas about how to reliably ensure that a powerful reinforcement learning system doesn't accidentally optimize for something we don't want. The current paradigm is "turn it off", which works well for now but seems like a fragile shield long-term.

> Yudkowsky is a philosopher

At least inasmuch as anyone who thinks about stuff can be considered a philosopher. But he strikes me much more as a self-appointed expert on matters where he shows no achievements.

He writes fanfiction about AI rather than actually doing stuff with AI.

Most people who think about stuff don’t leave a mountain of highly organized and entertaining essays for posterity.

That just means he's a prolific writer, which I never questioned.

I'm arguing that what he writes is fanfiction (which he takes way too seriously), and that he's not an expert in AI and therefore we shouldn't take his predictions too seriously.

There is long history of science-fiction writers painting correct visions of, at the time seemingly impossible future. Sometimes You don't have to be an expert to notice the trends in certain domains. I'd go even further - it could be easier to notice the big picture without being overly bothered with nitty-gritty technical details - yes, to further the field they are necessary, but to comment the direction the field is heading in and about its implications for society they are not. I don't necessarily agree with the author, I'm just making a general comment.

> "There is long history of science-fiction writers painting correct visions of, at the time seemingly impossible future"

That's a pretty weak argument. I love scifi, and I love for example Philip Dick's writing, yet I would not consider PKD's opinion on the future of AI/AGI particularly relevant.

James Cameron is not an authority on AI either.

If people said "Yudkowsky is a nice fanfiction author" it would be one thing. But he considers himself an actual AI researcher, and that's just not right. He is not qualified, and has no accomplishments in the area, other than writing fanfiction about it.

You keep hammering on with the same lazy slander. Yudkowsky was well known long before he wrote the popular Harry Potter fanfic, which incidentally is pedagogical / allegorical. Because his main roles are teacher and philosopher, and philosophers do that.

Here's a sampling of his non-fiction writing:

https://www.lesswrong.com/tag/sequences

https://intelligence.org/files/EthicsofAI.pdf

https://arxiv.org/abs/1710.05060

https://intelligence.org/files/AlignmentMachineLearning.pdf

https://arxiv.org/abs/1401.5577

https://intelligence.org/files/Corrigibility.pdf

https://intelligence.org/files/DefinabilityTruthDraft.pdf

https://intelligence.org/files/IEM.pdf

https://intelligence.org/files/TilingAgentsDraft.pdf

But you'll just say there's nothing of value there, and it's somehow figuratively "fan fiction", because he didn't go to college, and he doesn't work much on ML, which is clearly the end-all of AI.

"There is long history of science-fiction writers painting correct visions of, at the time seemingly impossible future."

Science fiction writers tell entertaining lies to amuse their readership. They are generally not really trying to "pain correct visions" of anything, and you are greatly exaggerating the extent they have success in this endeavor, which again, is generally not something they are even trying to accomplish.

Nick Bostrom is a professional philosopher who has also thought a lot about AGI and simulation scenarios. But just because educated, smart people can think a lot about something doesn't it mean it will necessarily happen. I'm sure there's quite a few people who have thought at length about warp drives and wormholes. That doesn't mean we'll ever be able to make use of them.

Nick Bostrom is also an egotistical grinder, who couldn't programme a microwave.

I agree, that guy's also a whole lotta bad takes put all together. Like a slutty potato.

In the few contact I've had with philosophers, I've got the (maybe wrong and oversimplified, but nonetheless evident) impression that they find philosophy fun and mentally stimulating, and they enjoy academically talking and writing about edge cases no matter their feasibility.

I think it's a net good when philosophers mentally explore scenarios most people don't consider, regardless of their plausibility, but they will not be taken more seriously that Peter Watts looking for inspiration for his next scifi book unless they offer some kind of evidence to sustain their conclusions.

The handful of philosophers I've known are very smart, interesting people, but except for one who teaches philosophy, they don't live off philosophy. Most have a side job, and one lives with his parents. It's as if philosophy was more of a mentally rewarding hobby than a job.

Mentally rewarding hobbies can be fruitful, though. Consider the case of Oliver Heaviside, the person who came up with Maxwell's equations of electromagnetism in the form that we use today, as well as several other useful things.

I have no doubt that he was thought a crank by many, nor that many other actual cranks were claiming discoveries based on Maxwell's work. One can only tell the difference in the rear-view mirror.

1. https://en.wikipedia.org/wiki/Oliver_Heaviside

Why do people have to use the derogatory word «crank»? It’s perfectly possible to just be smart, sincere and wrong.

Talking about paradigm shifts again, not a single one of them matched the consensus at the time.

Isaac Newton spent a lot of time thinking about alchemy and religion, for heaven’s sake. During his lifetime, it wasn’t obvious even to the smartest thinkers whether science, the Bible or quasi-religious rituals was the best tool for understanding the world.

Conventional thinkers expect that the future will look like what they know, and it leads to frequent, overconfident dismissal of everything that is unconventional.

By all means, disagree and explore in your own direction. But don’t go around degrading those you disagree with. It’s just so average.

> Why do people have to use the derogatory word «crank»?

The word "crank" has the connotation of someone self-deluded, aggressively promoting their beliefs and reacting badly to critique, and who decides for some reason to ignore the normal channels of peer-reviews, academia, and scientific research.

You can be wrong, and then you can be wrong and also lack formal education, forgo presenting your findings in peer-reviewed journals in the relevant area of your research, and decide that instead of joining the mainstream, you can completely sidestep it by creating your own "research institute" (of which you are of course a "fellow", because why not). Your findings and papers can then be self-published on the internet, bypassing any quality controls. Bonus point if your theories paint a fringe doomsday picture, "Roko's Basilisk is out to get you", "the most important existential threat to humanity is malign AI", etc.

About the only item in the crackpot index that Yudkowsky doesn't tick is the "they are trying to suppress my truth!", to his credit.

Each of those can be of little importance, but all taken together paint a pretty definitive picture.

I really don't understand the hate. Honestly. This isn't merely disagreement.

> Mentally rewarding hobbies can be fruitful

Unreservedly agree. Even if they're not as fruitful as Oliver Heaviside's effort, a mentally rewarding hobby is ultimately your business, and as long as you don't harm others, no one has grounds to judge you.

If Oliver Heaviside had started giving conferences warning that Maxwell's equations demonstrate FTL communication is possible, which could trigger a paradox that collapses reality, he wouldn't have such a good view in the rear-view mirror unless he offered some kind of evidence. Maybe Oliver would have been right, maybe not, it would be impossible to judge without concrete evidence.

This is very uncharitable. He’s a prolific neopositivist-ish philosopher with a distinct voice. He’s a good decision theorist. He doesn’t publish much himself, but he directly mentors, advises, and collaborates with people who do.

He wrote a thesis on decision theory.

He wrote a thesis in the sense of he actually wrote a thesis for a known university, that is referenced and quoted in academia by decision theory scholars, or do you mean he self-published a "thesis" in the sense he wrote something that nobody in the field pays much attention to and that cannot be published in peer reviewed journals?

If the latter, anybody can do that.

> Yudkowsky doesn't strike me as a crank. Why do you say that?

He is a minor internet celebrity whose only claim to fame is writing fanfiction about AI, the whole "rationality" cult, and is a self-proclaimed expert on matters where he shows no achievements (like AI) while making unsupported doomsday predictions about evil AI, the Singularity, etc. Also, that deal with Roko's Basilisk that he now wishes never occurred (oh, yes, "it was a joke").

Mostly, someone with no studies and no achievements making wild doomsday predictions. Doesn't that strike you as a crank?

An analogy would be if I made wild assertions about the physics of the universe without having studied physics, without any lab work, without engaging in discussion with qualified experts, with no peer reviews, and all I presented as evidence of my revolutionary findings about the universe was some fanfiction in my blog. Oh, and I created the Extraordinary Physics Institute.

What are his qualifications?

Isn't this appeal to authority? He's obviously pretty smart and a lot of people take the risk modeling thinking seriously, and his arguments and output deserve to be evaluated on their own merits.

(Also on the tech practicioner side there's obviously lots of major figures that don't have formal qualifications)

If we're going to talk about logical fallacies instead of answering the question, your post is an example of the argumentum ad populum fallacy.

If someone is going to market themselves as an AI expert, I expect them to have experience and qualifications to back up their opinions.

The trajectory of progress gives evidence that it is in fact possible and likely. Any venture into the unknown could be unsuccessful but if you see progress you can start to make estimates.

And yes most of the “robots will kill us” comes from people who aren’t building the algorithms. This could be biased in people not thinking their work is harmful but is most likely that once you see how the sausage is made you are less worried about it.

> The trajectory of progress gives evidence that it is in fact possible and likely

100% disagree. In fact, I'd argue that the opposite is often true, where you see initially a fast rate of progress that results in diminishing returns over time. It's like estimating that a kid who was 3 feet tall at age 5 and 4 feet tall at age 10 will be 10 feet tall at age 40.

I have very strong skepticism of any sort of hand-wavy "Look, we've made some progress, so it's highly likely we'll eventually cross some threshold that results in infinite progress."

Pareto principle; we get 80 percent there and that last 20 becomes the new 100.

We keep diving into one infinitely big little number pattern fractal after another, chasing poetry to alleviate the existential dread of biological death.

The idea we can fundamentally hang the churn the universe given the vastness of its mass and unseen churn is pretty funny to me.

Information may be forever but without the right sorting method you can’t reconstruct it once scattered. Ah, our delusions of permanence.

Ray Kurzweil works on AI at Google and he wrote The Book about the coming singularity.

I consider Kurzweil unreliable on this subject, as well as on the similarly kooky topic of immortality. He belongs to the "internet celebrity"/"guru" cadre rather than to the scientific community doing serious research on AI. In fact, his actual subject matter expertise isn't in AI. He is a "futurist", not a real researcher.

I was definitely thinking of Kurzweil, not only Yudkowsky, when mentioning internet cranks.

Working for Google is not enough. It's like when someone who's an accomplished physicist decides to give their opinions on life, biology, evolution, etc: not their area of expertise, so we don't need to hold their opinions in particularly high regard.

Reply on news.ycombinator.com