Hacker News Reader

> 1. For "the singularity" to happen, we probably need something more to happen than just chatGPT to ingest more data or use more processing power.

It's not actually clear what "the singularity" is? Is it something running out of control or it's still controllable? There is a blurry line. People are afraid because they think it's sort of uncontrollable explosion.

The second question is about AGI. What is it? Is it something 'alive' or just a generic AI calculator with no 'creature' features. Like self preservation at least.

I think our view of these two things will change soon as we get a close up picture. Pretty much like Turing test doesn't look great anymore. As even dumb chatbots can pass.

I personally define AGI as a technology capable of improving itself exponentially.

But I realize my definition is in the minority. :/

Of course if we ever manage to make a 1:1 cybernetic brain that works exactly like a human's brain, and is also a complete black box, we'll have achieved AGI. I'm not sure how useful that will be, but I'll have to admit it is AGI.

So maybe I should say, "interesting AGI" is technology that can improve itself exponentially. :-D

Yeah, if you could input data set X with quality Q(X) and output data set Y of the same size with Q(Y) > Q(X), you'd really be on to something. But I don't think such a system exists yet, or even close. Inputting the internet, outputting a sea of garbage with a handful of diamonds that people have to spelunk through the garbage for seems the best so far. Madlibs is a pretty equivalent activity, and while fun, certainly not anything one would consider AGI. We need a revolutionary improvement on automated spelunking to get anywhere. Maybe we'll get a good spam filter as a side effect!

But even if you had a system, there's still the resource cost to run the algorithm (needs to be bounded or else you've just made a finite jump) and the gains you make need to not decay (or again you've just made a finite jump).

And all this needs to be compared against investing in humans - which seem pretty clearly to have AGI properties (but with some really bad constant factors. 20 year training time - ridiculous!)

To me it seems things are a long way off and least a couple of major innovations away. But at least there's some ideas of the problems to tackle, which is a big step up!

> can improve itself exponentially

This is close to singularity. Except 'does' instead of 'can'. A big difference ;)

Probably we need several AGI terms. Because sub-human robot capable of doing many not pre-programmed thing is sort of it. Still not smart enough to improve itself.

Actually most humans, the smartest creatures, cannot improve even current AI. Demand for self improvement will put its IQ in top 0.01% of all known intelligent creatures. Which is probably too much for just AGI, we may not recognize it when it will be already here. And there is another question. With such IQ do we really want to keep it slave forever?

I personally define AGI as a technology capable of improving itself exponentially.

Have you thought this through though? I mean you'd first have to know in what way to improve, which would become more challenging as you became more perfect, you'd have to want to become perfect (which let's be honest might be boring) and then I guess there's the fact that if you kept evolving / improving yourself exponentially, then you'd no longer exist because you'd likely be morphing into other forms all the time?

In a way, maybe the only thing I can think of that is observably doing something like this is the universe itself.

> If you want to save lives, it's probably more useful to think about lives saved today than hypothetical lives you could save 100 years from now. Not that you shouldn't consider the long term, but your ability to predict the future gets worse and worse the farther you project out.

This reasoning, taken to its logical extent, would mean completely defunding all medical research and instead redeploying all research to best utilizing our current technology.

That is, hopefully, an obviously faulty decision.

We must balance research with using current findings to help people. Stopping all drug development and telling people with currently untreatable diseases "haha, too bad, we aren't even going to try", is cruel.

The ultimate goal of many technology utopians is the end of all death. We have examples from the natural world of incredibly long lived higher life forms, and simple life forms that are immortal, so it isn't scientifically impossible. The number of lives saved is literally, infinite. It is really hard to argue with "infinite upside".

(The absolute societal shit show that such a technology would bring about is a rather important question...)

> "we're killing more people by delaying time inventing the hypothetical life saving technology" is not very useful either, because you can't actually say how many lives would be saved versus harmed.

For many future inventions, of course estimates can be made of the number of lives saved.

The more pie in the sky research is, the larger those error bars become. "LLMs may one day help us sift through research and cure all forms of cancer" is, technically true, but the error bars on it are so high that you don't want to devote all the world's resources to LLMs just on the hopes of eliminating all forms of cancer 50+ years from now.

"This company is working on AI Vision+Robotics technology so people don't have to do this super hazardous job that claims a bunch of lives. They are an estimated 5 years from productizing and they already have purchase contracts in place" is a much different statement.

I would prefer cults about nerd topics (which only someone with a desktop or even laptop computer would read about, forget mobile device users) over nationalism based on phenomenological skin color.

And e/acc is more of a little club than it is a cult. But I guess you could also consider it to be a criminal gang very much not unlike the Crips and the Bloods. Why? Because they're part of a counter-culture and naturally they want to diverge from the mainstream thought process. And the criminal activity e/acc engages in is non-compliant computer engineering. Which the Cathedral (pardon my language) dislikes.

Basically, this is evolution in action. You're looking at the beginnings of a new species. Millions of new species are coming, really.

Well, this site often deals with fringe stuff. Take Lisp, for example. It is a fringe language, no matter how zealous its advocates are. So is Haskell. But you see articles about them posted here far out of proportion to their real-world use and impact.

And I disagree that "most of their assumptions go unchallenged". I see challenges to the assumptions of lesswrong, e/acc, and other such stuff often here. (Maybe less often than is warranted, but still fairly often.)

"It is the mark of an educated mind to be able to entertain a thought without accepting it." - often attributed to Aristotle, but apparently not actually from him. Still a good thought. We can post about this stuff, and discuss it, without buying it.

I agree with your pushbacks.

9. Anyone who doesn’t recognize Roko’s basilisk as a bad ripoff of Pascal’s wager is an ignoramus. Anyone who is seriously concerned about it for more than an hour should consider a vasectomy.

10. I can’t think of a better form of “effective altruism” than paying for LessWrong members’ vasectomies. Not because I’m concerned about their genes. They’d just be terrible parents, and gods those kids would need lots of therapy.

As with other accelerations manifestos I’ve read, I’m still not unconvinced e/acc isn’t a shitpost at best, and drug induced psychosis at worst.

Hey, buddy (yes, I am intentionally being condescendingly familiar) these cults/movements are self-serving and cannot be reasoned with via rationality under the assumption they’re good faith actors.

This is akin to falling for TikTok/etc. ragebait and trying to have a discourse in the comment’s section.

The subcultures allow for the exchange of material and non-material goods — primarily money and “woo.” Everything else is just a tool to that end.