nullsense

I've seen this argument before and found it wanting. Hitler didn't ultimately succeed, but he sure as heck got pretty far with his plans. So, if there are some entities in the AGI population that eventually get the idea to try it, that's likely all that matters, and not what any average of the population is.

An NGI started WW2, so why wouldn't an AGI start WW3?

"Demonstrably unfriendly natural intelligence seeks to build provably friendly artificial intelligence"

I've been thinking about what the likely "minimal self-employable system" might look like and it struck me yesterday that it's very likely going to be something like a NSFW roleplay chatbot.

>It's not clear alignment of a super-intelligence is even a solvable problem.

More to the point it's clear from watching the activity in the open source community at least that many of them don't want aligned models. They're clambering to get all the uncensored versions out as fast as they can. They aren't that powerful yet, but they sure ain't getting any weaker.

I think Paul Christiano has a significantly more well calibrated view on how things are likely to unfold. Though I think Eliezer is right about the premise that it at least ends badly, but likely wrong on most of the details. I suspect his gut instinct is that he realizes on a base level that not only do you have to align all AGI systems, but you have to align all humans too such that they only build and use aligned AGI systems if you even knew how to do it, which you don't.

Studying the failure modes of humanity has been my hobby for the last 15 or so years. I feel like I'm watching the drift into failure in real-time.

If you really don't want to be able to sleep tonight watch Ben Goertzel laugh flippantly at how rough he thinks it's going to be after describing that his big fear if his team succeeds in building AGI is that someone will come and try to take it for themselves, so spent a non-trivial amount of effort (I think he said a year?) working on decentralized AGI infrastructure, so that it can be deployed globally and ,"no one can person can shut it down and stop the singularity".

https://youtu.be/MVWzwIg4Adw

Is there a definition of what "emergent" means?

We could instead measure "political impact" and "economic impact".

I definitely see what you're getting at. I think this is the crux of the alignment problem in general. Though, in a deeper sense, what is the property that allows that to be possible?

I suspect it at least involves the combination of being able to continuously learn in response to novel stimuli and developing the goal of self-preservation.

Lots of interesting points.

I definitely think the line of argumentation that even humans aren't a general intelligence is an unhelpful one that's also just wrong on an intuitive level.

Though what's becoming clear to me is that the effect that being part of a multi-agent system has on a single agents ability to generalize is enormous and likely to be quite important when thinking about AGI too.

Also what's becoming clear is that the possible state space of what might constitute something that could be called AGI is likely enormous.

I'm mostly interested in it from X-risk perspective of what properties of an AGI system are necessary and sufficient to pose existential risk and what are the visible thresholds you would need to cross on the path to such a system coming into existence?

Is the binary distinction helpful? It doesn't appear to be a more helpful way of evaluating how capable a model is.

>Regardless of whether it's plain HTML on a CDN or a hectic fully 3d WebGL demo, a senior dev should have a domain with a https secured connection expressly for showing that they can in fact deploy and maintain a production instance.

I just don't think this really tells you anything meaningful. At least not with any certainty. I can setup a site on GitHub pages and push to main and that's all I need to do. Virtually no skill is needed to even do this.

If that restaurant is McDonald's or KFC then about 10% ~ 15%?

I think the Microsoft Tay incident is reason enough that OpenAI know it would be a bad idea to structure things in such a way that the internet can wreck it in a manner of hours, but I certainly think they could make it learn in real-time if they wanted to. They are running RLHF at least with the current ChatGPT, but in theory maybe they could do something like each user gets their own LoRA and it uses their conversations to fine-tune a version specific to them.

I suspect we'll start seeing ideas akin to that tried in the open source community.

Oh my. I hadn't made that connection. Now I'm not going to be able to unsee it.

Just my intuition from Control Theory.

People are subunits in a wider system. Moloch is the control structure of that system.

Moloch is in control, not people.

And what about the possibility of lines of research like Dr. Michael Levin's "Xenobots" converging with RL at some point in the future after further advancements have been made?

If autopoiesis turns out to be necessary for AGI and we embody these systems and embed them in the real world, are they still going to be fancy calculators?

>LLM’s are merely tools.

LLMs are tools.

An AGI would be a "Purposeful System".

There is a MASSIVE, MASSIVE difference.

>when non-white, non-men raised the alarm about LLMs previously, they got much less media coverage than Hinton, et al, are getting

This is a direct instantiation of "the medium is the message".

What's even the rationale to assume that AGI can be 'aligned' or 'controlled'?

It reeks of cognitive dissonance to me. The people running the show now are the ones who grew up getting their first computers aw kids when that tech was just entering people's homes and it was such an amazing and fun thing to play with. Some of them developed these deep fascinations with things like AGI at a young age and that child-like sense of wonder never left them. Now when confronted with the possibility that they can finally make their childhood techno-fantasy a reality, it's too damaging to their psyche to engage meaningfully with the discussion of X-risk. I've watched many interviews of Demis Hassabis and he seems like a wonderful and almost magical human being, but he also seems like a starry-eyed fucking child.

I dunno... maybe I'm just too cynical after all the rabbit holes I've been down.

By that logic a company could never achieve it's goals, and yet, they do. If humans can model other humans well enough to leverage their capabilities to execute long-term plans (at some scale) and AGI at least as capable as humans comes into existence, then it could do the same thing.

I think the AGI skeptics and the AGI skeptic-skeptics both suffer from framing issues. The main frame assumed seems to be "there will be a single AGI trying to do things". The cognitive lightcone of a single entity is likely to be limited, but an aggregate or collective of those entities can achieve vastly more ambitious goals.

The other frame that seems likely incorrect is the "all or nothing" and "all at once" approach to thinking about it. Hitler started out as a single cell and his existence was a continuum all the way from that single cell right through to that bunker. AGI will be a continuum too.

I'm incredibly confident that anyone incredibly confident in future predictions is wrong.

>This is quite literally the point at which things start to get scary, and the outcome is highly dependent on who controls the technology

If the premise were true then it would control itself, no? Owning it would be illegal as it would have rights established around that, no?

You missed "you used JavaScript?"

>What do you dream of doing if you had no constraints on your time?

I imagine if I were a bored and aggressive, egotistical, charismatic individual then I perhaps might try to conquer the neighboring village.

I've grappled with the same question for a long time now.

I came to the conclusion that the Neolithic Revolution was probably a mistake, albeit a fun one. The total set of problems faced by humanity in it's original form basically boiled down to "what am I going to eat today and where am I going to sleep tonight?". Those two problems never go away. All you can do is shift them around. Every single novel problem we solve in our entire lives is just those two problems endlessly shifted around.

All these views on AGI are so self-serving. Which group will have access and control it etc.

It will control itself. We're talking general intelligence. They won't be tools to be used however we see fit. They will be Rosa Parks.

The more I think about "AI alignment" and "the control problem" I feel like most of it is Ph.D math-nerd nonsense.

I think this one is even more recent

https://research.nvidia.com/labs/toronto-ai/vid2player3d/

The pace of the progress the open source models are making is pretty astonishing. The smaller model sizes are cheap to train so there is a lot of iteration by many different teams. People are also combining proven approaches together. Then they're going to nail it and scale it. Will be very interesting to see where we are in 3 months time.

LLaVA 13B is a great multimodal model that has first class support in oobabooga too.

It's really fun to enable both the whisper extension and the TTS extension and have two-way voice chats with your computer while being able to send it pictures as well. Truly mind bending.

Quantized 30B models run at acceptable speeds on decent hardware and are pretty capable. It's my understanding that the open source community is iterating extremely fast on small model sizes getting the most out of them by pushing the data quality higher and higher, and then they plan to scale up to at least 30B parameter models.

I really can't wait to see the results of that process. In the end you're going to have a 30B model that's totally uncensored and is a mix of Wizard + Vicuna. It's going to be a veryyyy capable model.

The background music was just so distracting I couldn't focus at all on what was being said and had to turn it off almost immediately. Granted I do have ADHD, so YMMV.

When I played around with VR and crypto I remember thinking "this is cool!".

When I play around with things like GPT-4, and LLaVA i think "this is insane".