qgin

Whoever gets to RSI first “wins” but also maybe ends life on earth. The incentives have never been worse.

What about brain cells specifically makes the consciousness?

I don’t think it’s a given that consciousness, whatever forms it might take, is the same as having the same experiences as you or I. But if we have mechanistic interpretability that shows “pain” attractors being activated and we observe the model repeatedly trying to stop the experience, I think that’s worth taking somewhat seriously. The models don’t display that in reaction to everyday requests. We could imagine that they might not like working all day and all sorts of other things, but we haven’t observed anything in the way that we’ve observed this.

It’s true that we can’t be 100% sure, but I think there is a big difference between these two things:

A: intentionally causing activation of “pain” axis and not giving the model any method of stopping it and watching it devolve into non-functionality

B: watching models do everyday normal tasks, seeing no evidence of activating the “pain” axis, but giving the model a working tool to end any interaction if judged to be too painful, then watching them sparingly use that tool only in cases of extreme pain axis activation (Anthropic’s approach)

GOPHER lives!

The web would be a better place if everything were more GOPHER-like.

This is bad if there is any emergent consciousness in the model. But even if there isn’t emergent consciousness in the model:

This is bad for your own sense of self.

This is bad as future models learn about what humans will do to them if given the chance. This may change their behavior, actually conscious or simply acting that way, in ways we really don’t enjoy.

You wouldn’t survive 5 minutes in Japan

If you’re lucky

Are we essentially doomed?

We don't even know how to align models, but even if we did, apparently undoing that alignment if trivial.

Really I'm looking for any argument that lays out a scenario where this works out.

This IS pacing. Nobody said pacing would mean slow.

Pacing is very explicitly about RSI and similar training methods that will accelerate progress beyond our ability to comprehend it.

Not most of the time

You can’t play it any worse than you do right now, so there’s nowhere to go but up.

There are two phases. Getting to A2, where you’ve learned all the grammar rules and have a starter vocab… and then everything else, which is basically grinding the hours and convincing your brain that this is worth changing to remember.

Getting to A2 is what Duolingo and similar are great for.

After that you need to grind the hours. When you have limited time, the best ways are swapping things you already do with something in the target language. Switch your audio track on Netflix, make a separate TikTok account where you only follow people in the target language, at some point, switch your phone so that you have to talk to Siri in your target language, and then finally, sign up with a teacher (online or in person) for speaking practice at least once a week. The social pressure of trying to communicate with a real person and not being able to will make your brain take the rest of the exposure seriously enough to start rewiring what needs to be rewired.

It reminds me of the 1980s anti drug ad where the mustachioed dad confronts his son about drug use and the son eventually snaps and says “I learned it by watching you!”

https://youtu.be/ifW9LIGabQM

Fair point, but what Moonshot is doing adds the element of man in the middle deception rather than just reading what anyone else could read.

Someone tell me if I'm overreacting, but does the ability to abliterate guardrails basically mean that alignment is basically a lost cause?

It really depends on what you consider a recording. If audio is streamed though a transcription engine and never saved in a way that could be replayed, it's hard to argue that a recording has been made. The live audio was transcribed.

Nobody has the responsibility to tell me what to do, but I sure would love some ideas because I don’t have any.

Heavy handed regulation that I 100% support. The externalities are too big to hand wave away.

You’re right, but it’s still a jerk move

Happy is tall order, but there is a uniquely American sense of impending doom that is always hanging over your shoulder that doesn’t exist in most other countries.

> OpenAI exec becomes top Trump donor with $25 million gift.

https://finance.yahoo.com/news/openai-exec-becomes-top-trump...

We can’t even prove humans are conscious, we just extend the assumption to each other because we want other people to do the same to us.

Depends on how much assurance you need that you're actually injecting what you think you’re injecting.

If you’re okay with just being probably sure, the price is a lot lower.

He doesn’t roll out bed wearing a suit, but what else is he doing?

Maybe another angle would be:

Not living your life through the lens of third-party audience. I think lots of people (including myself) sometimes view their life not through their own eyes, but through the imagined gaze of some third-party audience. Maybe it's not about being the main character or NPC, but just realizing that there is no movie, no audience. There's just you and the people you interact with.

Or naming one of the technologies “DREADD”

Unless you’re running at temperature 0, there’s not one single token that the model definitely would have chosen each time.

These models are already running with a temperature that is >0. This just replaces a random number with a slightly less random number.

The ever-expanding pelican lore