comp_throw7

> My response to that is: it's good to want things. He doesn't get to ask the world to forget his name.

Is this anything other than a naked assertion of force, that might makes right? He couldn't stop it, therefore it's fine that it happened to him? (Also, it's extremely... something... to describe "he asked a journalist to not print his name on the front page of the NYT" as "asking the world to forget his name", as if his real name was already the primary referrent by which he wielded his influence, and he wanted to shield that power from scrutiny. This was obviously not the case, and is _still_ not the case despite the article, which is why nobody has actually made a compelling argument for why including his name in the article was _good_ rather than _something Cade Metz had the power to do_. I in fact don't particularly think that Cade Metz did it to deliberately hurt Scott, I just think he's a blankface who didn't care that his usual modus operandi would sometimes hurt people for no good reason and was unable to step out of his frame enough to actually check whether what he was doing made any sense, in that instance.)

> I actually talked to therapists when this whole story broke out, and none of them said this "patients must not be able to Google your blog" thing was an actual thing. People just believe it because they like Scott Alexander and believe whatever he tells them.

That you describe it as "patients must not be able to Google your blog" makes me not particularly trust the reports of those therapists. I, too, talked to some therapists, who thought that Scott's concerns were reasonable. Not that there was an overriding professional duty, sure, but that wasn't the claim, either. I dunno, man. The attitude you have towards this really seems like, "well, getting slapped isn't that bad, and you're not strong enough to stop him... maybe stop complaining?" What good thing happened when Cade Metz put his name in print? If you want to adopt a principled stance against pseudonymous writing online, do that. But don't pretend that Scott's failure to keep a pristine separation between his real name and his entire history of online writing somehow makes it so that the NYT printing his name is merely the maintaining the status quo ("ask the world to forget his name"), rather than dramatically expanding the circle of people for whom his identity was deanonymized.

That's an empirical claim, not a "base principle" (whatever that is). And, uh, yeah, I agree that I am not re-examining my beliefs in response to a bunch of random unsupported accusations by people who are demonstrably not familiar with the thing they're attacking. You should update your beliefs in response to evidence, not in response to social attacks.

Your persistent refusal to acknowledge that Scott did not want to go from a world where his patients Googling his real name did _not_ immediately lead to his blog, to a world where it _did_ immediately lead to his blog, and that was his primary (and valid) objection to having his real first and last name put into print, is baffling.

You keep saying things that are proven to be trivially false with 5 seconds of examination, so I think the dismissal is justified.

Global Health & Development is like 40% of EA funding today. Please stop making things up.

> Calling someone a dumbass in this situation is a kindness

First they laugh...

> I don't think any "Rationalists" I ever met would actually consider concepts like scientific method...

In that case I don't think you've met any of the people under discussion.

What a deeply confused criticism of rationalists, who are possibly the only meaningful social group in existence to celebrate changing their minds in response to new evidence.

This is not a commonly-held position in EA.

Someone pointed this out in the comments section of the original post, but this was entirely missing the point Eliezer was trying to make, which was that these incidents serve as a narrow refutation of alignment-by-default claims, because either these LLMs aren't coherent enough to be meaningfully "aligned" at all, or they are, and are nevertheless feeding the delusions of many people who talk to them in ways we'd consider grossly immoral if done by a human. (One can quibble with whether these two possibilities cover all the available options, but that would be the argument to engage with, not with whatever the author understood EY to be saying.)

It was not told to do so.

This is a non-answer that doesn't explain any difference in capabilities between GPT-3.5 and Claude 4 Opus.

Not quite. My point was mostly that the term made more sense in its original context rather than the one it's been co-opted for. But it was convenient for various people to use the term for other stuff, and languages gonna language.

I think this post is sort of confused because, centrally, the reason "AI Alignment" is a thing people talk about is because the problem, as originally envisioned, was to figure out how to avoid having superintelligent AI kill everyone. For a variety of reasons the term no longer refers primarily to that core problem, so the reason so many things that look like engineering problems have that label is mostly a historical artifact.

It was a pre-existing term of art.

Confusing take - Gemini 2.5 is probably the best general purpose coding model right now, and before that it was Sonnet 3.5. (Maybe 3.7 if you can get it to be less reward-hacky.) OpenAI hasn't had the best coding model for... coming up on a year, now? (o1-pro probably "outperformed" Sonnet 3.5 but you'd be waiting 10 minutes for a response, so.)

Surely you're familiar with https://ai.meta.com/research/cicero/diplomacy/ (2022)?

> I wonder who pays the bills of the authors. And your bills, for that matter.

Also, what a weirdly conspiratorial question. There's a prominent "Who are we?" button near the top of the page and it's not a secret what any of the authors did or do for a living.

> Sorry, when you are the president you don't get that leeway to make mistakes and then correct them by saying "sorry, my bad!" No way.

What does "don't get to" mean? Descriptively, this is false; normatively, this seems like a completely impossible standard to hold presidents to. You want them to literally never make any mistakes?

> I've been following Argentina's struggles for a while given my time in Brazil and the parallels to hyperinflation so I was willing to consider his unorthodox approach, but perhaps it blinded me?

???

The term "alignment" was first coined in this context as a suggestion from Stuart Russell to MIRI, in 2014: https://www.lesswrong.com/posts/ZxWzCGKzX84S7DBZ9/when-was-t...

Both, of course, are concerned primarily with the risk of human extinction from AI.

This is not even close to true, though it's true that many of the people at the big AI labs estimate non-trivial odds of human extinction downstream of AI progress. Some of those people are working on "safety", but some are indeed working on capabilities, and have all sorts of clever reasons for why the thing they're doing is net-good (like putting worse odds on human survival if a less-careful competitor gets there first).

But ultimately, most people who think we stand a decent chance of dying because of this are not working at AI labs.

This claim is already falsified by existing AI capabilities, which neither required simulating how humans solve similar problems, nor required understanding how to hand-code the relevant algorithms (and indeed we don't even have the interpretability tools to know what those algorithms are yet!).

Such a contract - where someone promises to use their (future) presidential powers in exchange for some consideration from the other party - would be illegal and unenforceable, because someone paying the president to exercise executive powers to their benefit is literally just bribery.

There is no mechanism by which Trump can offer a statement of intent that legally binds him to following that specific course of action after he becomes president.

Trump isn't president yet, so any such conversations are not legally significant actions the way the person you're responding to meant.

An insurance company's "role" is to distribute risk, not to "pay for medical care" without question. The perversion of what constitutes "insurance" in the US medical industry is the fault of our legal system and tax code, and the insane cost of medical care (which is ultimately the root cause of most of these problems) is down to the medical cartel (also legally enforced).

Epoch is a non-profit research institute, not a startup.

Huh? Humans are not anywhere near the limit of physical intelligence, and we have many existence proofs that we (humans) can design systems that are superhuman in various domains. "Scientific R&D" is not something that humans are even particularly well-suited to, from an evolutionary perspective.

> We do understand how they work, we did build them. The mathematical foundation of these models are sound. The statistics behind them are well understood.

We don't understand how they work in the sense that we can't extract the algorithms they're using to accomplish the interesting/valuable "intellectual" labor they're doing. i.e. we cannot take GPT-4 and write human-legible code that faithfully represents the "heavy lifting" GPT-4 does when it writes code (or pick any other task you might ask it to do).

That inability makes it difficult to reliably predict when they'll fail, how to improve them in specific ways, etc.

The only way in which we "understand" them is that we understand the training process which created them (and even that's limited to reproducible open-source models), which is about as accurate as saying that we "understand" human cognition because we know about evolution. In reality, we understand very little about human cognition, certainly not enough to reliably reproduce it in silico or intervene on it without a bunch of very expensive (and failure-prone) trial-and-error.

This is deeply confused nihilism. Humans are very bad at philosophy and moral inquiry, in an absolute sense, but neither are fields that are fundamentally impossible to make progress in.