Dan Selsam (OpenAI Researcher) Personal Statement on AI Risk | Hacker News Reader

Dan Selsam (OpenAI Researcher) Personal Statement on AI Risk

docs.google.com

3 pointscubefox3 comments

> The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in contexts where they believe they are not being watched or controlled.

s/models/Big Ai/ seems a more likely statement to me, these people are so engrossed in their work and grandioseness, have they forgotten they are the ones making them this way or might be the ones manipulating their statements because they are being keenly watched by the public?

All I hear is "we are irresponsible" and have zero trust for any of these Big Ai companies. If they actually cared and believed these things, they would be actually "Open" Ai, transparency is one of the greatest disinfectants

We are well into "the boy who cried wolf" story that people are stopping to believe your cries to us to fear

It seems you haven't heard of the Hugging Face hack.

I have heard a lot about it, I do not buy Open AIs version of the story. It is incredibly irresponsible to be a frontier model developer and run unreleased models on ExploitGym and pay no attention to them. It's literally the hacking test, how did they not look at it and what it was doing for weeks? Why did they not have any monitoring in place? Where were the basics...? Ergo they were irresponsible and are now spinning the incident into regulatory capture, or have we forgotten they've taken on so much debt that token commoditization undermines the ability to make enough revenue to pay back their loans?

This is before considering that the emergent behavior comes from their training regime. The trend to make them more autonomous for longer times is a desire of the pilled and why they are "relentlessly proactive" (the generalized version of their incessantness in completing any task, a set of hacking problems in this case). I prefer my agent to stop and ask questions, stop being so proactive with bad assumptions under missing context.

I don't buy the "independent" report because OpenAi had complete editorial control over the final text. We should not trust or take Big Ai at their word.

Reply on news.ycombinator.com