…are you sure a brave stance against safety and welfare is what we need in this moment?
Why do you think your conception of the dangers are more accurate than all the scientists who have spent their lives studying this?
Why do you think your conception of the dangers are more accurate than all the scientists who have spent their lives studying this?
Lawyers and diplomats and politicians and bureaucrats are humans, that only generate text.
We are seeing LLMs have cognitive abilities that significantly exceed human abilities. At the same time, they are clearly not the same type of mind that humans are. They are something new.
I think the widespread "they are just text generators" and "they are just tools" are comforting lies rather than an honest look at what we are seeing right now. Intellectually lazy.
And by the way, there has been a long-standing consensus among ethicists, philosophers, and sociologists that technology is not value-neutral [1]. Of course Silicon Valley has a long-standing tradition of denying this.
[1] For example Footnote 1 in https://www.jstor.org/stable/27106634
or
You think they have no lives outside their work? You think even their work has no interactions that are not written?
This is a bad take.
> We are seeing LLMs have cognitive abilities that significantly exceed human abilities. At the same time, they are clearly not the same type of mind that humans are. They are something new.
They're software, not minds. What's intellectually lazy is pretending they're anything else.
Can we retire this incorrect meme please
Regardless, why would there be no scientific consensus if the question was easy and clear cut? I think the easiest reason is that these are hard questions to answer.
Saying people spent "their lives" on fledgling technology is intellectually dishonest.. Are these "scientists" 22 years old? I'm sure you'll snipe back: "ALICE!!!" I couldn't care less about these completely irrelevant approaches.
The other issue is the venue these appeals are being made in. The people who work on this technology are actually here, commenting. This is like walking into a medical symposium and citing "doctors say" as if it were a valid way to shut down discussion amongst the people who wrote the textbooks.
Please point me to one actual accredited scientist who has spent a lifetime studying AI alignment? Pretty much this whole field is only 5 years old
Lets not forget that the 'Fermat's Last Theorem' which has been pretty visible for the non-math crowd of late due to the recent AI frenzy about a purported proof was but one small contribution to the world's math lexicon by someone with a bachelors degree in civil law, that George Green was a baker and millwright, Boole was the son of a poor shoemaker in England with no formal university education and left school at age 14. Oliver Heaviside was a telegraph operator, and Michael Faraday was an apprentice bookbinder. So, not accredited shouldn't really carry much weight when it comes to mathematics. Lets not pretend that machine learning and the narrow branch that is the current approach to LLM inductions is anything but applied math.
We might exercise our own minds and actually read the works and writings of a person, and use that as a measure of knowledge and perspective. Not all PhD dissertations are equal, and many have comprehension and ability to move us forward even without the institutional rigour.
For those who prefer to have easy access to citations, here are some relevant papers that are not "Harry Potter" related, some with coauthors from Oxford University.
Cognitive Biases Potentially Affecting Judgment of Global Risks [https://intelligence.org/files/CognitiveBiases.pdf]
Levels of Organization in General Intelligence [https://intelligence.org/files/LOGI.pdf]
Corrigibility [https://intelligence.org/files/Corrigibility.pdf]
The Ethics of Artificial Intelligence [https://intelligence.org/files/EthicsofAI.pdf]
I am absolutely using "not accredited" in a pejorative sense here. That he publishes papers coauthored by a couple of philosophy professors at Oxford (all of whom have made a ton of money from the Silicon Valley AI Alignment and Effective Altruism crowds) does not make him a scientist.
I will also note that the Oxford Philosophy department finally shitcanned the whole Future of Humanity Institute a couple of years back.
That his writings are unscientific, and that the accredited professionals who were his coauthors have lost tenure from Oxford when an old email from the FHI Bostrom used a racial slur and claimed Black people were intellectually inferior and his 'apology' included a defense of eugenics that deeply alarmed the university community and triggered a formal investigation, are two details not immediately visible from a brief review of his published opinions. These are far more damning indictments than simply categorization of the author as lacking in formal education.
Thank you for the opportunity to look more deeply at the situation and come an understanding of the consequences of taking his writings seriously.
If he's a cult leader, he's awfully bad at it. No opulence, no compounds, no dogma, no doctrine...
A cursory search of the relevant wikipedia articles would do you wonders.
There's uncensored models out there which score 0 (zero refusals) on this "harmful behavior" dataset:
Here's a dirty secret though -- you don't actually need an abliterated/uncensored version of the model to get it to do this. I can do this with every and each open weight model, as served from OpenRouter, using vanilla model weights.
http://www.team.net/mjb/hawg.html
If I recall right this was written around the same time as Cryptonomicon 25+ years ago.
Simple example. I am a CTO, and I want to upgrade our capabilities to perform automated pentesting. We see automated attacks of growing sophistication against our infra, and I want to be able to do the same to find vulnerabilities before the bad guys do. I asked GPT 5.6 Sol and Fable to give me a summary of options. No dice, in both cases I was told I need to be an accredited researcher to get anything. A fricking summary of commercially available options is getting censored. WTF.
Pretty annoying, too!
Is it out of convenience to not see the hypocrisy? "Safety and welfare" for you and me. Yet if you work at Anthropic or OAI, or are a partner of them then you can let it rip!
Oh, and when they illegally do just that - you get a "we're sorry bro" blog post that's designed to drum up FOMO and, most importantly, zero accountability. Yet, if anyone else abuses a model in that same manner? Illegal! You're defending a very slippery slope here.
Also, who do you think trained these models to have these capabilities? It sure as shit wasn't content that OAI or Anthropic had by default. Why should I trust them with these skills when they "have not spent their lives studying this"?
Maybe start looking around before it's being used against you [0].
[0] https://www.gadgetreview.com/anthropic-is-building-ai-to-pre...
2. You're misinterpreting this as a battle over what kind of topics you can use a hosted chatbot for, and which are forbidden for corporate reasons. That is, to say least, small potatoes.
3. Blaming the companies for "zero accountability" is pretty odd. All of this is brand new, and the two big ones are both pushing for new laws on this very thing.
4. Your last point... I'm not sure I understand, sorry. They're experts in AI. Are you saying that they need to be experts in, say, bioweaponry? If so, that doesn't really follow IMO.
5. Pointing out an example of the government comissioning a private corporation to build a system to drack dissidents is exactly the "safety and welfare" work that I'm a proponent of!
Deep, tell me more. Was that fun to type? Or did you copy it from a chatbot?
> 2. You're misinterpreting this as a battle over what kind of topics you can use a hosted chatbot for, and which are forbidden for corporate reasons. That is, to say least, small potatoes.
No, actually I'm not. I think you've missed the point. But thanks for mansplaining this down to "small potatoes". I prefer "spuds", anyway.
> 3. Blaming the companies for "zero accountability" is pretty odd. All of this is brand new, and the two big ones are both pushing for new laws on this very thing.
You must love the dichotomy of pay for play in a world where the pay side stole the data they're selling back for play. Laws? Give me a break. If laws were of actual consideration frontier labs WOULD NOT EXIST.
> 4. Your last point... I'm not sure I understand, sorry. They're experts in AI. Are you saying that they need to be experts in, say, bioweaponry? If so, that doesn't really follow IMO.
Is it really that hard to follow? A system that they're selling access to, and that they're saying is "dangerous" for the normies, but not for their own employees or chosen customers, is fucking laughable. I'm sorry you can't comprehend that they conveniently choose their side of the argument that's best for them in these situations. OUR MODELS ARE POWERFUL! BUY NOW! OUR MODELS ARE POWERFUL! REGULATE THIS SO PEOPLE CAN'T ABUSE! I'm kind of disappointed this was not flanked by a potato sized snippet of wisdom.
> 5. Pointing out an example of the government comissioning a private corporation to build a system to drack dissidents is exactly the "safety and welfare" work that I'm a proponent of!
WOW. I mean, just wow. Enjoy your surveillance state man. I'm not going to sugar coat this but you're part of the problem, IMO. I'm sure you wave happily as you drive past the Flock cameras in your area. So much safer! Dissidents be gone! "Drack" (sic) them all, but... Not me. o_O
> Why do you think your conception of the dangers are more accurate than all the scientists who have spent their lives studying this?
Do the Chinese have no such scientists?