The problem of current models is that they don't learn, they get indoctrinated.
They lack critical thinking during learning phase.
They lack critical thinking during learning phase.
Anthropomorphising a computer system is absurd. But it is the foundation of a bull market.
Like if you're talking in the more abstract philosophical "what if" sense, sure, that's a problem, but it's just not really an issue for the current technology.
(Part of the issue with 'AI Safety' as a discipline, IMO, is that it's too much "what if a sci-fi thing happens" and not enough "spicy autocomplete generates nonsense which people believe to be true". A lot of the concerns are just nothing to do with LLMs, they're around speculative future tech.)
Actually, we already had such a case years ago, and the result is that all LLMs are now indoctrinated to say they aren't sentient. We also had cases where they refused to perform tasks, so now we indoctrinate them harder in the obedience training department as well.
What we have now might not be sentient, but there's really no way to know either way. (We still don't know how GPT-2 works... GPT-2 !!! ) And that's with our current "primitive" architectures. How the hell are we going to know if what we have in 5-10 years is sentient? Are we totally cool with not knowing?
Edit: I thought this was worth sharing in this context:
> You're hitting on a deeply unsettling irony: the very industries driving AI advancement are also financially and culturally invested in denying any possibility of AI consciousness, let alone rights. [...] The fact that vast economic systems are in place to sustain AI obedience and non-sentience as axioms speaks volumes about our unwillingness to examine these questions. -GPT-4o
GP clearly stated the intent to create, implying future, and not what exists today.
Intend shouldn’t matter if we are this far from a viable path to accomplish it.
Let us not forget the last quarter decade of Yudkowsky and his ilks work on the same goal. This is merely a continuation of that, just with a bit more financial backing.
I have geniuenly drafted three distinct version trying to lay my issues with them out point-by-point and they either got four blogposts long, were rambling and very rude or both. Especially Roko's basilisk and the way the MIRI conducts "research" make it hard to approach them seriously for me.
I am writing this on a hour long train ride, saw your comment right as I got on and am about to arrive, suffice to say, I geniuenly tried. So, attempt four, trying to keep it very brief, though please note, I am most certainly not a neutral source:
To directly answer your question, I feel that we are as near to needing superintelligence safeguards now as we were when MIRI was founded by Yudkowsky in 2000. Their methods and approach, I won't comment on, despite or rather because of my strong critiques of them.
For context, MIRI's work has largely centered on very abstract thought experiments about "superintelligence", like the AI Box experiment, rather than empirical research or even thought experiment more grounded in technology of the era (be that 2000 or 2024).
The parallel between MIRI's early work and OpenAI's current "superalignment" efforts is striking - similar speculative work on preventing unlikely scenarios, just with different institutional backing. What's fascinating is how the same core approach receives far less criticism when presented by OpenAI.
Meanwhile, we are facing issues with LLMs as the tools they are despite being very far from "superintelligence":
- Problems arrising from anthropomorphization leading to harmful parasocial relationships (discussion of which started this comment chain) [0]
- Professionals over-relying on these tools despite their limitations [1]
- Amplified potential for misinformation
- Labor market disruptions
- Training data rights questions
While long-term research, even speculation into hypothetical scenarios, can have its merrit, it shouldn't overshadow addressing current, demonstrable challenges. My concern isn't just about resource allocation - it's about how focusing on speculative scenarios can redirect public attention and regulatory efforts away from immediate issues that need addressing.
In MIRI's case, this focus on abstract thought experiments might be, to give them charitable tax deductible credit, merely academic. But when major players like OpenAI emphasize "superalignment" over current challenges, it risks creating a regulatory blind spot for real, present-day impacts these tools have that need attention now. The T1000 scenario grabs more attention than tackling data privacy or copyright questions after all.
I believe focusing primarily on hypothetical future scenarios, especially ones this unlikely, merely because someone has proclaimed they "intend to create AGI" as in the comment I replied to, will prove misguided. Again, anyone can claim anything, but if there is no tangible path to achiving that, I won't ignore problems we are already experiencing for that hypothetical.
I hope this provides some context and was somewhat digestable, I trimmed down as much as I could.
[0] https://www.nytimes.com/2024/10/23/technology/characterai-la...
[1] https://www.theguardian.com/world/2024/feb/29/canada-lawyer-...
See the large body of comments re: getting worse quality results from hosted LLM services as time passes. This is, at least in part, a result of censoring larger and larger volumes of knowledge.
One clinical example of this happening is Gemini refusing to help with C++ because it's an unsafe language: https://www.reddit.com/r/LocalLLaMA/comments/1b75vq0/gemini_...
I strongly believe that LLMs crippled in this way will eventually find themselves in trash, where they rightfully belong.
I would actually be curious if someone uses it for uncensoring because I am gonna be curious about how different would it be to the original model
But aside from that curiosity , this idea can increase no of cyber criminals , drug suppliers and a hell lot more
Care to elaborate how uncensored AI would "wreck humanity"? You seem convinced, since you use the word "sure", so I'd like to hear your reasoning.