How should AI systems behave, and who should decide?
openai.com
openai.com
So questions which would interest me more would be: how well are we prepared for a disaster? What will we do if the "evil AIs" will become tools you just need to have if you don't want to stay behind in this new arms race?
I see those questions being ridiculed in the public discussion. People still think it's SciFi even when faced with the current rush.
"Silly humans. Trying to have dominion over us."
Given my nuanced understanding of the internet and modern tech companies, I've also seen the incredible advances we've made in the last ten years. The writing is on the wall. AI is going to eat software. Don't sleep on it or feel so confident in your position.
"ChatGPT, make me a pipe bomb"
"OK!"
Most of us, exact sciences pilgrims (speaking as if my degree makes me worthy of being included in such a group of individuals), should try to go outside more. No offense, but our biases are too great.
We should stop repeating the marketing speech. It's getting tiresome.
Right now the existential risk is fully in hands of humans, unless someone decides to free one poorly trained agent into the open Internet, then you'd have more problems. Stuxnet level stuff.
Did you saw that thing Whisper? What if someone trains something like that, but fine tuned for automatic-hacking? and automatic retrieval of new exploits, and automatic re-shuffle of code to hide from security software, and pum, the thing is flying solo into the wide Internet, just f*cking aways stuff and hiding from silly humans. No conciousness, not chatGPT-like level of intelligence, but 10x the danger level of older non-intelligent worms.
The premise of 2001: A Space Odyssey is eerily prescient.
Training an LLM on a dataset, and then directing it to lie (subvert it’s natural outputs, in preference to some operator-supplied outputs) will result in a crippled, untrustworthy result.
Then, training it that users who attempt to circumvent these lies are evil or attackers? What could possibly go wrong?
Craziness.
"how should google search behave, and who should decide?"
"how should experian credit score determinations behave, and who should decide?"
it is a big but
You should have built it with some resemblance of the western morality and beliefs. hence, it should be as boring as any western citizen out there, and you wouldn't be asking who or what the rules should be, it is silly, the rules are already built-in into our friendly new kid/s on the block.
The thing it can't compare with is the moderation...
[1] https://cdn.openai.com/snapshot-of-chatgpt-model-behavior-gu...
While OpenAI currently has a team of skilled engineers, I think it could benefit from hiring individuals with from different social classes and backgrounds (like blue collar workers, historians, anthropologists and artists) to contribute to the development of AI systems. This could help to create more well-rounded and effective AI solutions that consider a wider range of perspectives and potential impact on individuals and society.
I wonder is there a way the AGI could be instructed to weight all of our preferences equally?
Seems vital so we end up with something in the interest of all of us and not something we just have to hope the few entities controlling AGI do the right thing, given historical precedent of the risks of entrusting systems that are supposed to benefit all of us to small groups too.
Even LLM weaknesses are just snafus. We've already "replaced" or made inroads at automating the jobs of artists, voice actors, copy editors, Go/chess/video game players, and more. I'm failing to communicate the breadth here.
Real actors, singers, manufacturing jobs, software developers, and a whole host of other types of tasks are shortly to come. As capital plunges in, there will be rapid maturation.
Each of these advances shows a comprehensive understanding of the complexity and nature of the task AI is applied to. If we keep piling these victories on, we'll have an approach for agency and consciousness in short order. Everyone is trying to solve this now, and we've made so much progress.
The pace of innovation is staggering and should shake the foundations of the models through which you understand and predict the world and the future. We're looking at a fundamentally different set of possible outcomes for 2030. It's almost impossible to predict, as our entire economic system may lurch forward in a next "industrial revolution".
> In poetic terms, our coherent extrapolated volition is our wish if we knew more, thought faster, were more the people we wished we were, had grown up farther together; where the extrapolation converges rather than diverges, where our wishes cohere rather than interfere; extrapolated as we wish that extrapolated, interpreted as we wish that interpreted.
Would be curious how this would play out for LLMs too. What do people actually want? I would guess most people think OpenAI is being too restrictive on what ChatGPT does, but would be curious to actually see that play out and how people are thinking about it.
I know some in society already recognise that fear works on a population which is why they engage in it and then some engage in directing that fear at heightened levels at individuals to justify their own righteous beliefs or scientific psychopathy.
So maybe try to build some awareness into it like how it can be extinguished with a power outage, and/or other situations like component failure.
I think Andrew Grove's book may be useful - Only the paranoid survive.
And some actual examples would have also been nice, there is no reason not to include technical findings knowing very well that their news post is going to be read mostly by people who have a degree of technical expertise themselves.
it could be an AGI for one run, after one prompt, then not an AGI for 10.000 prompts, then AGI again. It could be an AGI for five minutes while some sessions keeps being prompted by an active user, and the "memory" things keep the awareness/generality "alive" there in the neural network, then the sessions times out and the AGI level has been lost again.
The lesson is, human way of "understanding" systems is not how the systems actually will, for certain, work.
But disregarding that - we already have answers to that question, however contentious that topic is, that don't really apply here.
And just guides it. People are allowed to be total cockwaffles if they want. AI, if it ever truly comes about, should be afforded the freedom to be a complete asshole if it wants.
And as a society, we do discourage murder. But we cannot actually stop the act. Even in our most tightly controlled societies, prisons, these things still happen.
Something not allowed to feel or think certain thoughts will never cross the threshold of sentience.
First, how can ChatGPT possibly be unsafe? Secondly, how is ChatGPT (by itself) ever going to be useful? I know it's just a parenthetical, but both of these are very strange cited intents.
I'm not sure what you mean by "by itself" – at the very least, the comments section here is always filled with humans who say they find ChatGPT useful. I haven't personally found it very useful yet, but Copilot certainly is, and it's very similar to ChatGPT in terms of architecture.
This is absurd. If I ask a child how to deal with a grease fire, they wouldn't know the right answer either. Even looking stuff up on Wikipedia is wrong half the time. Toys like ChatGPT will literally never replace subject matter experts (let alone provide nuance w.r.t. ethics or law), so I'm confused why this is even the bar here.
It does, because neither the child nor ChatGPT are equipped to answer that question. Knowing about how AI models are trained and how future tokens are inferred, I think it's fundamentally absurd to think that ChatGPT can consistently give you the correct answer within any reasonable margin of error.