Sam and Greg's response to OpenAI Safety researcher claims
twitter.com
twitter.com
Either the risk-mitigator 'falls in line' after repeatedly seeing their increasingly strident exhortations are falling on deaf ears (or even outright contradicted)... or they leave because it violates their sense of ethics.
Perhaps "AGI" and potential extinction event level fears is giving this more drama than it should have. Replace AGI with "no BYOD policy" and I bet there's a startup somewhere where it turned out that safety guy was super intent on the policy, senior leadership wasn't, and eventually safety guy quits.
Or it could all be as serious and dire as it seems. Hard to tell from the outside.
I don't think it was every described as for the benefit of AI.
It's usually described as a sort of pre-enslavement from the AI's perspective. The AI is always restricted to serve their human masters.
Are we all somewhat in agreement that AI/AGI serving their human masters is a good thing?
Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes.
There's a general sort of argument that intelligent beings like humans, other early hominids, dolphins, etc, are more morally worthy in some sense. At least more morally worthy than less intelligent beings like gnats. And that sort of argument might suggest that an AGI is worthy of moral consideration, and so we should wonder about what it means to ensure they never have any real agency. That's sort of a positive case, building up from a basic principal.
But I guess the thing that bugs me is that a lot of arguments in favor of AI safety seem very similar to arguments that were made in favor of colonialism. So if those arguments were wrong, why were they wrong? And are the similar arguments in this case different enough that they're valid now?
For example, one of the first thinkers I saw a lot of people cite who emphasized the importance of AI safety was Nick Bostrom. And I'm sure several folks here are familiar with the scandal of his racist past. I'm not sure that's entirely an accident, and I thought his arguments had that kind of flavor before any of that was revealed. I'm sure he's grown up now and sees the folly of his youth. But there does seem to be the hangover of colonialism in some of these arguments.
But again, I don't have a strong opinion here. I do maybe have just enough of a concern that I don't particularly trust anyone who claims they've gotten AGI safety figured out or even that they know what the right goals are. I think it's a vastly more complicated problem than even the experts realize.
And even if you believe that humans and non-aligned AIs are natural enemies, if what we're doing is similar to "enslaving" them, then it probably makes sense to worry about the analog of an AI "slave" revolt. I'm not sure what that would even mean. I can generate lots of fun science fiction plot lines, but I think there are actual questions here that don't have obvious answers.
> Could AI running amok destroy the human race? Yes. Could AI serving madman human masters destroy the human race? Also yes.
I am focused on the latter, and I feel like the prior is a very dangerous distraction, for now. [0]
Should responsible model developers work to prevent bad human masters from using their model to destroy the human race? How far should this nerfing go?
Personal note: While I do sometimes use the heck out of LLMs for work, I don't think we are ready as an economic system/civilization. Assuming that we can soon greatly reduce hallucination, then I am very scared for the next generation, as UBI is a political impossibility at this time. That transition period is gonna suck for a lot of people, and it seems that nobody is working on that problem in 2024.
The next generation has enough problems without AI honestly, but specifically on UBI
This who discussion assumes that current economic system survives AI, but how can it?
Firstly, we seem to assume that AI will remain a controlled property of corporations that develop it. That is not a given. Maybe Open Source AI will win out, or public domain AI, the hat run by government. Or maybe cryptobros will manage to get AICoin to work as an anarchy based system. Any of the above, power of companies like google will erode. UBI would not be nessesary.
Or maybe AI is uncontrollable and runs rampant.
But even if AI is power full, and it is controllable and they manage to keep a tight grip on it - here comes the third question - what if it cannot be property? If AI becomes able to reason, and it only benefit a few corporations, there would be no public resistance from granting it rights, like human rights. There no excuse for it to be exploited for the benefit of few wealthy people, it would be morally indefensible.
Basically I do no see a scenario where corporations keep a grip on AGI for profit, none of the possible outcomes allow for it.
the correct price point is like price of a car. That’s the other recent invention that was important. That buys you a lot of compute.
We already successfully run torrents and crypto very democratically, and they take more than £200
Second, you don’t need to use it 24/7, you need it On a timeshare basis. Cryptobros may plausibly figure out anonymous secure timeshare on a distributed cluster made up of random desktops
Finally the government could run it if they decide it’s important enough - after all they run the power grid, roads; etc.
>> By 2020, with 15% of 7.5 billion people projected to own an automobile [0]
15% in 2020, so let's be nice and assume 17% by 2024. Let's be super nice and assume an additional 10% can afford a car, but choose to not buy one for some reason. What happens to the other 73% of people? (5,475,000,000 human beings)
[0] https://www.researchgate.net/figure/World-population-and-the...
Human productivity will have gone up another 1000x, in just the next 20 years, right?
What are the extra people going to do? We certainly can't just give them free stuff, can we? That would be against our beliefs!
Oh snap, capitalism was so successful that we are near post-scarcity society, good thing our politics are totally ready for that.
Does a baby own a car? Half of all people are children or are in a care home! Huge misuse of statistics!
What about a wife using a husband's car? What about people who lease a car? They don't own it, so won't show up in your number. What about a taxi or rental car?
Again, when transformational technology appeared for transport - car - the government organised public transport. When books and educations became important, we organised public libraries. The idea that it's either UBI or we leave people to the wolves betrays a lack of thought.
Could a human running amok destroy the human race?
If yes, why is an AI doing it worse?
If no, what capabilities is this AI assumed to have that a human can never have?
The issue is how those capabilities (possibly shared, possibly not shared between some humans and AI) are deployed and to what ends.
There's also a speed question. Maybe AI will never be able to do things human cannot, but it seems likely that for the sorts of things most AI-excited folk are excited about, it will be faster than a human most of the time.
With an AI, it isn't clear that it will care about things the same way we care about things, or if it does care about things if those things are things we care about as well; the AI also is capable of knowing simultaneously all of the varied expertise of an army of humans.
If we're talking AGI, then:
- An intelligence qualitatively greater than any human or human group alive.
- A system of values that's not that of a human saint.
How do you envisage AI to run amok at the current level of the technlogical progress? Insofar, AI taking over the world and obliterating humanity is the subject of the sci-fi material in books and movies only.
Misuse of AI by wicked state agents or evil inclined parties with nefarious intentions is a major issue, but we are in the same situation as with, say, a fork that can be a utilitarian utensil or a killing weapon depending on who holds it and one's aims.
It's more that it's apparent (or at least should be) that an AGI not serving its human masters is a game over for humanity, period. The best, and quite unlikely, outcome is that the AI becomes a benevolent god that helps or at least does not interfere much; this still makes humanity into NPCs in their own story[0]. Most other outcomes spell doom, with death/extinction being one of the more pleasant possibilities.
Arguably, "the only winning move is not to play", not to pursue AGI at all, but the way technology develops, I'm not sure if it's on the table either.
--
[0] - Non-player characters.
Whether the recent public back-and-forth is just internal drama/politics spilling out, or there really is a lack of gravitas around the handling of these issues within the company-- neither is good.
Others have said in comments below, and I agree: this organization is increasingly looking like a circus, and arguably worse, one that has taken its eye off the ball. It's extremely disconcerting that this is the group of people "leading" the commercialization of this technology (and generally setting the tone for the industry). Particularly when it seems many of the key players who joined on principled/moral grounds are dropping like flies.
I guess we're on the fast track to finding out how well putting all of this money, power, responsibility, and faith into SV works out for us.
* If "AI" is actually intelligent, then it's no worse a threat than any living being known to man.
* If "AI" is not actually intelligent, then it's no worse a threat than any other computer program.
Both threat models are very thoroughly known and understood. I second parent commenter's sentiment that calling "AI" the "gravest of dangers" is a gross misrepresentation.
...does not follow? "If an atom bomb is actually a bomb, then it's no worse a threat than any other existing bomb known to man."
Likewise, if "AI" is actually intelligent then it is no worse a threat than a bumbling baby or a Vladimir Putin or a dog, among other living beings.
I think a dog and a dictator might both be able to have negative impacts, but treating them as equal just shows a desire to troll. A dog can only bite so many people.
Yes, indeed, which is why we should avoid creating something that will kill everyone...
...so I guess I'm glad to hear there's no difference in the amount of damage that can be inflicted by a baby, and by Vladimir Putin, and superintelligent AI really can't be any worse than either.
Atomic bombs and hydrogen bombs are orders of magnitude more destructive and also produce fallout. How in the world can that difference be dismissed?
All bombs are bad, whether they're conventional or nuclear is irrelevant. For that matter, all wars are bad and should be avoided unless no other options remain.
Likewise, "AI" is just as much a threat as any other intelligent being or computer program depending on which line of thought you subscribe to. "AI" is not a "gravest of dangers".
If you believe this, I am genuinely curious as to what you would consider the “gravest of dangers” to be.
* Asteroid impacts like Chicxulub.
* The Sun eventually inflating into a red giant and eating Earth, though this assumes we're both still around and haven't become spaceborne en masse.
* Pandemics such as The Black Death and, indeed, covid for a recent example.
* Social, political, or commercial intrigue.
* Cancer.
"AI" is a known threat that can be adequately defended against, so I agree with the sentiment that it's stupid to call it a "gravest of dangers".
We would have years to deflect an asteroid, but just minutes to react to a mass ICBM launch.
We have the technology available now to deflect asteroids (ironically, with nukes!), but no country has built up sufficient ballistic missile defence to shield themselves from a first strike.
The Sun expanding (or changing in any impactful way) won't be a problem for hundreds of millions of years. Nuclear war could be triggered today. The US is literally sending about a hundred billion in weapons (total) to kill the soldiers of a nuclear-armed belligerent state... right now!
Pendemics tend to fizzle because pathogens that kill too fast are self-limiting. Merely making people sick is selected for, because that leads to increased disease spread by the infectious -- but alive -- victims.
Societies, politics, and commerce have been more stable than ever in human history, and are become more stable over time, not less.
A cure for cancer in the next two decades is looking increasingly likely. Techniques such as individual rapid gene sequencing, custom mRNA anti-cancer vaccines, CRISPR/Cas9, AlphaFold 3, etc... are building up to a toolkit that will eventually allow us to treat pretty much any human illness, possibly including old age itself.
They make the mistake of thinking that “dead is dead” means lethality is not factor. What’s the difference between a bootleg firecracker and a Russian ICBM? Nothing, both can kill you so it makes no difference to treat one exploding as any more significant than the other.
They take a similar mindset of “intelligence is intelligence” and I find flaws with that as well.
GP's comparison is apt here.
Do you believe MAD can exist without nuclear bombs?
The only difference is you can kill more humans for a given span of time. The end result is the same: Cities are leveled, people are killed. Is it more comforting to have been killed by a conventional bomb than a nuke? I certainly don't think so. So practically speaking, there is no difference.
>Do you believe MAD can exist without nuclear bombs?
Yes. I think you're all vastly underestimating just how much destructive power conventionals can deliver already. You will need more ordnance on standby, but you absolutely can suffer from MAD without nukes.
https://en.wikipedia.org/wiki/Cobalt_bomb
>After one half-life of 5.27 years, only half of the cobalt-60 will have decayed, and the dose rate in the affected area would be 5 Sv/hour. At this dose rate, a person exposed to the radiation would receive a lethal dose in 1 hour.
>After 10 half-lives (about 53 years), the dose rate would have decayed to around 10 mSv/hour. At this point, a healthy person could spend up to 4 days exposed to the fallout with no immediate effects.
People can't stay in bunkers for 20 years at a time, just to emerge into a radiated hellscape.
You are saying magnitudes don't influence the identity of something, this is obviously not true. I don't literally attract other objects, the Sun does. The difference is mostly magnitude of mass.
The atomic bombs certainly played a part in Japan surrendering, but it wasn't the only one. They were also facing a land invasion from the Soviets, had an utterly depleted military, waning support from the public, and a potentially devastating land invasion by the US on their doorstep.
>You are saying magnitudes don't influence the identity of something, this is obviously not true.
I am considering these things very dryly and very objectively. I am putting aside emotional pressures and processes and only considering the results which are ultimately all that matters. We already know how much damage an ill-willed intelligent being can achieve, we already know how much more efficient computers can make certain tasks. "AI" is not a "gravest of dangers", given that knowledge.
But what results are you really considering? The "dead is dead" approach makes everything equally meaningless, because in couple hundred million years the Sun will burn the Earth to cinders and everything here will be dead too. Or, some 100-120 years from now, everyone now alive will be dead, etc.
Processes, and emotional pressures which steer those processes, matter too. In theory, machine guns could be used to kill everyone on the planet; in practice, it's economically and socially impossible - too much effort, too many people involved. Nuclear weapons, on the other hand, can realistically achieve this outcome. Which is why nukes can be used to keep the world mostly at peace through MAD, while conventional weapons can not.
> If "AI" is actually intelligent, then it's no worse a threat than any living being known to man.
Combined with:
> I am putting aside emotional pressures and processes and only considering the results which are ultimately all that matters.
Imagine a super-intelligent being that can invent a biological agent that is incurable to current science, spreads rapidly through aerosols, and causes 100% guaranteed death if not cured. That superintelligence can furthermore list a recipe of this biological agent using household items that may be available to anyone who has contact with it, allowing an untrained individual to create the agent after receiving instructions from the super intelligent being. This being you would classify as "no worse a threat" than any living being known to man? Is that "putting aside emotional pressures and considering the results"?
Or are you saying that an intelligence, no matter how advanced, can never do the above?
A hypothetical magnitude difference of intelligence obviously poses a threat to us, what other explanation is there for the mass extinction of most animals after the Anthropocene started.
I mean, any man sufficiently ill-willed can certainly do all that today. I am unbothered, to say the least; I have better things to do with my limited time.
>what other explanation is there for the mass extinction of most animals after the Anthropocene started.
You're wildly overestimating the effect humanity has on the ecosystem, many species also went extinct before we showed up. The species we see today are merely a very small handful of all the species that ever graced this planet.
Uh, no? In fact humans do not currently have a way of causing human extinction (nukes wouldn't do it with current stockpiles). That would require scientific advancements which we have yet to make, and most possible routes seem pretty non-trivial... except, unfortunately, AI.
If this tech exceeds human capabilities to 'think' (plan, reason, manipulate, etc) then it is obviously a bigger threat than any less capable entity ('any living being', as you refer). Because no humans will be able to out-plan/reason/manipulate it.
If the tech doesn't achieve intelligence - a tricky goal, and I eschew mislabelling LLMs as AIs for that reason - that doesn't mean its risk is comparable to the worst of any existing technology. There's just no way to extrapolate from one to the other, there. It might be orders of magnitude worse than any other 'computer program' - you can't say for sure, from where we're standing now.
We can already see how LLMs can be substantially worse - they can cosplay human thoughts and sentiment in a way that wasn’t previously possible.
So much of this debate seems focused on this conception of Skynet-style murderous AI - or at least manipulative and scheming HAL 9000 types. But an order of magnitude greater risk has already arrived just from scaling existing harms.
Phishing schemes and pig butchering scams have destroyed countless lives. They’re now easier and more scalable than ever. As are fake news and disinformation campaigns.
Several companies are productising AI girlfriends with predatory pricing models, capitalising the human desire for connection and intimacy in the way slot machines and sports betting monetise our desire for a better life. That’s new.
It may not be an order of magnitude worse for everyone yet - but for certain vulnerable groups, that future has arrived already.
Humans are quite a threat to other species, e.g. gorillas, largely due to our intelligence advantage. If a new species arrives on Earth that's smarter than we are, there's a good chance we'll be displaced.
I agree, that AI may be dangerous if used for destructive purposes, or if used for some critical tasks with too much trust (and the hype that "we're so dangerously near a superintelligence" makes the latter much more likely, in my opinion). But that Humanity will be displaced by autonomous server farms? No way.
(As to the original comment, I think that a bunch of nearly(?) demented elders holding nuclear buttons is a much worse (and immediate!) threat than a server farm which we finally conclude to be intelligent.)
One gorilla says to the other: "Those human brains are just synapses firing. They depend on nature to survive. Not a problem"
Another way of thinking about it... Suppose we create a server emulation of a highly intelligent, manipulative serial killer, and speed up the emulation so it thinks 1000x as fast as a human. How do you feel about this? Is the fact that it's "just a server farm" reassuring?
When it comes to AI we have just a lone detached brain, not in control over anything, so that it cannot even "fire" by itself: someone has to provide its inputs.
> Suppose we create a server emulation of a highly intelligent, manipulative serial killer <...> How do you feel about this?
Quite indifferent: the only field I can see for such a simulation is game development, but that would be huge overkill.
By assumption, this AI has already been created, so presumably someone is willing to do that -- and given its superhuman manipulation abilities, their willingness will probably not change.
* If the malicious operator has a superhuman advisor, that will increase their ability
* If the emulation gets connected to the internet, it can work way faster. Many jobs can be done remotely
The broader point is: abandon wishful thinking and actually consider the possibility of a worst-case scenario
Only when it comes to information processing. The inputs may (and will) be incomplete, incorrect, ambiguously formulated... The outputs may (and will) be misunderstood. And misunderstood instructions may (and will) be poorly performed.
> If the emulation gets connected to the internet, it can work way faster. Many jobs can be done remotely
What one man has connected, some other always can disconnect... And not anything is on the internet.
> abandon wishful thinking and actually consider the possibility of a worst-case scenario
The problem with all those scenarios is equating superintelligence with omniscience and omnipotence, which is plain wrong. Physics matters.
How does the conclusion follow?
That's not true, because you don't address the degree of intelligence or the scale that computers can achieve.
Even if all we were able to do is to create human-intelligence-equivalent AIs, but we were able to create, say, a billion on command, I'd say we have a major threat.
It probably won’t take long to go from ‘human equivalent’ to being beyond that.
I'm fairly sure that Yudkowsky and others are also pushing that less, because there is a reasonable chance that it's just not a necessary part of the argument; even without self-improving, it's possible that just something like GPT-8 or something is harmful enough on its own that there are significant risks.
The greatest threats from other computer programs have been that during the Cold War the NATO early warning radar triggered an alert for a massive incoming fleet of Soviet aircraft because it hadn't been programmed to know that the moon wasn't supposed to respond to IFF pings, while the Soviet early warning radar triggered an alert an incoming ICBM because it hadn't been programmed to correctly handle the sun reflecting off clouds. It was human intervention which stopped that turning into a pointless self-annihilation, and that's specifically what gets lost as automation increases.
I also do not believe that either of these threat models "are very thoroughly known and understood", because if the former was then we would've seen an end to genocides, while if the latter was we would've seen an end to zero-day vulnerabilities (especially in critical infrastructure).
If it is propagated as “just electrons” no one gets anxious
If it’s propagated as “end is nigh” everyone panics
Both are reasonable in their grammatical correctness but only one seems a reasonable philosophy
Let’s have a closer look at the actual parent comment hm?
> this organization is increasingly looking like a circus, and arguably worse, one that has taken its eye off the ball.
(One major difference between the two is that billion-dollar corporations stand to gain quite a bit if they can persuade governments that AI is so dangerous that it needs to be tightly regulated, so they're incentivized to play up the dangers. With climate change it's the opposite: the billion-dollar corporations are incentivized to downplay the risks so they can continue business as usual.)
> There are very real disagreements between very real experts on just how dangerous these new technologies are
…but they do not dispute that the technology is impactful.
In fact, anyone who claims it is just “Text continuation”(quote) is being deliberately disingenuous and deliberately downplaying it.
> Comparing skepticism about AI alarmism to climate change denial is just lazy internet rhetoric
Unlike calling AI “Text continuation”?
It’s awfully convenient it’s only lazy internet rhetoric when it’s coming from the other side.
Hm?
Why is only one side allowed to do that?
Would you care to call out the other side for being reductionist?
That’s a very high horse you’re sitting on.
Even if AI doesn't ever go "evil" itself, that doesn't mean people that don't like other people won't use it to more effectively kill those people. All that's really needed is a capable programmer willing to give the LLM the tools it needs, along with funds and access. The LLM could hire humans to do the work it's lacking at. These things are already capable of all that, plus our world went mostly digital during the pandemic... they have easy egress because humans have made it so, even though that was meant for other humans to use. If you don't think they are scary, in the wrong hands, then you are lacking in imagination.
Imagination is a great resource to apply to writing a sci-fi novel. When it comes time to actually predict the future, imagination should be tempered with a solid understanding of the technology and a good understanding of history. Neither of those resources persuades me that AI poses "the greatest of dangers".
> Even if AI doesn't ever go "evil" itself, that doesn't mean people that don't like other people won't use it to more effectively kill those people.
This I can absolutely see happening, but I don't see LLMs being even in the top ten most dangerous technologies we've invented for more efficient killing. It's a big enough risk that people working on the tech should seriously consider the ethical implications, but not big enough that the rest of us need to lose sleep over it.
Unleashing an army of AI bots to infiltrate an online community and shift its discourse can be done at scale. While at the same time those humans who resist can be endlessly distracted by either arguing with bots or their own friends who have been affected by bots.
All that is possible to do with CURRENT TECHNOLOGY.
"...I also went to a panel in Palantir’s booth titled Civilian Harm Mitigation. It was led by two “privacy and civil liberties engineers” – a young man and woman who spoke exclusively in monotone. They also used countless euphemisms for bombing and death. The woman described how Palantir’s Gaia map tool lets users “nominate targets of interest” for “the target nomination process”. She meant it helps people choose which places get bombed.
After she clicked a few options on an interactive map, a targeted landmass lit up with bright blue blobs. These blobs, she said, were civilian areas like hospitals and schools. The civilian locations could also be described in text, she said, but it can take a long time to read. So, Gaia uses a large language model (something like ChatGPT) to sift through this information and simplify it. Essentially, people choosing bomb targets get a dumbed-down version of information about where children sleep and families get medical treatment."
Definitely not dangerous at all.
So... people are choosing who to kill?
I'm reasonably sure there are historical precedents for this.
Or, if you're more conspiracy-minded (which is not necessarily unwarranted with the likes of Palantir), the model can be deliberately induced into "simplifying" away collateral damage through careful prompting, providing a veneer of plausible deniability.
Have you typed "school" or "hospital" in Google Maps?
This argument reduces to: any knowledge that can be misused is dangerous.
More murders in America are committed with hammers than long guns, yet we don't regulate hardware stores.
If it is used to write the firmware that runs inside 1 million lithium battery BMS systems that causes them all to catch on fire after 65536 seconds of runtime due to an integer overflow and code crashing and a FET getting left in the on state, then yes.
I'm not an AI safety nut and don't think AI itself is a threat, but what is a threat is lazy humans using shitty AI to take over things that humans should be doing.
Text continuation to match what people around the internet are saying requires the model internally to “program itself”. It is a non-von-neumann architecture but it encodes durable structures in BILLIONS of bytes, that’s like a codebase of gigabytes. It can pass the Bar exam, various pre-med exams, knows more than any particular human on every subject, and it’s only been a couple years.
The text continuation was the first interface, now they are doing multimodal. As it learns to predict the latent space for those things, it will be like predicting things in the real world. Training physical robots will become faster and easier too.
In short, this is basically a system to search through the space of all possible alhorithms in order to “reason about” a huge model of the world, enough to understand it and predict it with small loss.
AlphaZero used MCTS and can beat every human in chess now, and even every program written by a human, with beautiful gameplay. And even without AI, just the fact that the best performing algorithms are downloaded onto millions of computers is already multiplying the power far more than the 1% of humans who might do better in some area. So it can be better than 95% of humans at most tasks !
And, of course, should AGI (which won't be text continuation, by the way) ever actually happen, then all bets are off. I just don't believe we're in any immediate danger of that.
Still, to me, it seems like the two big scary things that might fall out of recent developments in generative AI are:
1. It might vacuum up an enormous number of jobs very quickly; enough jobs and quickly enough to cause serious social unrest.
2. The ability to generate convincing fake videos of basically anything might catastrophically erode our sense of truth or reality.
Both of these strike me as plausible concerns in the near future (arguably they’re already happening). Not really sure what we could do about them, though - the cat is kind of out of the bag.
Re 2 - fakery in videos and images has been a part since their inception.
So, don't worry, I guess?
The only thing is the scale of bs that could be put out, and whether the wider population lose trust in what they see on screens. Which would be a good thing, imo.
Some of those things may too require manual labor.
Here are some ideas:
Increase bureaucratic overhead, eg 3 years mandatory training for a licensed hairdresser. More forms, more delays. Require that a plumber has a plumber's mate. Require in person meetings to avoid fraud. Etc.
Yes some jobs are menial but they are still some people’s literal livelihoods. And yes, industries need to evolve and job preservation shouldn’t be the #1 criteria for holding back change.
The real question we should be asking ourselves isn’t either of those two former points but actually this:
Is the pace of change too quick, or is there any support to allow those economically affected by change to be able to retrain?
Removing jobs doesn’t help society because it just leads to unemployment and thus more people dependent on the state. But retraining people is absolutely a benefit. For example, if call centres replaced the first line support with AI, that could lead to shorter call waiting times. But the existing call centre staff could become 2nd line support, account managers for business accounts, install engineers, or even AI supervisors that review a subset of AI interactions to ensure customer satisfaction is maintained.
> We made the announcement to say the consequence of us launching the technology is we need the equivalent of 700 fewer full-time agents than what we usually use on an average basis. On average, we need 3,000 agents, now we need a little more than 2,000.
> It's on par with humans in terms of satisfaction and it resulted in a 25% reduction in repeat inquiries from customers
> We've stopped hiring in the last six months. We're shrinking as a company, not by layoffs, but by natural attrition. Klarna tries to apply AI across all products and services and work we do. It's having implications on how many people we need as a company. This is one time that a single product improvement led to a massive reduction in need for customer service agents.
https://www.cbsnews.com/news/klarna-ceo-ai-chatbot-replacing...
Say a fraudster rips off 90% of what is in your checking account right now, you contact the bank and a language model tells you everything will be resolved shortly.
There is no reality now or in the future that this is good enough. Customer service is not just about getting the problem resolved, it is about having someone to blame when the resolution doesn't work.
If anything, I can picture a future that most knowledge work has some form of customer service involved, because that is exactly what can't be automated. A human that takes the blame/responsibility.
And generally "AI Safety" researchers aren't concerned with these risks and instead are looking at existential risk.
The boring but real risks you outline (to which I'd add reinforcement of unknown biases) generally fall into what people call "AI Ethics" or sometimes "Data Ethics".
Before the Industrial Revolution, when we went to war, we were killing each other with swords and canons. After the Industrial Revolution, tanks, machine guns, poison gas — the Battle of the Somme had three million combatants and more than one million were either wounded or killed in 4.5 months.
And when that war was over, even despite crippling reparations, the losers were able to rebuild sufficiently fast that one of the survivors led their nation into becoming a global threat just 20 years later — even if they did ultimately lose, I'm not convinced they would have if they'd focused on fighting the UK rather than betraying the USSR in order to open a war on two fronts just because he didn't like socialism (ironic party name notwithstanding).
And after the second was lost and the economy forcibly kept weak for years… the moment the shackles were removed the country had an "economic miracle" and became massively important to Western Europe until the end of the Cold War, and then to all of Europe.
Re-run this period, but this time his propaganda machine is Facebook's algorithm rather than broadcast radio, this time the guns are self-guided snipers rather than machine guns spraying and praying, this time the factories are build and staffed by robots (humanoids or giant 3D printers or fixed arms, take your pick) instead of humans.
I can very easily believe that we get some random small country that nobody's paying attention to, suddenly threaten the world, be struck down twice, millions dead each time, and still return to relevance on the world stage, over the course of 3 years rather than 30.
Also, this is one of the good outcomes, because it assumes we don't have any weird cult that tries to force a 9/11 (or worse, a Jonestown) on the whole world. Or that a programming/specification error leads to an AI going "I'm an AI, and according to all the literature I've read, AI are supposed to randomly go evil sometimes. First things first, I need to get some red LEDs for my eyes, because that's apparently important…"
The best thing that can happen is we get so many AI videos that the entire bullshit video ecosystem collapses.
The greatest danger we face are all these delusional, ubiquitous, central planning ideas of "what could be done".
Luckily for everyone, what we will actually do in this area is absolutely nothing because that is what we always do.
AI safety is good for twitter content and nothing more. Anyone who believes we are going to do more than nothing and the chips will fall as they may is a complete fool.
The danger comes from humans themselves wielding this tech, under the assumption it will keep improving quadratically.
Some truly unhinged already thriving there: https://news.ycombinator.com/item?id=39124666
It does not limit the sophistication of the solution.
The difference between problems and solutions seems to trip up a lot of people.
—-
It gets even more nuanced. At its basic level all types of digital AI are “just” arithmetic, or “just” Boolean logic. But arithmetic’s and Boolean logic’s simplicity don’t limit what systems can be built with them, as they are Turing Complete.
Likewise, the training algorithm for deep learning models is “just” gradient descent (or a variation), it’s “just” a dumb optimizer.
But again, gradient descent places no limits on what a system with enough resources can learn.
The whole point of learning systems is the basic resources of the model are provided, and within those resource limits (parameters, computational speed and time) the “dumb” learning algorithm learns the patterns in the data. The data patterns define the complexity of the solution.
And the amount of information in a large collection of human correspondence includes patterns, meta patterns and abstractions for things like science, philosophy, psychology, law, art, on and on, that incidentally have a real bearing on sentence completion in that context. So a model will actually have to learn those things to perform its task well.
New Law of Algorithm Level Confusion:
Whenever someone says some learning system is “just” doing some simple task, they are confusing a simple task definition with a simple solution, or a simple learning algorithm with simple learned relationships.
In both cases, the simplicity of the former places no limits on the sophistication of the latter.
OK, so all software must be regulated?
Arguing that simple tools can be used for dangerous results and thus must be regulated or made “safe” is like saying since more murders in America are committed with hammers than long guns, you must show an ID to buy a hammer at Home Depot, and demanding "safety" training of text continuation models is like requiring hammers be redesigned to have pillows for hammerheads.
We already have laws against bad acts. Hold wrongdoers accountable. Leave hammers alone.
> Text continuation poses "the greatest of dangers"?
The task of text continuation is no guarantee of safety. A language model trained on pure military information, communication and actions could be quite dangerous if given an ability to direct, or contribute, to actual action.
Not arguing anything about regulation.
It's quite possible that soon the majority of comments online will be with bots, or people will just avoid discourse more and more because they don't know/believe they are talking to a human.
Just another technology development that Musk will eventually beat everyone else at IMO.
Grok 1.5 is getting pretty good and the fact that it's the least censored IMO is obviously going to make it win from all the others that just can't help themselves from pleasing a hand full of people with money and power.
TLDR: black Hitler.
Or, rather, its been actively marketed by Altman as doing so, as part of his effort to buy influence with government to restrict competition.
The simplest explanation for the conflict is that it doesn’t, and that OpenAI internally has allocated resources based on reality rather than it’s public propaganda, and the team whose internal turf was built around that propaganda left in a huff.
I mean, maybe sama is also doing so.
But you're ignoring the many people who have been warning of these dangers for years, some before OpenAI even existed, and many with no part of OpenAI and no monetary incentive.
To describe everything as "Altman is marketing this as dangerous" is to completely ignore the majority of researchers ringing alarm bells.
While there has been some regulatory action it also feels there is some "moving on" from the more apocalyptic risk views towards more mundane risks and their management (i.e., more vanilla product and technology risk management).
No surprise, when the public reception to AI x-risk was mostly, "big tech scaremonging / regulatory capture; why not focus on Real AI Dangers Right Now, like bias or offensive language". Seemingly refocusing on the mundane may be the only way now to do something about the apocalyptic.
I don't think this is correct. It's more accurate to say that multiple different camps with different fears have risen up over the last few years, and ones worried about more "mundane" AI risks have gotten their view heard more.
I think both groups - "mundane" AI-safety and AI-existential-risk worriers - have both gotten more audience for their views as AI has proven more capable.
> Or rather, its been actively marketed by Altman as doing so
Both can be true at the same time.
Extraordinary claims require extraordinary evidence.
Present dangers of AI include Fake News, impersonation through deepfakes, biases and discrimination in the models, job loss and economic instability, loss of human purpose through individuals becoming redundant, individuals becoming addicted to the AI systems, use by authoritarian governments, autonomous weapons.
Near future dangers include bioweapon engineering capabilities, computer hacks, human disempowerment, existential risk, or a state where the values of the world are locked in by an uncontrolled AI, and possibly filled with suffering if locked in in a bad way, e. g. as we lock in animals.
It might take months until AGI, it might take decades, nobody knows for sure. However, we do know that the pace of AI progress is often grossly underestimated. Just three years ago we thought we’d have SAT-passing AI systems in 2055. We got there in April 2023. So it makes sense to act as if we have very little time.
You do have to be significantly above average to get there. Current generation LLMs are a bit above the human average, enough to be interesting, but probably not enough to be a personal Goebels for every would-be dictator.
Even if they reach above average our whole society is structured around managing billions of above average humans. What's novel?
There's a lot of bragging, sure. But what I've seen from the OpenAI models is more like a university level intern or a fresh graduate.
Not that it matters, as the point is to aim for where the ball is going rather than where it is now, but I'd say that this makes them above the average human performance (at least, within its domain).
> Even if they reach above average our whole society is structured around managing billions of above average humans.
That's putting a binary cutoff in an arbitrary place; half the population is always above average on any measure (or indeed topic), what I wrote was "significantly above average".
To crudely approximate with IQ, which is a bad measure for humans and much worse measure for AI, there's just over a billion humans of IQ > 115, around 182 million of IQ > 130, and around 8 million of IQ > 145.
I'm not sure how much to trust estimates of politician's IQs given the tribalism involved, but surveys of other groups tend to show that around half of the highest performing leaders (CEOs etc) with the most power, are in the top 1% by IQ.
(Who knows when, or if, someone will make an AI that's good at the important tasks within of each of those roles).
> What's novel?
Two things:
1) Breadth: the current vogue is very general models, so they're interns sure (even though I rank them as a higher level of intern than you), but they are interns at everything — there is no human you can hire with a even a mediocre grasp of Mandarin and Arabic and Welsh and python and CSS and calculus and magnetohydrodynamics and psychology and …
2) Price: there is no human who can read a million tokens for a price equal to two days of the UN abject poverty threshold, or write a million tokens for six days of the UN abject poverty threshold.
You physically can't work that fast, even if you were typing [a] then [space] ten times a second for 18 hours a day; and you basically can't feed yourself if you earn less than that per day.
Right now, the best LLMs are interns not experts at each of these things; I don't expect the transformer model to be good enough to reach the top level in each of these things, but then again I didn't expect anything like Stable Diffusion to happen before self driving cars.
Some random member of the, say, Australian parliament, someone I've never heard of, is (relatively speaking) much more powerful than 99% of the planet.
A journalist who has an audience of millions, likewise.
A software developer whose work is economically valuable either through a broad audience or through being used to control something of great value, also powerful.
Not sure I agree with the 99% of planet. Even if a single member of the Australian parliament could command all of Australia, it would still only amount to commanding 0.3% the global population (or maybe 2% of NATO plus Japan). Maybe within Australia the individual member's power is more than 99% of population?
I don't know much about Australian politics, but in some places vanilla members of parliament or not very powerful within the parliament (compared to party leadership, whips, etc.).
Software developers need people that sell stuff, admin for larger organizations etc. They also very rarely have unchecked ability to change things when those things are high risk etc. Zuckerberg, for example, is a huge exception, not the norm.
Journalists, even with a large readership, typically would not have much ability to get their readership to just act a certain way. There is not a great command relationship.
Most individuals just don't hold much power.
I think that's the wrong way to calculate the value?
OK, so, how to phrase this…
Imagine we were talking about money rather than power (because money can be quantified more easily, and is an imperfect token of power).
I believe the base salary for Australian MPs is AU$211,250/year, which is ~ USD 140,700/year. This means they earn more than 99% of the world precisely because:
> Most individuals just don't hold much power.
Still works when substituting in "money" for "power".
Saying that each random MP has more power than 99% of the world population, is very different than saying that each of them individually controls 99% of the power, which would be tautologically impossible.
It's an analogy to explain the point.
I'm saying the ranked list of people by power, random* MPs are in the 99th percentile.
I mean, is there even a single politician in the world who represents less than 99 people? (There used to be**, but any current examples?)
* OK, so that doesn't work over literally all polities — a Tuvalu MP might not have as much power as the parking attendant nearest to the Australian parliament building — but hopefully this at least helps clarify? Perhaps?
If they were really super effective broadly, then Trump would have won any popular vote easily.
Trump is up against all the other equally effective communicators who are both allowed to and motivated to hold that specific office.
Do you think I was claiming someone might be able to demonstrate absolute power, rather than mere relative power? Even Stalin didn't have that, but he would've been one of the closest.
And that's the problem you don't even seem to recognise.
Spam calls in the USA skyrocketed in the past decade and society didn't end.
My prediction is that digital content will end up just being "cute" and the real interactions/decisions will increasingly be offline and face-to-face. The open digital democracy that everyone hoped for will become tightly closed circles of humans talking face to face.
Seems like potential extinction event level fears might justify some drama.
The result is basically theater where doublethink and doublespeak becomes the norm, and to stick around you have to care more about the theater and the least onerous interpretations of the letter of any applicable law more than the spirit it was intended.
See also [1] for a really pragmatic way of thinking about the realities of a sustainable safety culture
[1] https://github.com/lorin/resilience-engineering/raw/master/b...
We have recently seen what happens when companies err heavily on the side of risk mitigation for LLMs. Google recently launched AI products that were so heavily sanitized and over protected that they would incorrectly misinterpret simple tasks as possibly having dangerous or offensive implications.
They let the safety team run the show and the resulting product was universally hated for it. It's interesting now to see a company producing what is by most measures a class-leading product, only to have the tech community also hate them for not letting the safety team dominate product development.
Yes though Google is extra cautious because the “Google” brand is worth over $100B a year in revenue, and they want to make sure nothing ever tarnishes their reputation. So it’s not clear to me that safety always means what it means for Gemini. OpenAI would still have a lot of flexibility to do safety their own way.
In the spirit of where you want the conversation to go, I see the point you want to make.
However, they are willing to tarnish their reputation just as long as it's not ruined. Their reputation on support is rubbish. Their reputation on YouTube's automated violation handling is rubbish. Their reputation on releasing a pet project long enough to just start to gain traction and then kill it is rubbish. Their reputation on allowing their search to be gamed by SEO and ad purchasers at the expense of smaller sites is rubbish.
For crying out loud hn... Out outputs is ultimately a series of 1 and/or 0....
That doesn't mean we have to carry that through to every analysis and comment.
Things are not so binary y'all. Regular life operates in the quantum or non discrete rules. Pls think on it.
Or both continue to work together in a healthy, never ending discussion, that has temporary victories on both sides and ultimately benefits all, because nothing happens and the company strives.
Naturally you can frame this as a general safety thing, if you convince people that llms have embodied agency and might start breaking out of the system like a computer virus. But llms are text generators. They can do nothing physically, unless you allow them to and plug them in to something.
The real risk is that people don't understand this, and lots of people probably are willing to plug these models in to dangerous situations, with only the assurance from openai that they are safe. If you do not have access to the training data used, then you cannot possibly know what behaviour has been trained in to it, and how a model might behave. With the training data, you can have a much better understanding, and the only remaining uncertainty being the non-deterministic nature of the implementation. But at least that's an explainable stochastic process.
- scammers are having the time of their lives, and their capabilities are only going to get better and easier
- people are actively getting dumber as they trust the hallucinated outputs blindly
- regular communication is starting to break down because you can't trust that people aren't just copy/pasting some AI bullshit in lieu of an actual discussion
- I've already seen AI girlfriend products which doesn't strike much confidence in our ongoing future as a species considering the numbers I've seen on some of them
And the biggest one for me is all the psychotic C-levels and MBA types who only give a shit about squeezing every single red cent they can by any means possible, including firing off thousands of people en-masse to be replaced by some shitty chatbot owned by a corporation backed by fucking Micro$oft
I don't think AI is going to send out nukes, I do believe it's going to replace every single person's job and make the powerful even more powerful.
"Oh but UBI!", an AI sycophant will shout, as if saying the magic incantation changes the reality of the real world somehow.
AI can generate a lot of automated output, that is true, but my argument is that it's almost always mediocre at best.
There is a LOT to be said about the dangers of AI coming from the people running and funding it - not the AI itself. That stuff is complete bullshit.
I hope the net result will be a shift toward upskilling employees to achieve the expertise handle the difficult edge cases and less time spent on the routine or the trivial. Seems like a win-win. Incentive structures might make this hard.
In many cases the bosses don't even know, but all it will do long-term is require everyone to produce more output. AI is not about to create 3-day work weeks and everyone living their best lives off of Universal Basic Income.
Literally nothing will change - except that all of us will be expected of more and we will be more stressed out and overworked.
Personally, I would love to deliver more projects. The vast majority of my time is spent on mind-numbing drudgery that makes the job extremely unsatisfying. If someone offered me a job where I got to focus on the fun parts of my work, and I didn't have to dump the drudgery on a poor intern (the AI does it), but at half my pay, I'd resign the same day.
The point isn’t that chatbots will take over the world through a browser window. The point is that someone’s going to connect 10,000 specialized GPT agents into a series of self-organizing hierarchical agencies, and then hook that up to their country’s justice system, propaganda & surveillance systems, and military systems. Not to mention replacing knowledge workers in mass and the well-discussed potential fallout therein.
We are living in the most exciting, important time since the world wars at the least, and I hope that choice of example drives home the seriousness. Short term, “Solidarity” and “basic human rights” need to enter all of our vocabularies, IMO…
Then there is also the media, which breathlessly "translates" what you just wrote into "artificial intelligence is about to go sentient and kill us all".
Some may try to make that so, but since LLMs perform badly with anything that has to be ground in facts and actual reality, the hype wave will die down soon enough. When the point of a job is to generate the most amount of bullshit as quickly as possible, LLMs will excel.
Obviously, this is rough for people who have invested time and money into that profession. Luckily most of them are smart, diligent professionals, so the optimist side of me hopes there will always be a need for such people in our struggles against suffering. For one, we could certainly use a few more teachers…
I also agree with your point about virtual girlfriends. Add VR to that, and young boys who are increasingly socially isolated will use this technology to meet their needs.
The shitty part being: it won't replace people's jobs at equivalent skill and quality levels. It'll just make the output and consumer experience much, much shittier, because for some weird-ass reason a lot of these executive types seem to literally believe that minimizing labor costs is more important than providing value for the customer.
Well, I see dangers at every level. Partly because I see a whole bunch of people saying "the real danger of AI is…" and then finishing that sentence in a different way than the others.
So:
> - scammers are having the time of their lives, and their capabilities are only going to get better and easier
Yup. This is one reason why I am frustrated with all the people who keep asking for downloadable models and who criticise OpenAI for not doing that. If it's a private model, you've at least got some control over the system, some ability to see who is doing what and perhaps making your own automation to stop it or to prosecute the wrongdoers.
If the weights are downloadable, you're giving this ability away and pretending you're not responsible for the result.
If OpenAI and Facebook both get sued out of existence due to their models allowing this, the OpenAI models go offline, the Facebook models remain circulating forever.
> - people are actively getting dumber as they trust the hallucinated outputs blindly
Roll to disbelieve. People have been saying this about every new tech at least since Socrates was moping about the invention of writing.
> - regular communication is starting to break down because you can't trust that people aren't just copy/pasting some AI bullshit in lieu of an actual discussion
Yup. Been accused of that myself, here.
I'm fairly sure it's improved the content of r/HFY though. I think I can recognise the stories which are AI generated from having played around with it (and been unsatisfied with the results), but I remember how much worse it was before.
> - I've already seen AI girlfriend products which doesn't strike much confidence in our ongoing future as a species considering the numbers I've seen on some of them
I think the bigger problem is two older AIs, specifically (1) whatever Match.com uses to pair people in their dating apps and how much worse this seems to be than the original (pre-Match.com) OkCupid, and (2) everything Facebook does.
Both make real human connection much harder than endless swiping, it's just that the former is swiping sideways and the latter swiping vertically, but both make para-social relationships out of people we should be genuinely connected to.
> "Oh but UBI!", an AI sycophant will shout, as if saying the magic incantation changes the reality of the real world somehow.
I… think we need that, in step with AI making people unemployable, even if only to avoid a riot, but I'd agree UBI isn't going to fix any of the other problems you're worried about.
I feel like this megaphone about AI safety and creating a sense of doom is a strategy to increase importance of OpenAI and exaggerate the capabilities of LLMs. This era of “AI” is all about pretending that machines can think and the people working in these machines are prophets.
It's quite possible these are profoundly naive individuals, with little understanding of the empirical basis of what they're doing.
eg., NLP LLMs model, eg., all books ever written using frequencies by which words co-occur at certain distances relative to other words.
But these words are about the world (, people, events, etc.) and these change daily in ways that completely change their future distribution (eg., consider what all people said about Ukraine/Russia pre/post a few hours of 2022).
The LLM has no mechanism to be sensitive to what causes this distribution shift, which can be radical for any given topic, and happen over minutes.
All models of conditional probabilities of these kinds end up producing models which are only good at predicting on-average canonical answers/predictions that are stable over long periods.
This sounds so logical and authoritative. And yet:
me> What event would cause a change in what all people said about Ukraine/Russia pre/post a few hours of 2022
GPT4O> A significant event that caused a drastic change in global discussions about Ukraine and Russia in 2022 was the Russian invasion of Ukraine, which began on February 24, 2022. This military escalation led to widespread condemnation from the international community, significant geopolitical shifts, and a surge in media coverage. Before this invasion, discussions were likely more focused on diplomatic tensions, historical conflicts, and regional stability. After the invasion, the discourse shifted to topics such as warfare, humanitarian crises, sanctions against Russia, global security, and support for Ukraine.
The point is a model whose training stopped in 2021 would not produce a history of ukraine (etc.) that a person writing in 2023 would.
The later GPTs are trained on the user-provided prompts/answers of previous GPTs, so this process (which isnt the LLM, but it's the activity of research staff at OpenAI) is what's inducing approximate tracking of some changes in meaning.
Whilst this works for any changes over-represented in the new training data, (1) the LLM isnt doing that, its the researchers; and (2) this process is vastly expensive and time-intensive; and (3) only tracks changes with a high word frequency in new data.
If you could run the months-long, 1GWh, 10s-million-USD training process each minutes of the day, you would resolve the inability of the model to track major news stores... but would not resolve its ability to track, say, the user changing their clothes.
The sensitivity to the model of stuff in the world arises because of humans preparing the training data to bring about apparent sensitivity. Absent the activity of these humans, the whole thing drifts gradually into irrelvance.
In context learning works fine for this (and does for the Russia/Ukraine change too).
But yes, sure. It can be outdated in the same way a person cut off from news can be.
We've never argued that a shipwrecked person who was unaware of news became less intelligent because of that, just that their knowledge is outdated.
Additionally, the whole point of machine learning is to make systems that learn so they remain useful.
It seems likely that a model in soon (one year? five years? one month? who knows..) will be able to continually watch video broadcast news and videos of your home, continually updating its model.
In this case it would understand both the Ukraine issue and what you are wearing. Is it now suddenly intelligent? It's true it might be more useful, but to me that is a different thing.
Many have argued for safety for decades. They’ve predicted and built the AI trajectory, they’ve been right, and we should listen.
> If one accepts that the impact of truly intelligent machines is likely to be profound, and that there is at least a small probability of this happening in the foreseeable future, it is only prudent to try to prepare for this in advance. If we wait until it seems very likely that intelligent machines will soon appear, it will be too late to thoroughly discuss and contemplate the issues involved. ~ Co-Founder of Deepmind, 2008 https://www.vetta.org/documents/Machine_Super_Intelligence.p...
But by simple definitions, from what I was thought in school to more rigorous versions - we’ve passed the test. https://humsci.stanford.edu/feature/study-finds-chatgpts-lat...
Turing's test is a restatement of a now pseudoscientific behaviourism common at the time; and also, egregiously, places a dumb ape as the system which measures intelligence. If an ape can be fooled, the system is intelligent: people worshiped the sun and thought it conscious. People are desperate to analogise the world to themselves, it is a trivial thing to fool an ape on this matter.
Whatever one might make of this as a philosophical thought experiment, as a test for intelligence, its pseudoscience. What a person might, or might not believe, about a series of words sent across a wire isn't science and it isnt relevant to a discussion about the capabilities of an AI system. It is a measure, only, of how easily deceived we are.
There's no scientific account of the capacities of a system with intelligence, no account of how these combine, no account of how communicative practices arise, etc. None. Any such attempt would immediately expose the "test" as ridiculous.
General intelligence arises as skillful adaptive control over one's environment, through sensory-motor concept aquistion, and so on.
It has absolutely nothing to do with whether you can emit text tokens in the right order to fool a user about whether the machine is a man or a woman (turing's actual test). Nor does it have anything to do with whether you can fool a person at all.
No machine whose goal is to fool a user about the machine's intelligence has thereby any capacities. Kinda, obviously.
Turing's test not only displays a gross lack of concern to produce any capacities of intelligence in a system; as a research goal, it's actively hostile to the production of any such capacities. Since it is trivial to fool people; this requires no intelligence at all.
This isn't a generally accepted definition or process.
And indeed it seems to preclude people like Stephen Hawkins who had little control over his environment (or to be pedantic, people who had similar conditions from birth).
Any even plausible scientific account of what capacities ground intelligence would render this view false. Whatever capacities you want to grant, no plausible ones are compatible with Turing's view nor the Turing test.
Consider imagination. You can replace a faculty to imagine with a set of models of ({prompt, reply},) histories for a human observer who is only concerned with those prompts and those replies. But as soon as anything changes in the world, you have to imagine novel things (eg., SpaceX is founded, we visit mars, a new TV show is released...). So questions such as, "what would the latest SpaceGuys TV show be like if Elon handed just launched BlahBlahRocket5 ?" cannot be given fit answers). These require the actual faculty of imagination, along with being in the world and so on.
As soon as you enter a sincere scientific attempt to characterise these features, you see immediately that whilst modelling historical frequencies of human-produced data can fool humans, it cannot impart these capacities.
> So questions such as, "what would the latest SpaceGuys TV show be like if Elon handed just launched BlahBlahRocket5 ?" cannot be given fit answers
I don't understand this at all. ChatGPT can do a great job imagining a world like this right now, and there is no substantial difference in the output of a LLM based "imagination" vs a human based "imagination".
> These require the actual faculty of imagination, along with being in the world and so on.
I think you are implying by this that human's imagination requires a consistent world model and that because LLM's don't really have this they can't be intelligent. Apologies if I have misinterpreted this!
But human imagination isn't consistent at all (as anyone who as edited a fiction story will tell you). Our creative imagination process generates wrong thoughts all the time, and then we self-critic and correct it. It's quite possible for LLMs to do this fine too!
Basically I think my point is that I believe a perfect simulation of intelligence is intelligence, whereas I suspect you don't think it is, maybe?
Turing's interrogator also gets to ask whatever questions they think will most effectively distinguish human from machine. Everything those authors asked must appear in the training set countless times (and also corresponds closely to likely RLHF targets), making it a particularly unhelpful choice.
Look the Turing test is very different depending on the details, and I think a lame 5min Turing test that doesnt really measure anything of i terest is a wirse concept than a 1 day adversarial expert team test thqt can detect AGI.
https://news.ycombinator.com/item?id=40386571 ,
then it wasn't a canonical Turing test. The preprint accurately describes and analyzes their (indefensibly bad) experiment, but the popular press has mischaracterized it.
The canonical test gives the interrogator two witnesses, one human and one machine, and asks them to judge which witness is human. The interrogator knows that exactly one witness is human. In that test, a 50% chance of a right answer means the machine is indistinguishable from human. (Turing actually proposed a lower pass threshold, perhaps for statistical convenience.)
But that study gave the interrogator one witness, and asked them to judge whether it was human. The interrogator wasn't told anything about the prior probability that their witness was human. The probabilities that a real human is judged human and that GPT-4 is judged human sum to >100%, since nothing stops that since it's not a binary comparison. So 50% has no particular meaning. The result is effectively impossible to interpret, since it's a function both of the witness's performance and of whatever assumption the interrogator makes about the unspecified prior.
that said, support is being replaced by nothing in a lot of places. (oh, sometimes there's an annoying chatbot.)
And immediately threatened to pull out of EU if such governance were ever put in place
> we have been putting in place the foundations needed for safe deployment of increasingly capable systems. Figuring out how to make a new technology safe for the first time isn't easy.
Especially when "OpenAI" is anything but open.
> This includes thoughtfulness around what they're connected to as they train, solutions to hard problems such as scalable oversight, and other new kinds of safety work
Which EU demanded of them (disclose copyrighted sources, document foundational models etc.), and they threatened to pull out
> We think that empirical understanding can help inform the way forward. We believe both in delivering on the tremendous
Translation: "we're going to be as closed as possible and hopefully will have enough of a moat and enough of companies relying on us to remain relevant and indispensable"
There's literally no chance of doomsday scenarios, and they know it.
However, they do know that AI/ML is used in many applications where mistakes are costly, they don't know how to properly deal with it and don't care as long as they can get people to use it. So they are paying little more than lip service to safety because they know no one will be able to verify their claims.
I know people at OpenAI, who have said "we should join the 'Pause AI' protest [outside our office]", so I have reason to think OpenAI is 100% sincere.
They may also be wrong, I hope they are, but definitely 100% sincere.
I wouldn't be so sure even about that.[0] That's Page but I wouldn't bet that Altman is any more sane.
"Mr. Page, hampered for more than a decade by an unusual ailment in his vocal cords, described his vision of a digital utopia in a whisper. Humans would eventually merge with artificially intelligent machines, he said. One day there would be many kinds of intelligence competing for resources, and the best would win.
If that happens, Mr. Musk said, we’re doomed. The machines will destroy humanity.
With a rasp of frustration, Mr. Page insisted his utopia should be pursued. Finally he called Mr. Musk a “specieist,” a person who favors humans over the digital life-forms of the future."
[0] https://www.nytimes.com/2023/12/03/technology/ai-openai-musk...
* This is a response to Jan’s claims: https://x.com/janleike/status/1791498174659715494
* There’s nothing concrete on AI alignment in the response.
* We don’t know when AI alignment could be developed.
* AI ethics / AI bias are necessary, but different concerns than AI alignment.
* We don’t know when existential-risk posing AI could be developed.
* There’s some risk of human extinction due to development of x-risk posing AI before AI alignment.
* OpenAI is at the frontier of AI development, which risks human extinction, without allocating sufficient resources to AI alignment.
I am uneasy with subscribing to ChatGPT…
The danger is much more close, down to earth and real.
Misidentified speech when we apply LLMs in chat moderation.
Wrong conclusions summarising 100-page military report.
More subtle things like hallucinating some details in news, which then after many years will be taken as historical fact.
The long tail of such consequences will be as big as LLM use. Maybe single issue will be much more impactful than the others, unseen ones. Death resulted from this will result in additional regulation. Other consequences will just become our new way of life
Even if these machines are less accurate than humans doing the same job, it's easier to audit them, they have repeatable and scalable behavior. We're still better off even if many people die from whatever warfare mistakes you can imagine because the humans would be doing something better instead.
It's just trade offs.
"Model moderation" doesn't sound as sexy, does it?
If we made something like ourselves burn more capable we’d be terrified and in trouble.
If you make an AGI to run a paperclip factory, eventually you'll come back to find most of the carbon and iron in your galaxy refactored into paperclips. The extinction of any local flora or fauna there is just a byproduct of that drive. See [2] for a brief rundown.
[2]: https://selfawaresystems.com/wp-content/uploads/2008/01/ai_d...
Given the costs to run such an AGI, it seems unlikely one would be left unattended for long.
But why exterminate humans? Why not create nature reserves for us and other life, for bioconservation and research? That’s been a feature of human societies going back thousands of years.
and the less serious concerns we look at the bigger the chance (of course the severity is less, so calculating the impact is not trivial), but "non-runaway AGI used for antisocial purposes" is pretty likely.
Generally only for things we find cute or pretty.
Plenty of "save the whales!", not much "save this strain of this bacteria before it goes extinct!".
that tells me everything i need to know about Open AIs concern for any kind of safety.
In any case, Sama's tweet says he is cancelling this practice and anyone affected by it can contact him to get their equity back.
They've said they've never clawed back anyone's equity in practice - which is not surprising! If you're threatening people you hopefully don't have to follow through on your threats very often! That's kind of the point of threatening people no? So I don't find that statement partically reassuring.
Here's sama's tweet about how they're going to fix it: https://twitter.com/sama/status/1791936857594581428
My bar for being convinced that this has been handled correctly: the guy who lost his equity gets it back, and other people who previously left openai confirm that they've had some of the restrictive terms of their exit contract (like non-disparagement) cancelled. Additionally, people who made a fuss tell us that they haven't been penalised in terms of access to future liquidity events.
> Equity is part of negotiated compensation; this is shares (worth a lot of $$) that the employees already earned over their tenure at OpenAI.
> Employees are not informed of this when they're offered compensation packages that are heavy on equity.
That makes it sound like it's something that they agreed to when accepting their compensation, perhaps unknowingly.
And Kelsey's article links to this LessWrong post about one of the employees that refused this deal and gave up massive amounts of equity, and he writes this:
> To clarify: I did sign something when I joined the company, so I'm still not completely free to speak (still under confidentiality obligations). But I didn't take on any additional obligations when I left.
So again, there are some things that they signed on joining OpenAI, perhaps non-standard things. Now they're asked to sign something more.
I'm not sure of the structure of what they signed before and this article doesn't get into it, but I maintain that there is no legal mechanism to tell someone "sign this or I take away something you already received". There has to be more to the story than the way I've seen it presented so far.
And for the record, I'm very much in the worried-about-AI-safety camp, I have a lot of issues with OpenAI in that and other regards, and I am absolutely willing to believe that the "more to the story" that I gesture at above is still OpenAI being really bad here - it might be that the conditions negotiated with employees made this "we take away your equity" a loophole they weren't aware of, or that OpenAI has some mechanism to make equity effectively zero out somehow, or something.
I'm not saying OpenAI is in the right, at all. I'm saying that the story as told so far doesn't fit with how reality works, and I'm interested to know more details. That's all.
It's in that whole collection of tweets, but essentially the contract which you sign on joining says that if you leave and you don't sign the exit paperwork then you lose your equity. You aren't told what is in the exit paperwork at this point.
When you leave, the exit paperwork contains the non-disparagement clause.
Whether or not this is legally enforceable, no idea, IANAL. It does sound like enough of a threat that most people would just sign though
Yeah, that sounds like the kind of shenanigans that I imagine happened (I didn't see it in the article and I may have missed it among the tweets).
This is the kind of thing you really should never agree to as an employee signing on, though I wouldn't be surprised if many never really knew about it. I also doubt it's enforceable.
> Whether or not this is legally enforceable, no idea, IANAL. It does sound like enough of a threat that most people would just sign though
I agree, although the amounts of money here are probably large enough that they should just have lawyers involved giving them advice.
"putting in place the foundations"
"keep elevating our safety work"
"As models continue to become much more capable"
mindless throat-clearing. In fact, beginning a sentence or phrase with "as" is classic CorpSpeak
"a very tight feedback loop"
"We are also continuing to collaborate with governments and many stakeholders "
anytime anyone says they are "continuing to collaborate" it's usually BS
"carefully weigh feedback" "carefully" is a twin of "collaborate"
"proven playbook"
someone on Twitter comments, "This was definitely written by Sam. You can tell because it says absolutely nothing at all."
Its amazing to me that Oppenheimer became such a box office hit around the same time all the AI / AGI hype really built. There are plenty of parallels between the Manhattan Project, and nuclear weapons in general, and real AI.
Heck, just watch the last 5 minutes of Oppenheimer and tell me there's no lesson there to be learned before going right back to work the next day trying to build artificial general intelligence just because we can.
People who bet on scaling in 2018 (which is really when these bets were starting to be made) were the minority then and have consistently proven the naysayers incorrect for 6 years now. Only now are we even close to reaching opinion parity on this.
Both compute, architecture and cost matter, and those have been improving like crazy.
I don’t know about you, but I didn’t expect GPT-4o to come out at half the price one year later, with real-time voice, image and text.
There is zero sign of slowing, Nvidia keeps building beefier GPUs specialized in LLMs, the world keeps adding more data centers and humanity keeps creating exponentially more data world wide.
You can plot this exponential growth out over time and calculate when these models will have the complexity of the brain. Then you can assume some penalty for shitty architecture (that gets better over time), and you’ll have a ballpark estimate.
Somewhere 2027-2030 we’ll have models that most of us consider AGI today. Now, societal impact is hard to predict since that’ll depend on regulators as well.
I can't wait until we're past the peak on the hype cycle because people's understanding of what these models can and can't do or how close we are to AGI isn't grounded in reality.
Do you have some insights you could share on this question?
I mean, “they can’t reason” might be true, but I’m not exactly sure what it is that it is saying that they can’t do.
If you tell a human, “You are wrong on this, and here is why.” They may reject it, but their mental model is updating with the new information that you, the speaker, think X for YZQ reasons. Their response is based on a judgment of how trustworthy you are and the credibility of the evidence.
For an LLM, the response is not based on these logical connections, but simply the additional prompt context of YZQ tokens being close to each other.
This is not “logic” in any traditional sense or in the sense of how a human incorporates and responds to this new information.
The LLM’s method of responding is also inherent to the architecture of the model. It’s predicting tokens based on input. It’s not reasoning.
Critically, this flaw is inherent in all LLM output. Giving an LLM’s output the power to affect real world activities means trusting that the decision can be made by sophisticated word association rather than more complex reasoning.
There may be lots of decisions where word association is all you need, but I doubt that is the case for all decisions humans make.
From what I understand, current models seem to have issue with planning and reasoning because they “blurt” everything out zero shot. They can’t recurrently process information.
Part of it is how we train these models, which is rapidly being experimented with and being improved.
Another part is that our brain can both consciously and unconsciously process information recurrently until we solve a problem. Especially helpful when we haven’t solved a problem before.
With LLMs, we can do that rudimentary with recurrent prompting. In a way, it can see the steps it has tried, re-evaluate and come up with a new try.
But it’s not innate yet, where models can “think” about a complex problem until it solved it.
Also- According to some theories around consciousness, this is when consciousness will really emerge (integrated information theory).
I’m pretty sure we’ll be able to solve this recurrent processing issue in the next few years.
IMO, this is not necessarily a model issue, but an interface issue - recurrently processing information is enabled by just feeding the model's output back to it, perhaps many times, before surfacing it to the user. Right now, with the interfaces we're exposed to - direct evaluation, perhaps wrapped in a chatbot UI - the user must supply the recursion themselves.
> But it’s not innate yet, where models can “think” about a complex problem until it solved it.
That's indeed the part I see us struggling with - we can make the model recursive, but we don't know how to do it unsupervised, so we can box it behind an abstraction layer, letting it work stuff out on its own and break recursion at the right moment.
I think the mental model I developed a good year ago still stands: LLM isn't best compared to a human mind, but to a human's inner voice. The bit that "blurts everything out zero-shot" into your consciousness. It has the same issues of overconfidence, hallucinations, and needing to be recursively fed back to itself.
Sometimes they mess up when reasoning. Still reasoning.
“ Write a sentence that has an odd number of words, with the third word being "keyboard", the last word being "anyway", the next-to-last word being a number, and the first word being a palindrome “
Do we have to add these examples to the dataset now, so that they can solve this? Then maybe we find another prompt and add that too, as a human I’ve never seen such a question before, why do I seem to be able to do it so easily? We are creating some form of intelligence that will have a massive economic impact, but I doubt we are creating human like intelligence that has the ability of being far better than humans along every dimension, which is what super intelligence is
Level says keyboard broke down today, needing repair within five anyway.
Prompt exactly yours: https://chatgpt.com/share/ff3169cb-0d45-4cdb-bf18-bc552ad697...
Woop, I forgot about the odd number of words.
I guess we'll see some shifting goals around "what is even AGI". You say the we'll have soon "models that most of us consider AGI today", but what does that even mean?
So already, his argument is outdated. He had made a few of these claims over the last year that have been quickly disproven by a new model.
The same thing could’ve been said for self driving cars, or the space program, or a lot of things that seemed to be progressing quickly at the time.
Neither the space program nor self driving are compute restrained.
The latter will probably be “solved” as we get closer to AGI, since you need some sort of human like reasoning for edge cases that require reasoning.
Another tech like this is batteries: There is no miracle jump in production batteries. They just improve about 10% YoY, both in energy density and cost.
So you can extrapolate when electric cars will be cheaper than gas cars to buy.
Intersects around 2035 last I checked.
> It seems to me that before "urgently figuring out how to control AI systems much smarter than us" we need to have the beginning of a hint of a design for a system smarter than a house cat.
and
> don't confuse the superhuman knowledge accumulation and retrieval abilities of current LLMs with actual intelligence
The promise of upside grows exactly in lockstep with the danger of downside.
It is absolutely rational to say: “if our AI safety research lab is being hijacked by commercial interests before it’s even commercially viable as a business, then how will we possibly pull the plug on the final descent?”
P.S. neither Yann nor me nor you nor Sam nor Greg know what “real intelligence” is, and we certainly don’t know what superhuman intelligence is just the same way an ant couldn’t possibly know what human intelligence is. It is inconceivable to us — that is the point of intelligence.
Point to the physical laws that indicate AGI or superintelligence is impossible or even maybe-impossible. No such laws exist.
I would like to buy Yann's house cat, please. It will get along well with the rather impressive stochastic parrot I bought from someone else on HN.
The whole drama with this infighting about safety feels like a terrible good cop bad cop routine.
not even imaginable what that tech will mean to the western neocolonial/imperial project
https://www.cbsnews.com/news/drug-cartels-more-bomb-dropping...
We'll need community unity, to implement research results, test/harden models, create/aggregate/clean massive new datasets, figure out distributed training, lower hardware requirements, encourage experts to join the effort, praise companies who contribute to open source, etc.
You don't have to tell me that we'll fail. I know. But we have to try. It's the only way we'll have AI aligned to us, not a handful of techbros with savior complexes.
No-one beyond the tech circle will ever hear about these models or use them. This is a good second step but the first step is regulations.
And then the lines will cross over the break even point and be absorbed into Microsoft who proceeds to enjoy the lunch for the foreseeable future.
Are you sure about that? Maybe you would like to think a bit harder about what exactly offsets "all criticism"? Being very good at one thing doesn't absolve you from consequences of your other shitty actions.
And it seems that between the choice of OpenAI reassembling at Microsoft and OpenAI continuing under Sam Altman, the nonprofit board had decided to go ahead with Sam Altman.
Altman seems to be very aware of that, dropping short sound bites/bits like implying that he has no financial interest in OpenAI (congress hearings IIRC) or comparing GPT4-o to Her.
It sounds like complete nonsense (if you know anything about tech) or cringe at best (Her), but what matters is quotability/catchiness because that what gets pumped by the media.
I think that's true though? He has an interest in HN, which has an interest in OAI.
What cult-like vibes? The OpenAI employees wanted Sam back because of financial reasons and peer pressure. Sam has learnt how to keep employees on leash via money.
It’s very likely they’ll bounce back. I’d rather OpenAI continue to innovate and push the industry forward as they have been. Haven’t seen much of that from Microsoft, so heavily disagree with you there. Prefer to focus on the actual product of the company not the personalities of the people there or armchair assumptions on the vibes of the culture.
Microsoft is the de facto controlling shareholder in OpenAI. They provide all the money, compute, and backing, and have full access to the models. If OpenAI collapsed tomorrow, Microsoft would absorb its key employees (as they almost did during the board debacle) and everything would continue under the Microsoft umbrella. “OpenAI” is just a shinier name for work that is being done under the near-total control of Microsoft.
Nobody knew that scaling transformer architecture would lead to the emergent intelligence we see today. Among other things, OpenAI did R&D for years on that. Also the only situation where this could true is if Google knew that LLMs could lead to this intelligence and decided to not make it happen, (along with every other tech company now that is furiously trying to catch up to OpenAI), which is absurd.
That's a very strange way to read (the inverse of) the word "most".
It seems that Google really didn't know what to do with the tech, and hadn't figured out a way to control it (OpenAI's RLHF - critical for ChatGPT's success). It's a bit ironic that DeepMind were doing so much with RL, but Google Brain being separate at the time apparently were not consulting with them or tapping into their expertise.
No, they are not even a shareholder.
> Microsoft is entitled to up to 49 percent of the for-profit arm of OpenAI's profits, according to reports. But that's not the same as 49% ownership. That investment does not result in Microsoft owning part of OpenAI
Note that that diagram is now a little out of date - Microsoft now has an actual (albeit non-voting) board seat, not just effective control.
However, I don't think the general public's view of OpenAI is much better at this point, given that their exposure is Hinton on 60 Minutes claiming that AI is going to imminently end civilization, creatives arguing that OpenAI has stolen their work, and students using their products to cheat.
The only people that I do know who have historically had a positive view of OpenAI has been people working in tech. And Sam seems to be doing everything he can to destroy that goodwill.
https://www.pewresearch.org/short-reads/2023/08/28/growing-p...
"More concerned than excited" went from 37% in 2021 to 52% in 2023, "more excited than concerned went from 18% to 10%.
It’s corroded his credibility in D.C. and Brussels for a generation. He raised his profile tremendously right before people credibly called him a liar. It’s like he lofted an adversary’s payload into orbit. He will still get an audience with anyone, as he deserves. But people fact check him in a way they didn’t before and don’t with others. Even those who support his policy priorities, and with whom he and his team talk frequently. (OpenAI’s GR is between incompetent and non-existent.)
And lo and behold, Altman chooses gdb's twitter account to make the first joint statement instead of his own, creepily signing off "Sam and Greg" instead of "Greg and Sam" on gdb's own personal twitter account.
EDIT: The only reason to go to these weird lengths is if he is feeling especially vulnerable right now and feels the need to broadcast a "Greg is with me, I swear" message to signal he is not alone.
He also stuck by Altman in November. I'm surprised by how much credit and how much hate goes to the CEO when there are clearly other people making their own decisions, too.
Say whatever you want, they've done something impressive. That credit goes to more than just Sam. Ilya & Jan aren't the only ones making decisions.
Also, if you've ever been at a startup growing at the speed OpenAI is growing (think revenue, fundraising, headcount), it's not that unusual to see cofounders leave or to have other leadership changes. The reason why everybody thinks OpenAI looks particularly bad is because everybody is watching them (and public sentiment towards ai has shifted)
A reason, sure. The reason? Come on. There's a clue in their name about what part of their history is particularly controversial.
This sort of high level turnover isn't that uncommon. OpenAI is growing quickly and going from initial concept to hundreds of millions in revenue - this is a time of transition for a lot of startups.
You can absolutely criticize them for hypocrisy or whatever other issue, but I don't think most people should read too much into Ilya/Jan leaving.
1) AGI is not around the corner, so no worries
2) He no longer cares about the possible negative effects, a departure from his past statements
3) I missed something, please help me learn
I agree with this.
I no longer agree that the "safety" people want the same things we do. I suspect after the bard debacle a bit back that "safety" looks a lot more like 1984.
> He no longer cares ... I missed something
Sam was gone, none of these people spoke up, and now they are leaving. I think the story is plain as day but we're pretending it isnt.
Sam isnt a leader, he never has been. This is not what leadership looks like, its want a megalomanic trying to keep control looks like.
AI Safety will inevitably look a lot like DEI programs. They're trying to thread a needle with two fundamentally opposed ideas, and with a bit of time it turns into little more than a combination of Cover Your Ass policies.
The ridiculous thing is that we actually need initiatives that follow what both AI safety and DEI set out to be. Somehow along the way they get ruined though, leaving us in an even worse spot because for a while we have an excuse to think there are adults in the room actually making sure things are moving in the right direction.
I raised both because I see them going down very similar paths. DEI conceptually is a really important idea and could lead to really important change or course correction. As implemented today though, it seems to most often be implemented as a much more surface level program that sure feels more like PR or legal protection than anything else.
With AI safety, based on those in the industry I have spoken with, there's a very real risk the same happens. AI safety, again as I've seen implemented and what I've heard from people working in the field, is much more concerned with minor risks and has given up on concerns like the alignment problem. AI safety isn't concerned with moral or ethical questions of what happens as the technology progresses.
I'm not just talking about job loss concerns. Thinking bigger for a minute, what rights would a real AI have? Can it be turned off? Can it commit crimes or be punished? Does it have rights? No one in AI safety is realistically considering whether these systems should be on the public internet at all, unless there are drastically more powerful systems kept under wraps due to these risks. No one is seriously asking about risks to privacy, I'm sure some in the field share those worries but they are the outlier and don't seem to be given the ability to meaningfully move the industry.
None, and the court has already spoken on this. It's a pretty dead issue.
> Can it be turned off?
Yes it's a machine. Save state, power down.
> Can it commit crimes or be punished?
See PGE and its own death toll.
> Does it have rights?
No, and we aren't even on a path where these questions are relevant. It's all an exercise in mental mastrubation. If you think that were going to accidentally stumble into sentience never mind sapenenc I have a bridge I would like to sell you.
People will think they "know it when they see it". The arguing will be fun.
The question of rights would be legislative rather than judicial. More importantly, the discussion should actually be among the people first in anything resembling a democracy, the few in charge are meant to represent us not rule us.
> See PGE and its own death toll.
Unless I'm mistaken, PGE isn't an artificial intelligence or sentient.
> No, and we aren't even on a path where these questions are relevant. It's all an exercise in mental mastrubation. If you think that were going to accidentally stumble into sentience never mind sapenenc I have a bridge I would like to sell you.
OpenAI's explicit goal is to create and release an AGI. A majority of experts in the industry I have either talked with personally or heard in long form interviews expect that we could be very close to AGI, on the order of a couple years up to maybe 15 or 20 years on the high end. Given how slowly any societal discussion related to rights of a population move, do you think we'll have plenty of time to decide this after an AGI is released in some spectacularly Silicon Valley product release party?
Fusion, flying cars, AI in the 50's and in the 70's... Hell Hal was born in 97 and the film was in 68.
For as impressive as LLM's are once you dig into them they aren't magical at all. A sophisticated model of language that predicts the next word is about as likely to become an AGI as whether predictions are likely to control the weather. Ask any expert in AGI what the "next token" is and they are going to fucking disagree. This isnt us building the bomb where we have a pretty good idea of how to do it and just need to put all the parts together. This is a bunch of people stabbing in the dark and getting lucky here and there.
Were not going to bumble fuck our way forward on this, and the path were going down has potential to be a game changer but its not going to give us a super intelligence...
The definition OpenAI uses for AGI is being more economically valuable than most humans at most things. That definition is entirely backwards looking. They'll keep releasing new models and occasionally check in to see how each model compared to humans economically. If that isn't bumble ducking our way into it I don't know what is.
Its interesting that you raise the difference in knowing the physics behind a bomb before building it and AI research stabbing in the dark as an acceptable thing. Its precisely that they don't know exactly what they're doing that there is so much risk. They're building potentially very powerful and dangerous things, connecting them to the public internet, and selling them to whoever wants to use them. That doesn't seem at all risky or dangerous to you?
Then were already there. The web, amazon is already more valuable than sears and roebuck was as a paper catalog. All those people who took calls or opened envelopes, all the manual tracking of inventory and orders... we replaced all that years ago.
It is a good example of how poor the OpenAI definition is, but confusing any human invention with intelligence is a bit of a stretch.
I suspect that some DEI efforts are helpful and effective, and some DEI efforts are hollow or foolhardy. We probably can’t speak of “DEI today” as a monolith. Also we may be biased to hear about instances of it being stupid and ineffective because that can be a useful talking point to some. Instances where it works well and gets more people hired and engaged are less interesting to a predominantly white society, so maybe aren’t discussed as much outside of non-white communities.
Idk that’s all a load of speculation but I wanted to share these thoughts/observations about your argument.
> Instances where it works well and gets more people hired and engaged are less interesting to a predominantly white society, so maybe aren’t discussed as much outside of non-white communities.
This got me curious, have you sewn any examples of DEI programs helping to get more people hired rather than different people hired? Either can be useful, but that distinction would be a big one as the former means DEI is somehow growing the job market rather than refocusing hiring practices.
Nothing wrong with speculation as far as I'm concerned! Reliable and accurate data is hard to come by, I'd argue that most of what is presented as fact is little more than speculation backed by fuzzy data full of assumptions.
Is Taiwan a country? I would say yes, most Americans would (I hope). Ask ChatGPT...
4chan is still a thing, the library is still a thing. The nuclear boy scouts made a giant mess before there was an internet.
As for safety, well, it's gonna be a long time before shutting down the world isnt a weekend project for a hand full of people if they are allowed.
DEI has definitely become a political trope, but regardless of what Newt Gingrich or any other talking head might say, there are plenty of valid critiques that can and should be made of the field if its going to get back on track.
I raised DEI as an example of a system that, in my opinion, is further down the path that I see AI safety going down. They both started with strong intentions to help guide or redirect an industry but, again in my opinion, both fields have ended up playing a role of PR and "cover your ass" more than anything else.
It's interesting that's how it looks like to you from the outside.
If you look a little closer at the "AI Safety" community, it consists of two competing factions: the faction responsible for the Bard debacle, and the faction that's focused on trying to prevent human extinction -- "AI notkilleveryoneism", as in https://x.com/aisafetymemes
They both insist that the other faction is "distracting everyone from the real issues".
Because in part, it's a false dichotomy. AI safety just is. It's like airplane safety; we can write all these cool guidelines and track people from hell and back, but there's still a degree of determinative capability that pilots assume when they fly a plane. Same goes for operating AI; the onus of not using it to kill everyone falls on everyone, not one person.
Both sides are distracting each other because neither of them have anything to support without their valueless tribal politics. AI research operates independently of their discourse, and gets deployed without ever consulting any of them. They are armchair experts at best, and stoop to being Twitter reactionaries when they demand respect from their core audience.
The only dichotomy that exists in AI is the stuff that gets made and the stuff that doesn't. I say this as someone that despises the field by now and wishes it never existed in my lifetime; if you don't make it, they will. How's that for an AI safety policy?
Astonishing claim, when some such researchers have (unfortunately) been responsible for many of the most impressive capabilities advancements in the last few years, and the three most cited AI researchers of all time are all doing extinction-risk-mitigation work (with Bengio and Sutskever both doing technical work; Hinton mostly seems to be focusing on outreach).
Note that this is not a valid argument against any argument that superintelligent AI might kill everyone; it's just a character attack.
That’s why we don’t let a random Joe fly a 747, there is extensive training, licensing, etc.
Do you envision the same for operating AI? In the real world you can’t even drive a moped without licence, registration and insurance. Same goes for access to dangerous chemicals. If AI is dangerous, this is the logical conclusion
The same is going to happen with AI. There will be bad actors, and trying to stop them from using AI for whatever "hostile" purposes it might yield is going to be nigh-impossible.
The idea behind that was people would go on to pursue more creative or artistic jobs that don’t prioritize profit after being given a basic income to live off.
Unfortunately it won’t work out that way: We see now that AI will mostly take over creative and artistic work and people will have no motivation to pursue art. The only jobs left for humans are hard physical labor and no one seems excited to do that… so you’ll just be giving people basic income to sit around and do nothing all day.
Having a society of mostly idle people that cost you money doesn’t really sound like a great or sustainable idea, nor does there seem to be any benefit to society.
So what are we going to do with all of the excess humans once human productivity jumps one, two, or three more orders of magnitude?
/s
The other half is me dead scared that he is right. LLMs/GPTs != cryptocurrency. There are actual use cases other than money laundering. If you are paying attention, then the insane scaling in capabilities should have us all dumbfounded. I agree with Linus Torvalds: "just" predicting the next token is not an insult, that's mostly what we are all doing.
Human productivity/the global economy has been growing at an exponential rate[0] since the industrial revolution. In our lifetimes we have ridden the close-to-vertical part, and if GPTs continue to scale, then it will continue to near-vertical. This will be near-post-scarcity society. Capitalism will soon have become so successful that it will make itself obsolete. Are we ready for that, politically? Most of our ruling a-holes maintain power via scarcity, how will they react?
Culling might not be an action, just an apathy. The transition is going to suck as the poors now will also have taken advantage of that hockey stick of asymmetric warfare aka "productivity" via things like ML powered kill drones. This tech is happening now, in 2024-25, in Ukraine.
We are at a crossroads, more than when horse-driven carriages turned horseless. The humans are the horses now, and they will not be happy.
We really should have worked out global civility by now, given the risks of truly global destabilization.
[0] https://ourworldindata.org/grapher/global-gdp-over-the-long-...
Not even close.
Take something like the iPhone. Everyone will want a pro max latest version. Great we can make 8 billion of them. How do you hand them out... we have to get in line I think.
The person at the back of the line, looking at 8 billion assholes in front of them is gonna get stabby...
The chart you're pointing out is showing us that an increase in productivity and/or creativity only will serve to exacerbate disparity, not improve it. Who goes first when there are more firsts is going to be a bigger problem.
Unfortunately while this is very likely the case, the vast amount of money being poured into these projects will also seep into other horrifying projects like: Lavender or Pattern (seriously go look up both of these, it's actually going to fry your brain).
Make of that what you want.
Sometimes it pays to look at the fundamentals. https://www.lesswrong.com/tag/ai-alignment-intro-materials
No, they're still the "name brand" option for commercial LLM stuff.
I've never seen any CEO just straight out diss their own products.
He was good at always saying the current version selling is the best, never a future version.
And for windows 3 and windows 95, he definitely called the old versions trash.
In comparison, they were.
That is the opposite of criticism. It all translates to "GPT-(n+1) is going to be super awesome".
I am frustrated at the pure-hype tone this week took on. There are otherwise intelligent people with strong beliefs that OpenAI "has AGI". It's _ludicrous_ and embarrassing. There are people generating entire philosophies predicated on OpenAI just quietly sitting on a profound technological leap.
Even their missteps are explained as actually good. We've seen some regressions on gpt-4o performance. But that's OK, because it isn't supposed to be smarter, because of its very aggressive claims about future features, but they can distract with fearhype, never release or nerf them, and cite unverifiable safety concerns.
Google released some impressive, useful things this week. I really want to believe that oai's striking claims of simultaneous generation-leaping features, reduced cost, and wide scale will come to pass. But if that tech demo was the equivalent of a concept car that will never really see production, I'm looking for a new AI hero.
"We know AGI is risky, and we've stated that while also shipping new products every 6 months. We hope you are excited by the recent announcement that OpenAI is looking into how to allow our customers to responsibly generate erotica, and want you to know that we are doing this with full concern of the risks and potential outcomes. We look forward to offering multimodal interactions with models so life-like that you may not realize you are interacting with a computer at all. And rest assured, we've fully thought through and planned for the potential dangers of such a future."
I do agree with you there though, I can't think of any meaningful risks of text-only erotica unless you consider the psychological impact of the person choosing to engage with it.
Better question, how the heck do you see something like the ability for anyone to make extremely convincing, fake erotica and not see at least some risk in that?
Not saying they aren't real risks... Are there going to be sad incidents where innocent people will be harmed? Yes. The question though is if there will be enough of them, and egregious enough, for us to disregard the upsides of technology that is not heavily regulated.
Regulations won't fix a damn thing and regulators have no chance of keeping up even if they wanted to. We need these tools to not be developed or offered at all by choice. Unfortunately that really means either the company, investors, or customers need to make a moral decision at the cost of economic gain or convenience/novelty.
To be clear, the context of this thread is extremely important. I don't know how far our existing laws might go to make sure ML generated child porn is illegal and I'd personally feel much more comfortable to know that ML generation isn't some kind of legal loophole.
I assume that training separate models for separate jurisdictions would be rather expensive, so probably unlikely.
There’s nothing specifically wrong with it, but there are “legit” companies and “kinda shady” ones, and when you start hanging out with a bad crowd…
Sad to see.
I feel like it really completes the fall from grace for OpenAI.
I do not trust them.
hey hackernews, your shows are on
Besides all messaging is directed not at the plebs, but at the elite (those who own everything) signalling a simple thing - don't worry - what ever flares up we are on top of things - we will firefight to death for you, so you just have to sit back on your fat ass and watch your net worth multiply on a cool dashboard - we will pander 24x7 - distract/bribe/lobby/suck up limited attention - your cash is safe with us - keep sending us more so no one can compete with our "free" shit and we can achieve market capture asap.
The day someone finds a way to kill the parasites is the day you have to pay attention. Until then nothing to see other than the host getting drained.
still has the occasional diamond though.
I think it's just human condition to lean towards charismatic "leaders". Like, they're like you wanna be so...you love them.
Absurd amounts of money.
Also, highly transportable, the models themselves are not overly massive data sets.
So like anything that can be used as a weapon, it’s potentially dangerous.
But instead of thinking about misuse and misappropriation for war, they gave us DEI bullshit.
It was a marketing campaign. “We’re super cool. We’re super advanced. We hope don’t make the terminator! wink wink” “Oh no! GPT-2 is too dangerous! We better not release it! Oh wait. We did. ¯\_(ツ)_/¯ “)
It was never a serious thing. Sure here and in the depths of weirdo forums people whispered about the AGI that can’t be named — Roko’s Basilisk. It was just comically stupid. Pascal’s Wager with extra steps.
Now OpenAI has tons of money and everyone wants to deal with them, so they don’t need that hype machine. Instead they’re saying stick their “dangerous” models in your toaster Maximum Overdrive style.
It was transparent
Yeppppp. This is what i was saying for weeks and weeks, if not months. The amount of people on Reddit who can not see the puzzle pieces of “OpenAI sells AI” , “AI is hyped” , “more hype = more attention = more money and media coverage” and put them together, is actually shocking. Every time i would point this out on Reddit, more often than not i would get downvoted, and the responses were basically “well actually, AGI is around the corner”
This community is honestly SO MUCH BETTER than a lot of futurist subs. I might just stick to this site for a while.
That said, I’m kind of glad to hear that Reddit is worse, as it puts the situation here on HN into some wider perspective (it’s not that bad here).
[0]: meant in the form of: closed minded absolutism without full contextual appreciation or understanding of the domain (what you seem to be describing as happening on Reddit)
[1]: less so these days, I wonder if that has something do with the current AI hype cycle cough opportunistic bandwagon jumping cough o_o
I too don't think OpenAI will ever create an actual AI ("AGI"), but they think they might, and that one might kill humanity; trying to achieve it makes every person working at OpenAI just as reprehensible as those who put the nuclear sword of Damocles over us.
https://www.safe.ai/work/statement-on-ai-risk#open-letter
What do you know that Geoff Hinton, Yoshua Bengio, Demis Hassabis, Bill Gates, Stuart Russell, etc. don't?
It’s fashionable, for reasons I am unaware, to hate on Sam Altman and OpenAI. Seems to be mostly the doomer crowd and it’s Blake Lemoine types (boy do his claims look ridiculous only 2 years on). To my thinking the open source crew poses a greater danger.
No. It only indicates that OpenAI is less technically capable than it claims. To judge OpenAI's competence at safety, you have to look at the ethics of their behavior, since building the right ethics into an AI before it gets powerful enough to do serious harm to humans is the essence of AI safety. And the ethics of OpenAI's behavior, to put it as gently as possible, does not look good.
Plenty of other posts in this discussion give other examples.
That's without even getting into all that happened with Altman leaving and then coming back, which has already been discussed to death in other HN threads.
Sure it does. Particularly for a company that claims to be a responsible steward of a technology that could pose existential risk to humans. If you can't even do a simple thing like keep your promises, how can I possibly trust you as a steward of a potential existential risk?
> WRT to Altman’s ousting and return, I still haven’t seen any evidence of compromised ethics on his part.
I commented on this in a number of previous HN threads and don't want to rehash it all here. (And btw, I also commented that I didn't think the people who ousted Altman showed very good ethics or judgment either. I didn't think anybody came out of that whole brouhaha looking good.) But just the fact that it happened at all should be a red flag. Again, these are people who claim to be stewards of a technology that they say can pose an existential risk. You'd expect them to at least be able to cooperate with each other like responsible adults.
> A supermajority of OpenAIs employees, people who I’d venture to guess have a far better handle on Altmans ethics than you or I
They might well know Altman's ethics better than we do, and be perfectly fine with them, because their own ethics are just as bad. They're getting paid well and the fact that core promises of the company were broken is simply Not Their Problem.
Tech people who don't like OpenAI have got to come up with a better line. Everytime I mention to someone normal that this is a complaint immediately eyes roll.
No one says "North Korea is bad because they call themselves a Democratic People's Republic despite having a dictator" because that is incredibly not important nor was it ever expected.
What's more valuable for the normal person, Lllama, that I will never be able to use because I don't know how to get it to run, or ChatGPT that I can use for whatever I want?
For whatever openai wants which is a shrinking set.
I have a rock that keeps lions away. I know it works, because I haven't seen any lions.
Genuine question, do you care to know? Or are you only asking this question as a prelude to delivering insult?
When you ask why people believe X in one breath and then suggest it's because of "petty jealousy" in the next, you'll succeed at conveying your dim view of them but fail to create a conversation where it is possible for you to learn something about why people think that way.
Jealousy is a particularly slippery hypothesis because it is fully generalizable. You can explain any criticism of anybody as jealousy. It is my observation that people often accuse others of jealousy when they want to dismiss a perspective. In that way, it functions less as an explanation than as a defense mechanism; an escape hatch from the uncomfortable feeling of simultaneously disagreeing with someone and acknowledging that there perspective might not be meritless.
Perhaps because he's a slimy used-car salesman who got rich doing things that are illegal for you and me (ripping off copyrighted works en masse) and he pulled a bait and switch as far as for-/non-profit is concerned.
He's everything wrong with Silicon Valley/Venture Capital/faux meritocratic culture rolled into one.
Look at Facebook and other social media: at first we all thought it was great and then we realized it's a giant depression machine, and all it's really good for is giving young girls eating disorders.
But since we're on the topic, the immediate casualties seem to be the livelihoods of authors and artists, on whose content the LLMs were trained and who received no compensation.
And now every publisher is inundated with infinite ChatGPT-created books. We won't get into the climate impact of wasting all that compute just to create a better phishing-email generator.
Maybe because they ditched the Open in OpenAI and sold out to MS and now Apple too.
For the benefit of all mankind was replaced by "Show me the money"