Sam Altman isn’t the answer to regulating artificial intelligence
bloomberg.com
bloomberg.com
Why? Because they see the writing on the wall. There’s no network effect here. Anyone can build AI. There are thousands of students going into grad school to study this stuff. The market is going to be flooded.
All of their head start will evaporate unless they can build a moat around it to keep new entrants out. What better way to do that than to have your rockstar pitch man walk into Congress and have them eating out of his hand? They practically begged him to head up a new regulatory agency!
aw heck, let's go so far as to say anyone willing to testify if front of congress primarily does so because they want to protect the best interests of others.
Would be a fucking swell world to live in.
So.. you're saying that every person that testifies before congress has bad intentions just because they testify before congress?
Say he did want what was best for humanity, how should he have acted differently in front of congress?
To me, from every interview I've heard from him, he doesn't seem like a Zuckerberg.
Don't assume they're they're to help. Don't assume they're there to hurt.
They are there to cover their own asses and nothing more.
Why is the assumption that they are only there to cover their own asses ok to make but other assumptions are not?
We don't know what he is actually thinking. It is entirely plausible that he is not a sociopath and actually wants to minimize negative impacts on society through regulation because he thinks that is the best path forward.
"Fancy RNN" is a pretty ridiculous assertion.
And no, researchers didn't expect what GPT-3/4 has been shown to do to be around the corner at all.
GPT's aren't chatbots. That's just a neat natural consequence that's happened. They're machines that reason, understand and follow instructions in plain language. And their abilities go far beyond what any expected language modelling to provide.
They are not machines that reason, they are approximators. It’s all just token matching based on data we’ve fed it. Further it didn’t happen over night but through successive improvements and at no point was the next improvement considered some infeasible thing.
Seems to me maybe you’ve bought into the hype here, and are confusing that with reality.
And meaningless distinction of the year award goes to..
"It's not real [insert property]" is not an intelligent argument. By all means, divine the way to distinguish results of the two. After all, what kind of important distinction can't be distinguished by results?
>Further it didn’t happen over night but through successive improvements
The only difference between GPT-3 and GPT-2 was scale. They didn't even change the tokenizer until 4. There were no "successful improvements" to smoothen the massive gap in capabilities between the two. So to say that was expected just shows more lack of knowledge here.
"Other than that, how was the play, Mrs. Lincoln?"
GPT-2 is two orders of magnitude smaller. In terms of forebrain neuron count this is the difference between a human and an elephant shrew or budgerigar. It was absolutely expected by reasonable people that 2 OOMs of scaling will provide a qualitative jump.
When GPT-3 was released, it was by far the largest artificial neural network ever trained. And I mean by far. Now there wasn't any big jump in hardware capabilities to spur this sort of gulf. It wasn't a case of "Oh now we can train a very large model"
So Want to know why there was such a gap ? It's because most researchers assumed the models would overfit the data or display diminishing returns long before 175b.
Brain neurons are not comparable to ann parameters.
They approximate a function that performs reasoning...
> Conclusion: computer power is unlikely to be the issue anymore in terms of AGI being possible. The main question is whether we can find the right algorithms.
> One of the big things influencing me this year has been learning about how much we understand about how the brain works, in particular, how much we know that should be of interest to AGI designers. I won’t get into it all here, but suffice to say that just a brief outline of all this information would be a 20 page journal paper (there is currently a suggestion that I write such a paper next year with some Gatsby Unit neuroscientists, but for the time being I’ve got too many other things to attend to). At a high level what we are seeing in the brain is a fairly sensible looking AGI design. You’ve got hierarchical temporal abstraction formed for perception and action combined with more precise timing motor control, with an underlying system for reinforcement learning. The reinforcement learning system is essentially a type of temporal difference learning though unfortunately at the moment there is evidence in favour of actor-critic, Q-learning and also Sarsa type mechanisms — this picture should clear up in the next year or so. The system contains a long list of features that you might expect to see in a sophisticated reinforcement learner such as pseudo rewards for informative queues, inverse reward computations, uncertainty and environmental change modelling, dual model based and model free modes of operation, things to monitor context, it even seems to have mechanisms that reward the development of conceptual knowledge. When I ask leading experts in the field whether we will understand reinforcement learning in the human brain within ten years, the answer I get back is “yes, in fact we already have a pretty good idea how it works and our knowledge is developing rapidly.”
> I suspect that for the next 5 years, and probably longer, neuroscientists working on understanding cortex aren’t going to be of much use to AGI efforts. My guess is that sometime in the next 10 years developments in deep belief networks, temporal graphical models, liquid computation models, slow feature analysis etc. will produce sufficiently powerful hierarchical temporal generative models to essentially fill the role of cortex within an AGI.
> Right, so my prediction for the last 10 years has been for roughly human level AGI in the year 2025 (though I also predict that sceptics will deny that it’s happened when it does!) This year I’ve tried to come up with something a bit more precise. In doing so what I’ve found is that while my mode is about 2025, my expected value is actually a bit higher at 2028. This is not because I’ve become more pessimistic during the year, rather it’s because this time I’ve tried to quantify my beliefs more systematically and found that the probability I assign between 2030 and 2040 drags the expectation up. Perhaps more useful is my 90% credibility region, which from my current belief distribution comes out at 2018 to 2036.
And here's Rich Sutton's famous Bitter Lesson, a month after GPT-2 [2]:
> We have to learn the bitter lesson that building in how we think we think does not work in the long run. The bitter lesson is based on the historical observations that 1) AI researchers have often tried to build knowledge into their agents, 2) this always helps in the short term, and is personally satisfying to the researcher, but 3) in the long run it plateaus and even inhibits further progress, and 4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning. The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach.
> One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.
Surprise indicates the mismatch of your mental model and reality, not the inherent weirdness of the latter. Both the surprised/alarmed people and people still in denial about the power of LLMs have to revisit their assumptions and ask if they were founded on any credible understanding to begin with.
1. http://www.vetta.org/2009/12/tick-tock-tick-tock-bing/
2. http://www.incompleteideas.net/IncIdeas/BitterLesson.html
When GPT-3 was released, it was by far by the largest artificial neural network ever trained and not because of any big jump in hardware technology. That's not the usual state of affairs for technology everyone or even most expect to pan out the way it did.
I believe he disagreed with the direction the board was taking it and tried to take over, which when failed he stopped funding it, and they went to investors
I have been running ML on my personal data and modeling worlds for the last few years. There’s no reason to keep OpenAI around given open source. My own data does not need a river water cooled data center.
OpenAIs ONLY moat is a government one so governments can keep up the free market ruse but have OpenAI in its back pocket for military and intelligence applications
It's nice to think that you can have so much money you stop caring about it but it doesn't really make sense. You don't get to personally keep billions of dollars without caring about money.
> It's nice to think that you can have so much money you stop caring about it but it doesn't really make sense. You don't get to personally keep billions of dollars without caring about money.
It is definitely not every billionaire that thinks like this but there are some that seem to not care too much about material possessions like Musk or Sam. Their endeavors seem to be the things that they care about the most and they don't seem to be doing it to attain the most amount of money possible.
I don't know that I agree with this strategy, but I can respect that it's a consistent viewpoint.
So what I hear from him is "Yes, this is dangerous stuff, and we're trying our hardest to make it safe, but we can't control everyone, so we need regulation ASAP because market forces are going to push everyone to race and release unsafe AIs".
> I can respect that it's a consistent viewpoint.
I don't think I agree that there's a great deal of consistency between that stated perspective and what OpenAI is actually doing. It's also, conveniently, a point of view that allows him to keep doing what he wants to do, and making bank by doing it.
Understand, I'm not saying he's a bad actor. I'm saying that it's hard to rule that out. From his WorldCoin stuff to this, there is plenty of reason to be suspicious of his motives.
I wonder if government will end up enacting any legislation or this scrutiny will blow over until something egregious happens (e.g. Cambridge Analytica). They've shown zero willingness to legislate any consumer data privacy laws so far.
This is what I find so hilariously suspicious about Elon Musk's sudden anxiety about the dangers of AI chatbots. Like, dude, you've spent the better part of two decades making cars that drive themselves and have _already_ killed people with their faulty, ironically oversold "AI." Now all of a sudden someone else's AI, which isn't in the drivers seat of vehicles capable of 100mph+ speeds, and which actually works, is an existential danger to humanity?
I can't see how one could do the regulating of AI but I don't think one can limit the worry-area to "clear and obvious" situations.
Which is fine; engineers are not divine beings. They can have their figurative value deflated like everyone else is these days. Engineered devices have killed a lot of people.
Name three. There are 50,000 gun related deaths in the United States per year, are any of them a bigger threat than gun violence? I struggle to find a single thing AI can do, sans being hooked up to nuclear weapons and time travelling robots with german accents, that's even imaginatively more dangerous than what this country already allows and decides doesn't require useful regulations.
I'm sorry if that's too political for HN but you know what, it actually does matter to compare real dangers (car accidents, gun violence) to conceived and made up ones (gender indifferent bathrooms, AI) when deciding how governments should prioritize policy and regulations.
> I struggle to find a single thing AI can do
How about job loss? If AI can do what proponents say, then it's easy to see that a large amount of job loss will result. If even 20% of people can no longer earn a living, people will die.
This is called creative destruction, and it's not even remotely a new problem created by this particular technology, it's a constant and ongoing mechanism of market capitalism by which technology streamlines or improves something that then requires less labor to accomplish.
When the 400k+ telephone switch operators lost their jobs to digitization, it didn't cause a mass die off of humans, and while I won't say it was a zero bad consequences event, they mostly found other work, work that was probably more interesting and more productive. We certainly could have regulated telephone digitization to prevent this "danger" and instead employed 4.5 million people, or about 3 percent of the labor force, to operate the phone network today as it would have existed technologically in the 1970s, but I'm happy we skipped the scare mongering and went the digitization route, which probably also was a prerequisite for the internet to be able to be used by.. users (which mostly used digital phone services via modems to connect). Would even consumer internet access exist today if they decided to regulate the "danger" of losing analog phone operator jobs?
Either way, I fail to see how allowing only licensed and government regulated companies to work on machine learning will do anything to improve anyone's job prospects moving forward, as increasingly onerous requirements created by people that like to say things like "the internet is a bunch of tubes" make it so only a few rent-seeking monopolies will be able to do anything (legally) with the technology.
I'm not 100% against government regulation, but it's early, things are relatively benign so far (sans self driving cars), let's wait until there's some actual non-sci-fi concerns that emerge and then tackle them as specific things to regulate, rather than wrapping the whole thing into a giant regulatory framework that incorrectly calls it AI before we even learn what the problems are going to be.
True. But this technology (again, if the proponents are correct) threatens to do it on a truly unprecedented scale. It was bad enough -- barely tolerable in some cases -- in history. Getting it even worse seems like something nobody would want, let alone cheerlead.
I am deeply, deeply concerned that there are people willing to gamble with the lives of innocent others (not to mention society itself) like this. It's something we need to be addressing now, before it happens.
It remains to be seen if this is ultimately a real problem. If it turns out that it is, then not regulating the development of these systems in advance may be a huge mistake.
If there's a possibility of an asteroid coming that will wipe out life on earth, waiting until our telescopes can see it to start taking action may not give us enough time to react. We have to plan for the theoretical possibility.
Maybe we'll never be able to create artificial superintelligence. Maybe artificial superintelligence will be much easier to align than we thing. Maybe it'll be hard, but manageable, possibly because we have more time to work on the problem than we think.
All of those are possibilities, but the fact that if you ask 100 people about it, you'll get 100 different opinions, means that we have very little idea how things will go. Given that, I don't like the approach of wing it / hope for the best. If we end up stifling AI innovation unnecessarily, that may mean that our techno-utopia is delayed for a few years, but we get there in the end. On the other hand, if we under-react and things turn out really bad, we may not get another try. Scenarios where AGI kills a billion people are really bad, but scenarios where we create it and end up permanently disempowered or dead are worse.
I wonder sometimes if people are just really dissatisfied with their lives, that they react with such vitriol to the idea of AI progress being delayed. I know we have a lot of problems, but how many of them are technological vs. having poor human institutions and incentive structures? AI probably won't help much with that.
How would we even know what to do until we have a concrete target?
I think it's true that it may be very difficult to solve alignment theoretically, and solving it when we have real-world systems to look at may make it easier (but still probably not easy). But we need to be able to buy companies time so they can slow down as we approach what seems to be the "edge" of dangerous capability / consciousness / goal directedness, without feeling like they will lose out. And to be very strict about the conditions the models are trained under and what testing they undergo before release.
A lot of this is stuff that people are already suggesting, but not everyone is taking the risk seriously.
But sure, most of the concern about AI is about stuff people imagine happening in the future. So the question is whether future safety concerns are worrisome enough to want to get a head start on regulation.
Also, it’s not just deaths. Generative AI is, among other things, potentially an unlimited supply of disgusting imagery. That’s likely to be a big legal headache for lots of organizations.
We are not bothering to regulate AI controlling a 3500lb death machine that goes from 0 to 60 in 2 seconds but we are worried about the dangers of a chatbot.
So typical of our society to be captured by the more compelling narrative and ignore actual reality.
AI systems mis-identifying people and getting them arrested - minor problem. Systems firing people for underperforming - that's just a company exercising its right to enforce "at-will employment". Chatbots designed to make it difficult or impossible to cancel a service - almost standard. Announcing a hiring freeze while jobs are replaced with AI systems - that's productivity.
But a system that will answer "Rank races by intelligence" gets people very upset.
So what is meant by regulate ai is amend existing laws for the age of ai?
Even if you're nearsighted you can see how the internet will in short order be rendered useless by fake real-time interaction and digital malice. Look at how much automated ssh poking you get on an IP address. Look at how much email you receive without a comprehensive spam filter (it's much more than shows up in your Gmail spam folder). If we combine that level of relentless volume and speed with moderate skills at hacking, fraud, influence campaigns, abuse, and so on, the internet will break.
Look at the all the plugins GPT-4 has already. Do you really think if a system with those capabilities, with evil intent, running millions of threads, wouldn't cause untold levels of harm? If not, then at which point do you push for regulation? Do you wait until anyone can build a literal terminator in their basement? Openai is already working on robotic control models, so that's not likely not far off.
All I see are mindless silicon valley tech-optimists who can't see the forest for the trees.
You want to further regulate what I can build in my basement?
No, but other countries have. We just don't do regulation because we're America or something (and because the FCC has been gutted). No European country has the same level of spam phone calls that we do.
Good. What would be even better if AI researchers and engineers applied en-mass to be the in-house technical advisement staff for congressional members and executive/congressional regulatory agencies for AI.
1) he is truly scared AI can become dangerous and try to distribute the responsibility to more people 2) he anticipates that there will be some regulation anyway, and when that's the case, it is better to be involved and act friendly.
"...development of an AI that could end human life may only require a few hundred people in an office building anywhere in the world, with no equipment other than laptops.
The new existential threats won’t require the resources of nations to produce."
and
"The fact that we don’t have serious efforts underway to combat threats from synthetic biology and AI development is astonishing."
So (2) could be a factor, for sure. But (1) is consistent with what seems to have been his viewpoint for a long time. It's possible he's updated his viewpoint and is faking concern about safety, but given how recent advances have seemed to make a lot of people more concerned, I don't think that's likely.
Also, sitting right next to him was Gary Marcus, who doesn't have a dog in the fight as far as I know, and he agreed with a most of the recommendations.
You can count on your hands the number of high end chip manufacturers and GPU companies, as well as the number of companies with existing high performance clusters. Treat the production and use of datacenter GPUs the same way we treat enriched uranium. We've already started since database GPUs were added to the export control list.
Not by a longshot. See cryptography before the '90s for the most recognizable example.