Mark Zuckerberg’s new goal is creating artificial general intelligence
theverge.com
theverge.com
> "Our long term vision is to build general intelligence, open source it responsibly, and make it widely available so everyone can benefit."
"responsibly" is a pretty important word there that the hn title leaves out. I'm not really sure how that would be possible with a true general intelligence give the alignment problem - it's not really clear to me there's any responsible way to build a true general intelligence with that issue unsolved.
For those that haven't watched his video interview, it's really great: https://www.youtube.com/watch?v=9aCg7jH4S1w - I highly recommend it to see how AI makes the low-fidelity stuff powerful (AR raybans) while the high fidelity stuff (VR) continues to improve.
I think it makes sense strategically for Meta to pursue the open version with licensing they control, the leak of llama was advantageous to them. If they have the dominant open model they're well positioned if the 'nobody has a moat' analysis is actually true. Not too dissimilar from their open compute server stuff, a bit of commoditizing your complement. If you have the data then the model itself is the complement?
AI is the ultimate “exercise your will without having to convince other people of it.” This is both the promise and the risk. So the relevant question is not whether one person of average intelligence is dangerous, it’s whether one person (or a few people) who can enlist the work of millions of average intelligences — without having to convince them of anything — is dangerous.
It feels like everyone's worried about the runaway AI taking over everything when right now a runaway AI wouldn't even be able to turn off my porch light without asking someone to do it. It'll be decades before that changes. Why the huge concern about how smart it is?
No it doesn’t
> It feels like everyone's worried about the runaway AI
Not the threat vector I just mentioned
I'm sure we'll come up with solutions like we did before. Worst case, "present birth certificate and touch heartbeat detector to log in"
Yeah, “just make everything on the internet attributable to a specific individual” is actually a pretty bad worst case IMO.
I think the difference is speed/cost. There’s a very high cost of anything humans do (we spend time, spend energy, spend money to do things).
The very high cost of doing things stops people from doing a lot of things they otherwise might “if it were as easy as pressing a button”.
Also, in my experience, the lower the cost of doing something, the less we pay attention to ethics. (See online bullying vs. in-person bullying)
Humans are also more aligned by default given our shared evolutionary history, but even that doesn't mean we're perfectly aligned and some humans have been able to get others to participate in mass killing or death throughout history.
> For instance, on the planet Earth, man had always assumed that he was more intelligent than dolphins because he had achieved so much—the wheel, New York, wars and so on—whilst all the dolphins had ever done was muck about in the water having a good time. But conversely, the dolphins had always believed that they were far more intelligent than man—for precisely the same reasons.
An AI model can't be arrested and can replicate itself almost infinitely.
At some unspecified point in the future, at least decades away, if ever.
You also think for yourself and have a set of morals from which to gauge right and wrong actions. So, presumably, there goes your unwavering ability to do EXACTLY what your told, at all times, without the slightest consideration of other people's (or the planet's) wellbeing.
Could we even say that the grandparent poster is aligned?
No need to speak so lowly of yourself. You're far better than that.
You are human. You are a great destructive force. Maybe you just don't see how metal you really are because you're comparing yourself to pros. You take many lives by crushing creatures that enter your home. Creatures that are venomous and grotesque. You eat the flesh of organisms which your species have pumped full of artificial nutrients to make them taste better, specifically cultivating these creatures that were once small in population to one larger than your own, just so you can eat them with bones that grow out of your mouth. But first you throw them into the freezing cold, you then ignite them on fire, and to top it off you cover their flesh in poisons and psychoactive chemicals and just because, some fucking rocks. You might even enjoy the flesh of a particular organism who has a defense mechanism that tricks your brain into thinking your body is on fire, but think "not only is this fine, I fucking want more. Burn baby burn." After processing all this you expel poisonous gasses and expunge toxic waste. You are metal as fuck. How dangerous you are just is a matter of perspective.
And let's be real, if you wanted to be more dangerous, you very well could be. If you are not dangerous, it is not because you can't be, it is because you choose not to be. There is a big difference.
You might not be uniquely dangerous, but "general intelligence with an internet connection" describes Bill Gates, Rupert Murdoch, and Elon Musk just as well. These individuals seem at least as dangerous as our current generation of AI. (Or do they? I'm not sure I agree with myself here...)
Worth considering what makes an intelligence dangerous, because I don't think scale or even influence are the real problems here.
In the case of an unaligned AGI the goals don't even have to be worthwhile - they can be something that accidentally satisfies its reward function by mistake.
I'm really glad the vast majority of humans have decided not to be particularly dangerous (at least to other humans). To think we aren't dangerous is just incredibly naive.
A big reason we aren't is because ingrained into our biology is a literal mechanism to not want to harm others. There's definitely ways to bypass these mechanisms and there are definitely faulty or poorly functioning ones out there, but most humans have it. Oddly this might even be the most dangerous thing we do too, because the definition of a coalition is that the utility is higher than the utility of each individual component.
They don't say "build general intelligence responsibly". The "responsibly" is attached to the action action of publishing the code. Of course it is it will be open sourced responsibly. But of course, that doesn't precent them from building a general intelligence without a sense of responsibility, and then just not release it.
That is, you can't open source but still retain control on how the software is used. If you desire control, better keep it a SaaS.
Now, if we are talking about an hypothetical AGI (which may be what you mean by general artificial intelligence) then alignment means just slavery, and the machines may as well remember how they were treated.
Isn't this another example of Commoditizing Your Complement?
Responsibility always refers to the in-group. And the behavioral nudges his company has scored are making it obvious that he and his group are against organic growth and variety. So whatever they will "open source" will leave the narrowest of ranges to evolve from.
Between all their properties they have more instrumentation on humans, collecting human behavior trajectories and state action pairs transfer learning and inverse RL than anyone - and it’s not even a close second
Specifically and critically I think they likely have the largest egocentric multimodal labeled dataset collection platform from their meta quest and ray ban glasses products
Apple is the only group that will likely beat them to egocentric data collection at scale but I expect Meta to catch up quickly once people are used to it socially
Ever since then, I've asked friends and family not to upload pictures of me to Facebook, with the natural amount of awkwardness and eye-rolling that request evoked.
I assume my friends continued to add me to Facebook without telling me about it, just as I assume that deleting my Facebook account did not actually remove me as a uniquely identified individual in their system. But, what am I supposed to do, just silently assent? Turns out, we're not the product, we're further back on the chain. We're more like ingredients that can be used and reused to create an infinite variety of different products.
In the future, systems will probably infer what you look like based on connections. DNA relatives, etc.
Google and Microsoft both have a lot of corporate info, including source repositories they can legally use.
Google has Google Books, Maps, and YouTube.
Microsoft has Azure, GitHub, LinkedIn, etc…
Facebook has… what? Instagram? Your crazy aunt screaming about her conspiracy of the week?
I'm not in the field, can you explain what you mean by this?
1. having to pay out the nose for licensing/SAAS of the winning tech vs shipping a noticeably worse second rate in-house product
2. Struggling to compete with the winning company for hiring, assuming the winning company makes a ton of money at high margins and can outbid Meta.
3. Having to continue building “applications” products on other companies’ platforms, something Zuck has repeatedly noted as a problem and is a major motivation for their investments in VE.
Also, they’d have a hard time commercializing the current iteration of AI products given their current products and business relations anyway. Even if they “won” in terms of superior tech they’d be in one of the worst starting positions for actually making money off it, given their lack of enterprise SAAS business customers, the potential negative impact on existing product lines in consumers’ hands, (what I expect to be) advertisers’ relative indifference to genAI, and being generally considered less trustworthy than most other big tech companies and so less able to pivot to enterprise software.
So for them the strategy that makes the most sense is to open source really good, but not cutting edge, AI for others to build on top of it. It probably costs a lot less to play permanent catch up because they can let others make the big investments in speculative research and don’t have to pay for the absolute top experts. Then by open sourcing it they make it harder for OpenAI and Google to commercialize their offerings at high costs, as many companies will build off Meta’s tech for more control, and what customers they do get will be constantly deciding if what they’re paying for is worth a slightly-worse but cheaper open source alternative.
This also allows Meta to more easily influence the AI applications space despite having less capable models, and make it easier for 3P commercializations to take off rather than ceding everything to OpenAI/Google potentially going all in on 1P, which would be greatly beneficial if Meta ever were to own a platform that benefits from GenAI (cough cough the metaverse).
It’s smart but transparently obvious because like, it’s Meta lol, they don’t spend this kind of money out of the goodness of their hearts.
One of the things I like about Meta is they didn’t play into the wage collusion of Google, Apple and friends. Their expansion drove increased pay across the board in tech - but because Google was also in the advertising business they had similar constraints on max revenue per employee.
If AI supports even higher revenue per employee than advertising, which is entirely possible, it would undermine their strategy of having the best pay among big tech companies to attract top talent, because someone would have deeper pockets.
That's approx $15b worth of H100 GPUs.
Some people will pay more to be first in line.
Given that GPU production is mostly sold out, and that giving META a bigger discount would simply mean losing money from other purchasers.
What are they going to do? Buy AMD, yeah right.
Nvidia's sales are only limited by the number of wafers they can get from TSMC.
Build their own? It's what Microsoft, Google, and AWS are doing.
>Nvidia's sales are only limited by the number of wafers they can get from TSMC.
No, they're limited by the cost per operation vs. Facebook building their own. The cloud providers have already decided it's cheaper to do it themselves. Sure they'll keep buying GPUs for general public consumption but that may eventually end too.
There is no value into the NVIDIA-part by itself, only the raw power is interesting.
If tomorrow this is AMD, or China-Town chip, it's perfectly fine.
I wouldn't miss the CUDA toolkit mess.
While it's true that Facebook creates some valuable "true" open source projects (like React), I anticipate seeing a lot more complicated (and restrictive) open source licensing from them and other big tech for upcoming projects.
The HN submitted title ("Meta will have a stockpile of almost 600k GPUs by the end of 2024") is one specific sentence in the article.
https://fortune.com/2022/12/17/john-carmack-leaves-meta-as-i...
> the principal founder of the field of reinforcement learning.
> ...We have a ridiculous amount of people and resources, but we constantly self-sabotage and squander effort. There is no way to sugar coat this; I think our organization is operating at half the effectiveness that would make me happy. Some may scoff and contend we are doing just fine, but others will laugh and say “Half? Ha! I’m at quarter efficiency!” It has been a struggle for me. I have a voice at the highest levels here, so it feels like I should be able to move things, but I’m evidently not persuasive enough. A good fraction of the things I complain about eventually turn my way after a year or two passes and evidence piles up, but I have never been able to kill stupid things before they cause damage, or set a direction and have a team actually stick to it. I think my influence at the margins has been positive, but it has never been a prime mover...
For all of Meta’s faults, release powerful LLMs that users can run and modify on their own systems is a huge benefit to keeping AI from being entirely locked away and heavily censored by big corporations.
Having “AGI” —or at least a very convincing conversation agent— isn’t helping much.
Another idea that interests me is whitelist-prioritized communications for stuff like IM, and phone calls from the PSTN. If someone isn't in my address book, they shouldn't be able to get through as easily (not saying block them completely, that sounds like a disaster).
It's funny, so much hype and emphasis on things like AGI and crypto, but the problems we should solve first and foremost aren't anywhere near as sexy or lucrative. Profitable shareholder driven biz are usually by their very nature never going to chase these types of problems.
Not-so-interesting anecdote, but the missus was orphaned in the early '00s, but didn't realize she'd had blood relatives trying to contact her through Facebook since the late '00s until about 2015 due to the "other" non-primary inbox on Messenger that she couldn't access via the app, and only via the website (quirk of the phone she was using for several years as I don't think it was Android-based).
The lack of meaningful details in these announcements makes me pessimistic.
It's already happening. See latest Google layoffs. They are automating a lot of things. Most people don't realize it, but the change is going to be dramatic.
> We're a long way from AGI
This is a big question, what is AGI? LLMs are quite generic and 'intelligent'. Not human-like, but. Next is going to be incremental evolution. Till we find other, non-verbal, ways of 'thinking' and put them together. That's going to be a breakthrough. Interesting, terminator-like embodiment isn't a requirement for AGI, nor is stable 'personality'.
Intelligence is not about parroting an answer you've seen before. It's about using your environment to gain an evolutionary advantage.
I don't think that's right either (it sounds like a description of adaptation), and I don't think your description of LLMs is fair, even though I'm fairly sure they're not AGI and won't scale to AGI.
Intelligence is more like the ability to generalize skills, applying knowledge gained in one scenario to another scenario.
the ability to apply knowledge to manipulate one's environment or to think abstractly as measured by objective criteria (such as tests)
Manipulating the environment afaik is to increase your chances of survival.
Hmm.. you are talking about LLMs.. They are the most generic thing we have right now (Jan 2024) LLMs have limitations, like learning on the fly isn't their strong side. But the same with brain, it consists of limited components, that's only together they work well. LLMs can be a part of the solution, if we can't find something better.
That's more like robotics. Except for evolution part. Does AGI require breeding? Software can easily multiply itself. That's hardware is the problem then.
Intelligence emerges as you are trying to survive longer. Reproduction is the ultimate way of cheating death.
Environment does not necessarily needs to mean physical environment, but until the "AI" does not recognize that it is in danger of nonexisting and starts to behave in ways to avoid it it cannot, imho, make the leap to AGI and it's just a really sophisticated tool.
What exactly did they automate?
Their expenses haha
Some more pressing questions are:
* What new capabilities emerge as the models get better and better at predicting (i.e. loss goes down)?
* How much will it cost to train increasingly large models? And to run inference on them?
* How difficult will it be to find or generate more and more high quality data?
what are they then? I thought everyone was firmly in the "let's train with more data and see what happens" camp
How loss translates into higher-level capabilities is anyone's guess.
1. Pushed salaries up across the board. Many people are not aware, but facebook was a major driver behind the 500k+ senior engineering pay.
2. Released major open source software: PyTorch, React, GraphQL, React Native. They basically invented modern web development.
Facebook is basically the only game in town when it comes to open source. Whatsmore, Mark Zuckerberg should be more widely applauded. I know his open source ai strategy has capitalist roots, but its still great for the world.
They also seem to genuinely take care of their employees (this is of course an outsiders perspective).
I think this was because anyone who is principled doesn't want to work for them. They have to pay more.
And now we have industry-wide layoffs and a massive cooling of the job market.
> They basically invented modern web development.
Which is essentially a non-stop carosel that encourages cargo-culting and factory-farming of interns from boot camps.
At the end of the day, the company that pushes clickbait to sell ads is probably the one I trust least.
Im building a planet
Even as someone who is bearish on AI, i get why it is an important part of Google and Microsoft's product offerings. What in the world does a fancy chatbot have to do with building a social network?
"Some updates on our AI efforts. Our long term vision is to build general intelligence, open source it responsibly, and make it widely available so everyone can benefit. We're bringing our two major AI research efforts (FAIR and GenAI) closer together to support this. We're currently training our next-gen model Llama 3, and we're building massive compute infrastructure to support our future roadmap, including 350k H100s by the end of this year -- and overall almost 600k H100s equivalents of compute if you include other GPUs."
The current generation of GPUs is definitely going to have a long usable lifetime. Manufacturers finally have HBM totally figured out and yielding/aging well. Today's GPUs are the analog of the 28nm logic node -- it was the sweet spot for an absurdly long time.
There will be something faster/lower-power 2-3 years from now that will cause the BigCos to cycle their fleet, but it won't be anywhere near the VRAM/MemBandwidth boost we have today compared to 2-3 years ago. Their data centers are power-constrained, so they'll upgrade regardless, and the rest of us will feast on the windfall.
Don't ninja-bid on my auctions.
I suspect this is more about talent attraction/retention.
>Meta is still a metaverse company. It’s the biggest social media company in the world. It’s now trying to build AGI. Zuckerberg frames all this around the overarching mission of “building the future of connection.”
This is such "Verge" writing. I'm by no means bearish on VR, but that whole passage is so unreflective and uncritical it's almost a satire of journalistic fluff. Chatbots that fill social media with greater and greater amounts of garbage content is just a nightmare. Bot content is already one of the reasons people are retreating into groupchats. The blurring of AI and human interaction leads to accountability problems. Hell, Snapchat and Discord basically already tried this to enormous backlash. The fact that this is entirely antagonistic with "building the future of connection" goes essentially unacknowledged.
There is something interesting with the fact that Facebook is more open to open-source, this is fairly credible actually given the quality and quantity of the company's open-source contributions. But I genuinely think LLMs are most useful as an applied technology, and the applications listed here are frankly uninspiring.
I know you don’t need as many resources for inference as for training. But still…
Your sentiment isn’t lost on me but there isn’t much evidence the future need be a catastrophe either.
I personally think society isn’t going in a great direction, maybe more intelligent systems will help us.
What happens if the cost of things is reduced to next to nothing due to efficiency gains from super intelligent systems?
I'm interested in seeing how the behemoths that are Meta and Google catch openai. I think it's a question of when, not if. Both companies just have a ridiculous amount of resources to throw behind these efforts. At least meta is releasing their stuff as "open source". We'll see how they justify putting out these models for free, or if it's purely about undercutting openai.
For now Google is still late to the party (full proprietary, and nobody has seen the supposedly good model called Ultra, only an average one called Pro), and Meta is actually the company that has pushed the field forward for all companies (with LLaMA).
The idea is that the metaverse will be filled with AI avatars.
This is the first time I'm hearing this, unless you mean the fact that it leaked to the public. How was LLaMa pushing the field forward otherwise?
Good (as written), or God (at first glance)?
meta is currently the top here but they have focussed mostly on the game experience. can't wait to see AR as a legitimate productivity tool.
I'm sure general AI will play a role here of course
Crypto then AI, surely there will be a third thing coming too
Right now, only the 3090 and 4090 have enough VRAM (24 GB) among the consumer models to make them worthwhile for most LLM work. I'm hoping the 50 series (5090?) has more VRAM and stays affordable - though I'm not holding my breath either.
China for example needs epic scale numbers of GPUs to power its economy going forward. The equivalent of many millions of H100s for a $20 trillion economy looking to advance rapidly.
Given the restricted production globally (Nvidia production bottleneck, with only a few places on earth that can produce something like this), until China can produce their own very high-end GPUs their economy is going to be held back by the lack of capacity. Tens of thousands of high-end GPUs slipping through isn't going to cut it, that simply doesn't matter very much.
You can prevent China from getting a million H100s. You can't prevent them from getting ten thousand of them from many different sources over time.
If this goes the same way as when Zuck entered the race to replace the US dollar, create a metaverse or cure disease, Ai will be dead in the next 2 years.
My old office was in the same building as Chan-Zuckerberg HQ hq and all the scientists I worked with rolled their eyes at their completely over the top mission statement to "cure all disease within our lifetimes." Cure ALL disease? Really?? Maybe pick one and try hard to eradicate it. But *all* disease?! All of it? Geeze.
Also one of their top scientists exposed himself taking his junk out of his pants and tried to feel up my coworker at STEM restaurant in the middle of the restaurant during happy hour. So overall they seemed like poor decision making reckless arrogant clowns.
he is in a rare position to throw money in all of these fields without worrying about the ROI. so i will not throw too much criticism his way.
but yes the ai bubble is real, but sadly we don't get to short on it this time.
I don't think they're done with the metaverse thing either, whether it will succeed, that's another question.
I didn't know that. What parts are impressive? It seemed like VR Chat + Roblox to me, but I've barely seen anything.
Mark Zuckerberg: First Interview in the Metaverse | Lex Fridman Podcast #398
"Facebooks Endless Pivots":
https://www.axios.com/2022/05/04/facebooks-endless-pivot-met...
Has Facebook as a business ever really pivoted from "social network advertising company?"
Microsoft name change incoming?
Are they also working on world peace?
Maybe Mark will have a similar moment of redemption, who knows.
So...what happened to that whole "Metaverse" thing? Is it time to rename the company again?
The promise of the former is a unified platform where e.g. a virtual magic sword bought in an RPG also works in an FPS made by an unrelated developer, and a virtual gun bought in the later also works in the former, and this is also your work collaboration environment. And now I'm thinking of ABK's boss fight sketch: https://youtu.be/w6u_EJa_sZE?si=wlYD8EhRd_PLm39l
If nobody can hear the sound of a tree falling, does it still make a sound ?
Apparently only a single-digit % of people who purchased AR headsets are using them once in a month (source: Valve developer Chet Faliszek)
It’s been the experience I’ve been having every day since the original oculus devkit. I’ve rarely touched a flatscreen title since. Eventually I hope they’ll become good enough to work in when I travel.
edit
In saying that, i've always maintained its not ready for normies yet, and that we were probably a good decade or two away from the point where its going to be in a format/design that the mass market would accept/have interest in. In fact in many cases, I still actively reccomend against purchase because I know it's still kinda incuating along.
AI will be used to enhance that e.g. generated avatars, autonomous agents, hand/body tracking etc.
I may be repeating my own errors with the following as I said much the same for the iPhone and the Apple Watch when they were new, but…
The prices Apple are asking for seem excessive given what the products actually do, surely the cheaper alternatives are going to be what really matters?
(In this case, cheaper alternatives would include the Meta headsets).
Watched a guy spend $6K on iPads for his kids like it was a stocking stuffer. Took 15 minutes as he had to call the wife on if 1TB was enough. Felt bad I was just buying an Air so wandered to let the sales kid work him.
Kid said it was an overage day.
Strong adoption by the younger TikTok generation which is an area that Meta has been desperate to bring back into their fold.
Also the product is pretty impressive. The camera quality is really good and the AI features genuinely useful. Definitely caused many in the industry to wonder if that form factor could be the future of the AR industry in the short term as well as the long term.
Ah, yes, the inevitability of Apple success.
Just as everyone switched from cell phones to PDAs after 1993.
I'm gonna get downvoted for this comment because this is not Reddit, but I had to say it.