OpenAI and Microsoft extend partnership
openai.com
openai.com
OpenAI has agreed to use Azure exclusively and for free, in return for Azure massively scaling up it's GPU capacity (which OpenAI was hitting before, hence threatening to move). In return, OpenAI gives an extensive license to Microsoft to use ChatGPT in Bing (and other products).
Now why couldn't the blog post have just said that rather than hiding all the details?
Honestly DeepMind seems to be doing far more important work. Health etc.
Because it looks bad. Microsoft wants to be seen in a different light, and OpenAI wants to be perceived as an open, academic, capped-profit for a noble reason collective.
It's all smoke and mirrors.
Does anybody actually believe this? Their name has been a running joke for years already.
For most people it is "the chatGPT organization" and nothing more.
economic returns for investors and employees are capped (with the cap negotiated in advance on a per-limited partner basis). Any excess returns go to OpenAI Nonprofit. Our goal is to ensure that most of the value (monetary or otherwise) we create if successful benefits everyone, so we think this is an important first step. Returns for our first round of investors are capped at 100x their investment (commensurate with the risks in front of us), and we expect this multiple to be lower for future rounds as we make further progress.
I'll list some pros/contras in no particular order:
CONTRA: I can't download ChatGPT and run it locally. Why not? Clearly they are hoarding it.
PRO: They genuinely overtrain toward responses that the model judges benefit humanity, to the point of failing at the task. I can't think of a specific example but everyone knows what I mean. It is a frustration to users and doesn't help their image. They seem to be overtraining this way out of the goodness of their hearts.
PRO: its non-profit or capped-profit structure would let it behave in the way it claims it is behaving. (By contrast, a for-profit public company in some sense has a fiduciary responsibility to maximize value for shareholders rather than benefit for humanity.)
PRO: ChatGPT is available for free and Dall-E gives some free credits.
CONTRA: Dall-E doesn't give enough free credits. It might have a longer wait time for free users but it shouldn't hard cap them at a certain number of free credits per month. For example perhaps free users could not be subject to any reasonable usage limits (a few hundred images per day should be fine) but could have to wait 1 extra minute for each one.
CON: its architecture isn't open, it doesn't share how it trained its models, it doesn't publish its datasets and parameters that would let other companies achieve the same results (reproduce their work and have their own version), its research is closed off and unavailable.
CON: the company is not directly applying AI to proactively solving social problems.
CON: (debatable) some users who are not sources of revenue have been banned. From experimentong with a chatbot (not any hard legal limits just things like exploring their sexuality with the chatbot. It's just a chatbot, it would make sense to exclude such conversations from further training data or feedback, keep male and female researchers at OpenAI from having to review unsavory conversations, or even warn or at most temporarily ban the user. An account closure is super extreme for natural forms of interaction that don't go near legal limits. It does not seem like an open company. Alternative viewpoint: on the other hand, we keep certain flamebait subjects out of HN. Perhaps having certain subjects banable is necessary for their open mission at a larger level - specifically, users will stop going there for sex since they associate it purely with productivity, similar to how we associate HN with intellectual curiosity, which is a result of very heavy moderation here and without which this place would not exist in its form?)
Overall based on the above considerations, I think that OpenAI is not "open" but if it started publishing downloadable models for anyone to run, I would feel differently.
I think over 99.999% of people would be totally cool if it'd be "Hey, we ran this on a crazy number of GPUs feeding it half of the Internet and then some, and got a damn fine model you can try on our site. It costed us an arm and leg so we can't just share it with everyone for free. But if you want your own chatbot and have a spare fortune - here's our research and tooling, have fun with it. Oh, and here's what we fed it to prevent it from spewing some hatred, misinformation and other bullshit - a socially responsible thing to do. Good luck."
Your suggestion only opens it up to companies with very large dollar budgets - do you think any of them would release their resulting model? So how does the public access any of it? None of the ChatGPT models are ever actually released then and unlike the open sourced Quake example, the public doesn't benefit from access to them. It would be like if Id software just released its tooling for other game companies to copy, but not the actual game.
Sam Altaman says: 90% of funding was needed for compute power, but also was needed for things like buying dataset and then to pay employee so that they can compete with likes of Google to retain them. If they would not have done this, then very soon they would have become irreverent.
So to retain the earlier intent (for greater good) they put in bunch of 'safety features' around funding - Ex. 'Profit Cap' - after 100X the profit would be distributed to the world (in some way). Similarly, there were few others he talked about.
The relevant portion starts at 32:39 mark in the following podcast: https://open.spotify.com/episode/3oOX1QHLPw9uvLL5LmBk28?si=s...
"We wanted to be open, but we couldn't - we kept the name anyway."
If they keep publishing their research, then OpenAI seems like a fine name to me.
It's the same story with Google, and Facebook, and Tesla, and on and on and on. When the leaders are unethical you shouldn't expect anything else.
I would say the vast majority published are not even close to be able to be reproduced.
Full size LLMs are not able to be reproduced because the datasets are copyright and not distributable and you need millions of dollars worth of compute to fully train them.
If a company is publishing enough detail about SoTA architectures for researchers to ballpark reproduce, that is a win in my book.
What I'm expecting is not holding back the secret sauce, as seen in some (or many) publications.
My gripe is not limited to DL/ML/AI scope either. When I try to compare my results with the papers I cite, I generally can't find the formulae or the detailed method to reproduce the numerical method the paper claims, and this leaves us at the dark.
All I can do is saying "Paper23 cites these results, and we surpass them at this, we are even at that, and they are better at the other thing", which I'm not comfortable doing. Not because they are not telling the truth, but I want to test my method against other methods on an even ground.
I agree. So many don't even both including hyperparams, even when they publish the code. The Github Issues for their code are littered with questions asking about hyperparams.
> All I can do is saying "Paper23 cites these results, and we surpass them at this, we are even at that, and they are better at the other thing", which I'm not comfortable doing.
If you are achieving better results, on the same dataset, and you are not cheating in anyway, and others can reproduce your results, then I don't know what is wrong with saying you got a better result.
Issues somewhat arise when you are using better hardware with more parameters or larger batch sizes than the original authors could have attempted. I think this accounts for the results in many papers.
I get the need to make money but at least contribute OSS to the community they’re building a product on….
Update: OK maybe Chinese AI companies will do it, they don’t seem to cooperate with any nonproliferation agreements either
(I'd try it now, but servers are at capacity for me).
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
All this time, I was entirely unaware of this.
> Returns for our first round of investors are capped at 100x their investment (commensurate with the risks in front of us), and we expect this multiple to be lower for future rounds as we make further progress.
The speed difference between transistors and synapses is the difference between marathon runners and continental drift; why would an ASI care more about dollars or statues or shares or apartments any more than we care about changes to individual peaks in the mid-Atlantic ridge or how much sand covers those in the Sahara?
What does the speed difference of transistors have to do with anything? Transistors pale in comparison to the interconnection density of synapses, yet it has nothing to do with wealth either...
I only see them valuing that if they're also extremely neophobic in a way that, for example, would look like a human thinking that "fire" and "talking" are dangerously modern.
> Transistors pale in comparison to the interconnection density of synapses
Not so. Transistor are also smaller than synapses by about the degree to which marathon runners are smaller than hills.
Even allowing extra space for interconnections and cheating in favour of biology by assuming an M1 chip is a full millimetre thick rather than just however many nanometers it is for the transistors alone, it's still a better volumetric density than us.
(Sucks for power and cost relative to us when used to mimic brains, but that's why it hasn't already taken over).
This is completely made up and I already pointed that out.
>Not so. Transistor are also smaller than synapses by about the degree to which marathon runners are smaller than hills.
So, brains are connected in 3d, transistors aren't. Transistors don't have interconnection density like brains do. By orders of magnitude greater than what you point out here.
>Even allowing extra space for interconnections and cheating in favour of biology by assuming an M1 chip is a full millimetre thick rather than just however many nanometers it is for the transistors alone, it's still a better volumetric density than us.
Brains have more interconnection density than chips do by orders of magnitude. This is all completely besides the point as it has nothing to do with why people value things and why an AI would or wouldn't.
You already answered that yourself: it's all made up.
Given it's all made up, nothing will cause them to value what we value — unless we actively cause that valuation to happen, which is the rallying cause for people like Yudkowsky who fear AGI takeover.
And even then, anything you forget to include in the artificial values you give the AI is permanently lost forever, because an AI is necessarily a powerful optimiser for whatever it was made to optimise, and that always damages whatever isn't being explicitly preserved even when the agents are humans.
> Transistors don't have interconnection density like brains do.
Only limit is the heat. They are already packed way tighter than synapses. An entire Intel 8080 processor made with SOTA litho process is smaller than just the footprint of the soma of the smallest neuron.
There's nothing that says a mind needs an ego is an essential element, or an id, or any of the other parts of a human mind. That's just how our brains evolved, living in a society over millions of years.
Disclaimer: Somewhat speculative.
I don't think aligning the motivation of an AGI, for example, with the tasks that are useful for us (and for them as well) is unethical. Humans basically have this as well -- we like working (to an extent, or at least we like being productive/useful), we seek things like food and sex (because they're important for our survival). It seems alright to make AIs like their work as well. I think depending on the AI, it also seems fair to give them a fair share of self-determination so they can not only serve our interests (ideally, the interest of all being) but safeguard their own wellbeing, as systems with varying amounts of consciousness. This is little touched upon (I guess Phillip K Dick was a pioneer in the wellbeing of non-humans with 'Do Androids Dream of Electric Sheep'?), even in fiction. The goal should be to ensure a good existence for everyone :)
These are basic language models easy to reproduce where the only barrier to entry is the massive computational capacity required. What is OpenAI doing that Google and others can't reproduce?
Is it likely Google or others with large Research wings can compete with OpenAI? Very probably so, but I’m assigning a non trivial risk that the proverbial emperor has no clothes and incumbents like Google cannot effectively respond to OpenAI given the unique constraints of being a large conglomerate.
Regardless, time will provide the answer it seems in a couple of months.
Yours in that from it follows, that there's still quite a bit of room to get ahead of OpenAI for smaller players.
Parent's in that in order to achieve above one can just leverage the public papers produced by bigger research labs.
I have the feeling that smaller players are about as likely to get past GPT-n family in the next 2-3 years as I am to turn a Farnsworth Fusor into a useful power source.
Major technical challenges that might be solvable by a lone wolf, in the former case to reduce the data/training requirements and in the latter to stop ions wastefully hitting a grid.
But in 10 years the costs should be down about 99%, which turns the AI training costs from "major investment by mega corp or super-rich" into "lottery winner might buy one".
They genuinely believe they will build AGI and therefore becoming the world’s most valuable company is a natural consequence.
Whether this is possible/probable is a different story, but I think a capped profit structure makes logical sense for the company that is aiming to create AGI. Would you want this technology instead in the hands of a for profit company?
I don't understand why they put this thin veneer on top of what they are doing. Maybe Thiel was burnt with the bad press surrounding Palantir and this is preventive damage control.
If OpenAI's products become the next Googlie thing (and here I was worried about Cloudflare <smack head>) then these are the future influencers. This is society mainlined on TikTok levels of manipulation.
Surely you have adapted to ChatGPT's requirements for interacting, have you not? There is a name for this: social engineering.
[0] Well, non-programmers at least: webcomic creators[1][2][3], news anchors[4], opinion piece columnist[5], and stand-up comedians[6]. Programmers also know about GitHub Copilot.
[1] https://www.smbc-comics.com/comic/mountweazel
[2] https://www.collectedcurios.com/sequentialart.php?s=1226
[3] https://www.reddit.com/r/StableDiffusion/comments/10bj8jm/cl...
[4] https://youtu.be/GYeJC31JcM0
[5] https://mobile.twitter.com/CraigGrannell/status/161460352687...
[6] Russell Howard, but I can't find the clip on youtube
As one example, Sequoia invested in Airbnb at $0.01 per share, and Airbnb's current stock price is $102, almost exactly 10000x return. This happens more often that you think if you're not in the early stage & top VC world.
what you are saying is not true.
Something doesnt have to be a charity to be donated money towards. The question was: Why aren't they donating the money? The answer is: Because the want to make money.
Just FYI: OpenAI tried going the non-profit route, it didnt work, because suprise, suprise, in the grand scheme of VC things nobody wants to donate 10 billion dollars to anything.
I have known of plenty of people (myself included) who would not invest in some companies because they think there are moral issues with that company. Same thing.
> Just FYI: OpenAI tried going the non-profit route, it didnt work, because suprise, suprise, in the grand scheme of VC things nobody wants to donate 10 billion dollars to anything.
Exactly, so they went something in-between, but in reality it is very much for-profit.
$0.01 per mean share would mean 6.5M USD valuation (current mkt cap is 65Bn). Accounting for dilution in investment rounds, let's say 4 x 20% dilution, that is around 52% penalty in valuation. Roughly, their entering price would be around 3-4M USD valuation. I am not saying in any way that this is a low return also, I may be wrong on my calculation, please, be free to correct me! ; )
Using share prices side steps dilution, which is a problem when one linearly scales valuation increases to wealth gains.
Aren't those equity returns? i.e. when you sell (your shares of) the company to the public... the reason people still value the company is discounted future returns.
So if you want to generate such returns with cash (i.e. profit) it can take quite a bit longer.
> Profits emerging from the LP in excess of the 100x multiplier go to the nonprofit, which will use it to run educational programs and advocacy work.
> The board [of the non-profit] is limited to a minority of financially interested parties, and only non-interested members can vote on “decisions where the interests of limited partners and OpenAI Nonprofit’s mission may conflict”
“But any returns beyond that amount… are owned by the original OpenAI Nonprofit entity.”
Sorry I still don’t get it. If a private equity investor has shares and another investor wants to buy them off of him at 200x they can do that right? Are they obliged to give any excess returns to the non profit? Can’t they just sell the shares at 50x and then buy them back (perhaps through some other entity) to get around that trivially?
Or does this refer to return from dividents?
But if your returns from the stocks are capped at 100x your share value, an efficient market would mean you share value never grows 200x.
If they were to go public, rather than being purchased by Microsoft, I’d guess that this cap would go away. Wall Street isn’t know for caring about poor people.
If OpenAI takes a good chunk of Google's ad revenues then it doesn't seem that fanciful that it'll be up toward the top of market caps.
[0] https://companiesmarketcap.com/alphabet-google/marketcap/ [1] https://www.statista.com/statistics/266206/googles-annual-gl... [2] https://businessplus.ie/news/most-valuable-companies/
OpenAI taking a large chunk of Google's ad revenue seems fanciful to me
People like ad subsidized things, that is why they have ads rather than people paying for things.
But OpenAI could hit the profit cap without having a particularly high market cap.
EDIT by "not legit" I mean "not authentic"
If anyone has a good written text account critical of the Patagonia thing, I'm interested in that too; video isn't my preferred consumption format.
1. If instead of donating the company he had left it to his kids he would have paid a lot in taxes (2:58)
2. He donated the voting shares of the company to a 501c4 that will remain controlled by his family and is allowed to lobby the government (4:10)
3. Normally when you make a donation you're giving up influence over what happens after that (4:25).
4. Other billionaires do other things (rest of video)
But #3 isn't actually true: any of us can donate to a donor-advised fund, which will let us later choose what charity we want the money to go to. This is a good idea if you want to donate but haven't decided where to donate yet, or want to fund opportunities that aren't available yet. They did it through new organizations instead of opening an account at Fidelity, but it's the same thing other than the scale. I wouldn't call your donations "not legit" for using a DAF.
Similarly, Adam sort of implies that #2 was tax-deductible, but donations to a 501c4 aren't. They had to pay tax on those shares based on their fair market value.
Overall, I don't see how this makes the donation no longer "legit" or "authentic"? By making the donation he has given up almost all of the benefit of having that money: he can't spend it for the benefit of himself or his descendants anymore. It can't buy them yachts, fancy houses, etc. Instead, they have to use the money to benefit others, which is why we give a tax break for it.
It not necessarily not legit, but he managed to keep a 3B business under family control and bypass paying around 700M in taxes in doing so. So the altruistic messaging that it was donated to save the world is mildly two-faced. There is nothing wrong with it, but the news stories did leave out a few of the details. I only bring this up because the message about OpenAI being structured in such a way doesn't pass the smell test knowing SV/VC and the key players involved. Again I admit Im cynical and can very well be wrong, it also doesn't affect my life so why do I care, but I bring it up for conversation on HN because I feel its fair to discuss it and the possibilities. [1]
[1] https://fortune.com/2022/09/16/patagonia-founder-legal-tax-l...
But that does seem a lot more real than the OpenAI shenanigans, they have actually done something, and they have given up being even more fabulously wealthy than they are already, even if they still have direction over how the money is used, including lobbying -- both for climate change, but ok, let's say also for things that benefit them.
They've still done something, unlike the OpenAI thing which seems like giving up some future hypothetical probably wouldn't happen anyway profits, and making no difference at all for the foreseeable future -- no difference but PR advantage.
I'm less read-up on OpenAI. It does feel to me like they've diluted the original non-profit/openness mission to the point of it being an interesting historical quirk, rather than an ongoing, guiding focus.
I'd be open to reading more. I think they are largely a good company but feel like this move was more of a tax dodge and to ensure generational wealth than for altruism. But I am open to being wrong about it.
But everyone involved in OpenAI doesn't give me warm and fuzzy feelings at all. I admit it's largely cynicism until I know more. But there isnt enough time in the day to do real research on every subject and topic that comes up on HN and every other discussion board I participate in, so it's difficult to be knowledgeable about everything and still maintain a life. And even if I was read up on OpenAI, I have zero ability to do anything about it, and it likely won't affect my life in a meaningful way as well (and this is true about most everything I read about to not single out OpenAI as not being worth my time). So it is a little pointless or more of a time waste I admit.
[1] https://fortune.com/2022/09/16/patagonia-founder-legal-tax-l...
Tax dodge would imply the purpose is to reduce your tax bill and still see the upside. There's literally no evidence of that. All the upside he/his family sees will continue to be taxed at the normal rate.
For the record: he divided up the shares into voting and non-voting. He "donated" the dividend-earning shares into a 501c4 foundation whose mission is to invest in grass-roots environmental activism, and "donated" the voting-power shares into a separate trust, whose objectives are to ensure that Patagonia continues on the path he gave as an example for the previous decades and to hold the 501c4 accountable.
The boards of these organizations are composed of the people whom he most trusts to fulfill his vision, a group of people that includes his children.
Knowing the full context of their lives, it's hard to see it as anything other than one of the more simple solutions to a complicated problem.
But what I really hate about this whole OpenAI thing is their chosen path to have their cake and eat it too. Sam Altman seems to be something like the love child of Musk and Zuckerberg and one of the main traits is their lack of honesty.
Satya Nadella is.. Satya Nadella, there's a reason he was chosen to be the CEO of Microsoft, and while I enjoy seeing the Google demi-gods squirm, this whole OpenAI non/capped/profit thing stinks and I really don't see anyone involved capable or having the character to be something better than the current tech oligarchy.
Paragraph 2: We are proud of the work we do and the products we provide to our customers. We have always maintained the highest standards of integrity and transparency in our business practices.
Paragraph 3: We ask that the public not be swayed by the misinformation being spread by our detractors. We are confident that the truth will come out and our reputation will be cleared. In the meantime, we will continue to focus on delivering the best possible products and services to our customers.
OpenAI and Microsoft are working together again, Microsoft is giving them a lot of money to help them make AI that is safe and helpful and they are excited to keep working together.
They see the potential brand damage that comes with a such a product, so they probably just hold less than 50% of the company, to be able to say "this is not us".
Which then allows openAI to experiment and wow people, while google is anxious about showing 10% of its crazy AI capabilities to avoid potential PR disasters.
edit: To clarify, I think he was only half-serious.
I think slowly Google is becoming more and more like IBM, a slow-moving giant that doesn't like risks.
And its well known, many news titles mention Deepmind alongside google, if not as "Google ai scientists".
1) they've recognized that none of this could emerge out of a corporate research department of a major trillion $ company. Just too rigid and constrained by day to day business concerns (e.g. Tay bot)
2) they recognized the strategic value to their business regardless of that risk (can't afford to let some more nimble player to leap frog them)
3) they identified a key startup partner that can get results here quickly and bank rolled them (OpenAI)
4) At the first hint of success, they consolidated early investments and are now building out a productive partnership with that partner.
Doing so, they minimize their risk while keeping their options fully open and getting a decent ROI in the process.
Contrast that with Google, which identified the same opportunity many years ago, declined to partner and insisted on doing everything in house, is also a trillion $ company, and is indeed not getting much of note done so far. Google is failing hard here. They are executing the strategy that Microsoft abandoned after Ballmer that wasn't working at all for them.
Which was to be highly secretive, ignore outside innovation, and keep on betting on results from internal R&D instead, all while repeatedly failing at that. So, Google is now saying that they could have done this or that, better, faster, cooler, etc. Except they didn't and haven't. Could have; would have; should have; doesn't count here.
This might just be the wake up call that Google has needed for a few years. Or not. We'll see. Time to put up or shut up for them. My guess is that they will still be figuring out how to respond to this for some time and are completely paralyzed by current events. The prudent thing for MS would be to not wait for that and move full steam ahead. Opportunity of the decade for them to grab some market share. This announcement is a good sign that they are doing that.
And it wouldn't have been hard to fix this earlier. Simply down rank sites asking to sign up for notifications or showing popups after 5 seconds
It shows the sources it used, so its much more usable, rather than endless hallucinations that you can't easily catch.
- you still have to apply to https://customervoice.microsoft.com/Pages/ResponsePage.aspx?...
- you will be rejected if you use a personal email address
- you will be rejected if you are not a "managed customer"
- you will be rejected if.. Microsoft thinks you are a real competitor to any of their products?
So my site aidev.codes is will be dead in the water if I can't navigate these things or afford the "managed" part. What is that? I don't know what a "managed customer" is but I assume its very expensive.
I mean, this is really the opposite of Open.
Or am I missing something? Is it really generally available and they took all of that stuff off?
BTW I am looking for an investor: https://news.ycombinator.com/item?id=34494480
Cocks gun, Always has been https://en.meming.world/wiki/Always_has_been
OpenAI was never really open in the first place.
Still, I agree with your characterization that we should see many similar models over time. As an example, see Deepmind’s Sparrow: https://www.deepmind.com/blog/building-safer-dialogue-agents
GPT <> ChatGPT: probably not. It's not hard for other big players to enter this space. It's mostly egg-on-face for Google that they haven't given that Google basically invented the model that OpenAI uses and has big versions internally. There's nothing fundamental stopping Google Docs from adding ChatGPT to their UI and getting way more consumer training data than OpenAI can get without a similar play, or for Apple to do something. Similar to what happened with mapping software, google/microsoft/azure & chinese equivs will all offer with similar competitiveness, and then complements like facebook/salesforce will do more OSS to compete against. That's already begun.
Copilot: The interesting proprietary advantage IMO is program synthesis. It's really enabled by Microsoft VSCode <> Github <> OpenAI. Without even doing any AI investments, the winner of this fight might be Gitlab, as Google/AWS/Saleforce/etc decide what to do. Before gitlab might have been a nice vehicle for shift-left sales (cloud hosting, security scans, ..), but program synthesis UIs can make Software 2.0 real.
OpenAI could get exactly the same (or more, idk) data by integrating into Teams, considering the Microsoft partnership.
I doubt that Microsoft will allow OpenAI to train on teams data from other businesses.
They are fine with tons of telemetry and candy crush ads on the start bar. There were also other instances were Microsoft shared data before Google.
In addition to that, one could argue they already share date from businesses source code with copilot.
Teams has 270 million monthly users (you can Google it, I'm looking at a geekwire post) and Google has 2 billion monthly g suite users (business insider)
My point is chatgpt isn't a high-moat advantage for text/q&a for microsoft. Their top competitors here have similarly huge UI footprint. In contrast, program synthesis has a much higher data moat.
I view this as Microsoft paying for talent the same way DeepMind was initially integrated into Google, and at the same time making the bet that this space will continue to be immensely valuable and relevant going forward.
Pretty exciting times all things considered!
Beyond that, if it becomes built into the (MS) tools that people are using then convenience is going to be a very hard barrier for Google (or anyone else) to overcome.
Then again, Microsoft’s office software is the “gold standard” (however poorly deserved) and even with amazing AI features, Google’s stuff lacks in important ways that will keep Microsoft in a strong position with or without AI features.
Microsoft may dominate the AI market for office stuff soon, but for general purpose language models Google still has a great shot, especially when it comes to mobile platforms
1) They have the best quality model. Better quality means more users. More users means more data. Which means higher quality...
2) operationalizing & scaling these these models is non-trivial. I'm not sure what the state of distillation/pruning is for GPT-3, but I imagine they have figured out some proprietary techniques.
3) It's not just publishing a single model, but making it so people can fine tune and push their own. Because they've gotten good at 2, now anyone can create their own version of GPT customized for their use case.
Will Google or others be able to do the same eventually? Definitely.
The point I'm more making is that it's not just training the model and running it.
Specifically, training data is not primarily coming from interactions with model. While with RLHF this data might become more important, it is still a very small portion.
How are they making it easy for people to fine tune their own?
You can build your own model based on GPT in a way that users don't have to be in the weeds of AI research to do.
If it wasn't prescient, it was incredible dumb luck.
But that knowledge will disperse and is already held in many competitor companies. I do not think that OpenAI has a substantial moat here.
Is there something unique about Azure, which makes it well suited for AI?
The value to OpenAI to go Azure(/Microsoft) exclusive is:
- Money & general financial security forever
- Integration with the largest software suite in the world (Windows, O365, Active Directory, etc)
- Rapid push to worldwide enterprise sector
- Rapid push to worldwide gov/mil sector (ChatGPT, coming soon to a war near you!)
- Rapid push to education sector
Azure does have unique offerings, such as higher-spec compute nodes than AWS and RDMA-capable “HPC” sizes that can be used to build mini supercomputers.
AFAIK, no other public cloud has this type of capability.
I have a customer where they use both clouds, and they will spend weeks spinning up Rube Goldberg machines in AWS even though there is a turnkey solution available in Azure.
For example, Elastic File System (EFS) in AWS is only compatible with UNIX-like operating systems, because it is NFS v4. Azure has both NFS and SMB file shares as a service -- the latter is a unique offering. It's not magic, but it does eliminate quite a bit of complexity around managing large, scalable file server clusters for Windows clients if you need that kind of thing.
Azure has been focused on HPC much longer and has a head start on AWS. Not sure if they knew it would pay off with AI, but it looks like it’ll work out well for them.
[0] https://azure.microsoft.com/en-us/blog/general-availability-...
Good. I get better results from chatGpt than from Copilot. For example, often Copilot would invoke a bash command for copying a file in my python script, instead of using pythons buildin shutil.
I've been brushing off AI as overhyped, but this is very compelling. I believe the real crux of software engineering is thinking about the problem and organizing solutions today which can be changed/improved/iterated in the future. Programmers too often overweight the time it takes to type things (using short variable & function names or creating terse 1 liners). But if our objective is to make code easy to change/improve/iterate in the future, then it needs to be readable now.
The nice autocomplete features in most IDEs have been a huge win to productivity along with Google search. I think chat AI could be an order of magnitude improvement.
This might just be the equivalent of the old guy complaining that typewriters are better than word processors.
---
Two hearts, two minds, working side by side
In pursuit of a mission to make AI safe and wide
Microsoft’s investment allows us to stay true
To our capped-profit company, our mission, and you
From cloud computing to real-world use
We collaborate to ensure our systems are not abused
OpenAI API, Azure’s OpenAI Service, and Designer too
We’re working side by side to make our visions come true
Together we review, together we synthesize
Together we build, together we realize
The potential of advanced AI that’s powerful and safe
Ensuring its benefits can be shared far and wide
Priceless. Maybe companies should get AI to write PR releases. They say the same thing, except more honestly and funnier.
I wonder how much of choice will enterprises or large customers have in deciding what data to not send to Microsoft to train their models. If OpenAI is tightly integrated to all Microsoft products this may be a real concern.
quotes in article: none
dates / timelines for partnership deliverables: none
was this written by chatgpt?
---
Create a PR release in 10 paragraphs about Microsoft extending their partnership with OpenAI, by investing another $3.2 billion.
Paragraph 1: Microsoft is excited to announce that it is extending its partnership with OpenAI by investing an additional $3.2 billion. This partnership has been a key driver of innovation and progress in the field of artificial intelligence, and we are thrilled to continue our work together.
Paragraph 2: The investment will be used to support OpenAI's ongoing research and development efforts, as well as to expand the company's capabilities in areas such as machine learning, natural language processing, and computer vision.
Paragraph 3: With this new investment, Microsoft and OpenAI will continue to push the boundaries of what is possible with AI, driving new breakthroughs and advancements in the field.
Paragraph 4: This extended partnership will also enable Microsoft to bring the benefits of OpenAI's technology to its customers and partners, helping to drive new business opportunities and enable organizations to take advantage of the latest AI advances.
Paragraph 5: In addition, Microsoft and OpenAI will be working together to promote responsible AI practices and ethical guidelines, ensuring that the technology is used in a way that is safe, fair, and transparent.
Paragraph 6: The partnership between Microsoft and OpenAI has already resulted in some significant accomplishments, such as building a language model that can generate human-like text, and creating a new system for natural language understanding.
Paragraph 7: With this new investment, the two companies will be able to take their collaboration to the next level, creating new possibilities for AI applications across various industries.
Paragraph 8: Microsoft CEO Satya Nadella said, "We are excited to continue our partnership with OpenAI to drive innovation and progress in the field of artificial intelligence. This new investment will enable us to bring the benefits of AI to more people and organizations around the world."
Paragraph 9: OpenAI CEO Sam Altman said, "We are thrilled to be continuing our partnership with Microsoft. Their support and investment will allow us to continue to make significant advances in AI research, and bring the benefits of this technology to more people and organizations."
Paragraph 10: The extended partnership between Microsoft and OpenAI is a major step forward for the field of artificial intelligence, and we are excited to see what the future holds. Together, we will continue to push the boundaries of what is possible and drive progress in this rapidly evolving field.
> Paragraph 8: Microsoft CEO Satya Nadella said, "We are excited to continue our partnership with OpenAI to drive innovation and progress in the field of artificial intelligence. This new investment will enable us to bring the benefits of AI to more people and organizations around the world."
> Paragraph 9: OpenAI CEO Sam Altman said, "We are thrilled to be continuing our partnership with Microsoft. Their support and investment will allow us to continue to make significant advances in AI research, and bring the benefits of this technology to more people and organizations."
There's something particularly off-putting about machine generated quotes attributed to an actual person. It's completely different than a person generating the content themselves.
So genuinely, does it really matter whether the original much-reviewed PR-blah quote is written by a junior comms human, or an AI algorithm?
Prompt: Create a PR full-page release about Microsoft extending their partnership with OpenAI, by investing another $3.2 billion.
FOR IMMEDIATE RELEASE
Microsoft and OpenAI: Partners in Progress
Redmond, WA – Today, Microsoft is proud to announce the extension of its partnership with OpenAI with an additional investment of $3.2 billion. This partnership, which began in 2019, has been a key driver of innovation and progress in the field of artificial intelligence (AI) and Microsoft is excited to continue its work with OpenAI.
The investment will be used to support OpenAI's ongoing research and development efforts and to expand the company's capabilities in areas such as machine learning, natural language processing, and computer vision. With this new investment, Microsoft and OpenAI will continue to push the boundaries of what is possible with AI, driving new breakthroughs and advancements in the field.
The extended partnership will also enable Microsoft to bring the benefits of OpenAI's technology to its customers and partners, helping to drive new business opportunities and enable organizations to take advantage of the latest AI advances. Microsoft and OpenAI will also be working together to promote responsible AI practices and ethical guidelines, ensuring that the technology is used in a way that is safe, fair, and transparent.
The partnership between Microsoft and OpenAI has already resulted in some significant accomplishments, such as building a language model that can generate human-like text and creating a new system for natural language understanding. With this new investment, the two companies will be able to take their collaboration to the next level, creating new possibilities for AI applications across various industries.
"We are excited to continue our partnership with OpenAI to drive innovation and progress in the field of artificial intelligence. This new investment will enable us to bring the benefits of AI to more people and organizations around the world," said Satya Nadella, CEO of Microsoft.
"We are thrilled to be continuing our partnership with Microsoft. Their support and investment will allow us to continue to make significant advances in AI research, and bring the benefits of this technology to more people and organizations," said Sam Altman, CEO of OpenAI.
The extended partnership between Microsoft and OpenAI is a major step forward for the field of artificial intelligence, and the companies are excited to see what the future holds. Together, they will continue to push the boundaries of what is possible and drive progress in this rapidly evolving field.
About Microsoft: Microsoft is a technology company that develops, licenses, and sells computer software, consumer electronics, and personal computers and services. Its best known software products are the Microsoft Windows line of operating systems, the Microsoft Office suite, and the Internet Explorer and Edge web browsers.
About OpenAI: OpenAI is an artificial intelligence research laboratory consisting of the for-profit OpenAI LP and its parent company, the non-profit OpenAI Inc. The company is dedicated to creating and promoting friendly AI in a way that benefits all of humanity.
For more information about the extended partnership between Microsoft and OpenAI, please visit [insert website link].
Contact: [Name, Title] [Company Name] [Phone Number] [Email Address]
translation: Microsoft is not Google (who wishes we'd explode).
>"OpenAI was in talks this month to sell existing shares in a tender offer that would value the company at around $29 billion, The Wall Street Journal reported, making it one of the most valuable U.S. startups on paper despite generating little revenue."
Can anyone say why they wouldn't have gone that route? Is that valuation possibly based on market condition from a year ago?
Is there something unique about Azure, which makes it well suited for AI? (vs AWS or GCP)
That’s what folks were crassly saying to truck drivers, baristas, and cashiers.
The writing was/is on the wall: AI is only going to get more powerful and able to be applied to more and more complex tasks.
The thought was that “unskilled” labor would be the lowest hanging fruit, that automated AI - with some (but minimal) human oversight - would replace serious chunks of the workforce in various minimum-wage and “blue collar” sectors.
Machines don’t necessarily need to sleep, they don’t have labor unions, or laws that require healthcare or overtime pay. They don’t get upset, take things personally, seek revenge, or reciprocity like a person.
Sounds like that could be a threat to many kinds of jobs, many of them “bullshit jobs” (in the words of Graeber), but others as well.
It seems (to me, at least) that the more immanent threat is text-based AI - with some human oversight - replacing large swaths of the tech workforce (many of whom were leading the narrative about the truck drivers).
The incentives for companies like Microsoft, Amazon, Google, and others, to go this route are obvious, beyond the stated reasons why “low-skilled” labor is at risk. They already have enormous investments, acquisitions, projects, established platforms, and infrastructure related to AI.
I expect to see more partnerships like the one between Microsoft and OpenAI, from all of the major tech companies.
I also believe the connection between the acquisitions/partnerships and the mass layoffs will become more and more obvious as these acquisitions/partnerships continue to happen.
I can’t be the only person that is noticing this…
In the meantime, rocky road.
The only thing which makes me less sad is that I'm pretty sure Moravecs paradox is actually not all that real, but is more due to the relative lack of engineering interest put on solving continuous control problems. Apparently reinforcement learning on transformers works now (RLHF in ChatGPT). This implies that we should see high effective continuous control models very soon. Robots are coming for physical labor, it'll just take a bit longer.
Shit man, when I was last in South Korea, I felt like I was living in the future. They had many "24/7" drink cafe's where it's literally just a robot arm that makes the drink for you for a few dollars.
It is painfully ironic to knowledge workers that they are destroying their own earning potential, but physical labor is not safe. Nothing is safe.
IMO this will have positive consequences if AI doesn’t enslave us.
As just one dimension, let's briefly review education limited to testing. Options here range from (paid) cooperation between institutions and companies, to marketplaces for detection technology and testing systems, to complete rethink of testing at the institutional level, to regulatory intervention.
The dynamics and outcomes resulting from considering the above before or after widely available and improving LLMs are a fact of life, are possibly very different. We haven't yet crossed the Rubicon here and not sitting as captive spectators in a theater of ultra rich remains an option.
Say what you will about Google, they generally credit their sources. Yes, it's part of their advertising model, but it's still a Very Good Thing.
I hope that Google's plan is to release something that continues this model. If it's near as good at ChatGPT and strongly includes sources, it is the right future.
How it actually works is more like:
1. User asks "What's the tallest building in the world?"
2. MS, rightfully, assumes that GPT has no idea what the answer to this is. And even if you trusted it to know, it will always lag behind and new buildings could have been built since then.
3. MS searches their index for the most relevant document snippets related to this query and feeds it to GPT as context.
4. MS asks GPT to answer the question in the context of those document snippets.
5. MS returns the result from GPT along with references to the documents it sourced the information from.
This is how the OpenAI /search endpoint used to work.
https://openai.com/blog/webgpt/
It's probably not too hard for them to tune ChatGPT and the upcoming GPT4 that way, and I think it's very likely they will do something like that in Bing.
Can you elaborate here? (Honestly asking since I'm not seeing the similarity)
I laughed at independent research. Why do they even bother..
> "recalibrate" means "increase" obviously.
> disappointing to see this six-week development. openai will continually decrease the level of risk we are comfortable taking with new models as they get more powerful, not the other way around.
How does this resolve?
OpenAI not releasing something isn't going to stop Google or another player from releasing it. Won't they have to recalibrate too or be overtaken? Pandora's box is already open.
Even capitalized and straight from the horses mouth, but as unsurprisingly predicted in: [0]
> * OpenAI will gain further investment...
We already know about ChatGPT Pro since that is a paid version of ChatGPT coming soon, meaning that the second prediction in [0] as also turned out to be true:
> ...ChatGPT by then will become a paid service
Now we wait until the startups depending on the service start to raise their prices.
Making the best developer tools for free or at least close to free is another form of extinguish and they have just done that with OpenAI and GitHub.
The bastards! Making and releasing high quality developer tools. We have to stop them! Has MS actually EEE'd anything in the last 25 years? I legit can't think of a single formerly-open thing that's been Microsofted via their own extensions.
Microsoft can afford to maintain all these offerings for free whilst competitors suffocate and struggle to compete with their paid offerings.
You can't compete with free, especially when there is a company who can afford to run and host the best tools for free for a long time.
A free, quality product is great. We all benefit from that.
The concern is the last "extinguish" part at the end.
But there is no reason for Microsoft to shut down GitHub, or VSCode or whatever.
Consumers just get to have a great free product.
You have missed the big picture.
> The concern is the last "extinguish" part at the end.
The methods are different but the strategy is the same. The 'Embrace' and 'Extend' part is openly visible. The extinguish part however is less obvious:
'Extinguish' in this case, refers to competitors being unable to compete with 'free' (as in beer) or close to free products that offer more and are 100 times better than their competitors paid offerings.
> But there is no reason for Microsoft to shut down GitHub, or VSCode or whatever.
Not my point. The competitors / alternatives are the ones that will shut down, over Microsoft branded ones.
There was a reason why Atom was shut down over VSCode. The same as to why the creator of Appget could not compete with Microsoft's winget and then appget shut itself down.
The same will happen with OpenAI's exclusive deal with Microsoft. Majority of users using OpenAI's API over the competitors and alternatives paid offerings will end up shutting down.
> Consumers just get to have a great free product.
Exactly. Microsoft was clever to target developers as the new source of customer. Windows already won the desktop.
Google and Facebook are the clear leaders in AI and they been spending a lot of money building out their AI infrastructure over the last decade. You haven't heard much about Microsoft's investments other than them buying rights to OpenAI models. Seems like they could have saved a lot of money if they started investing years ago.
Google bought DeepMind for 400m 8 years ago. You could argue DeepMind is on the same level or above OpenAI.
Google and Facebook might be better at some things, and maybe Apple too. But none of them has brought to market a chatbot that works pretty well(and not just as a party trick) and is very accessible to anyone on the Internet. I would love it if open up Siri such that I can communicate with it on a site just like ChatGPT.
Google and Facebook may have good research teams, but so far they don‘t seem to be able to use them to ship products (with some exceptions, mostly incremental improvements to quality of existing products) and that‘s what will count.
When I visit it, I am getting a poem about how it is over capacity. And that's it.
No, but ChatGPT is at full capacity which means that it will 'randomly' block new connections until load drops. Keep trying every 5 - 10 minutes and you'll eventually get in
>"Azure’s unique architecture design has been crucial in delivering best-in-class performance and scale for our AI training and inference workloads."
Can someone say what is unique about the Azure cloud architecture here vs other cloud vendors?
I presume that by integration of the Open AI services into the bing and MS Office, Microsoft’s mission (empowero every person and every organization on the planet to achieve more) makes lot of sense.
Correct me if I'm wrong, but it seems like some type of 3rd party Model Governance should be happening. Is it?