OpenAI is exploring making its own AI chips
reuters.com
reuters.com
Let me add : there are numerous other AI chip startups.
- https://sambanova.ai/ (Enterprise AI and dataflow-as-a-service for established models)
- https://www.cerebras.net/ (AI accelerator, trying to compete with Nvidia)
- https://www.graphcore.ai/ (Another AI accelerator company, UK based)
- https://femtosense.ai/ (Sparse NNs on very low power chips, cool hardware and software challenges)
- https://sima.ai/ (ML accelerators for embedded applications)
- https://ambiq.com/ (Not AI, but low power chips for wireless using some fancy tech that reduces energy leakage)
- https://www.esperanto.ai/ (RISC-V based Tensor computes chip, founded by Intel Hybrid Parallel Computing Vice President Dave Ditzel)
- https://www.furiosa.ai/ (AI accelerator company which show good results in MLPerf benchmark)
- https://groq.com/ (From the team that built the original TPU at Google)
- https://lightmatter.co/ (Light tubes instead of copper)
Seems like OpenAI is exploring its own devices/OS as well, which makes sense to me, but it’s a vertical integration bet. This seems to be another big bet, but they could benefit from having their own optimized chips regardless of whether the device/OS bet wins out.
Extremely exciting times for OpenAI!
To me, it seems like a distraction for them to go into devices stuff, rather than making their stuff so relevant that device vendors (Android/iOS) can't ignore. At the moment they can because they got good enough competitor solutions.
If they found a theoretical way to infer something like GPT-3.5 without using so much RAM and can build a chip which makes this feasible in laptops or (holy grail) phones, they've got their moat for a 12-24 months, possibly more if they manage to patent it. Big if though.
But you could build a device with more memory...
If they just make android phones better, they risk Google benefitting off their efforts and replacing the models. Besides that’s a lot more work to partner and manage across the industry.
It seems risky to bet they can release consumer hardware (instead of server side) but that’s the ultimate big bet. Ask Meta how it feels to be “just an app”.
As far as gives you a competitive edge! For AI, to win you need better data and compute than your competitors.
Going up the stack to consumer devices seems like a somewhat speculative move, though I understand the underlying desire to secure a data moat.
Going down the stack to chips makes a lot of sense; if you can secure an edge in compute efficiency then you will beat anybody that doesn't have substantially more data than you do.
Making specialized chips to run LLMs is the logical next step.
That may be the endgame, but I think if it is there is a long time before attempts to jump to it aren't going to fail like every high-level-system-in-hardware for other than very niche applications, because general purpose (comparatively, even if specialized for running AI models) hardware will be good enough that the value of being able to upgrade the models it is running will outweigh any marginal temporary edge that current-models-in-hardware have.
The end-game is growing brains ;) (only partially kidding)
It might have some limited ability for updates if the hardware had the model code but the weights were in memory that was updatable.
It might do in-context learning even without that.
That's basically how our neurons work. New neuron growth and connection isn't much of a factor in learning. Rather it's the synaptic restructuring (equivalent to AI model weights) that change relatively quickly.
So we need to figure how how to "grow" mechanical brains. I envision this being done with a new generation of FPGAs tailored to this task.
If anything it vindicates even more their initial thesis about pursuing AGI as a business goal.
With that in mind, expanding their apps to ingest more audio and image data is an obvious strong move. (And you can see why a consumer device would help them get even more data from the real world, though it's less obvious to me that this is a win vs. just shipping apps.)
But I concede that I don’t have any concrete proof so I should modulate my certainty of tone.
Can't we have hardware companies that make hardware, software (AI) companies that make software, and data companies (or government institutions) that run the software on the hardware and deal with our data?
OpenAI wanting to vertically merge to make their own AI chips may seem harmless enough (it's a good business move, we can cut expenses)! But we can't forget that Sam Altman just a few months ago told Congress he supports making an organization that companies need permission from to being creating/utilizing advanced AI systems. And he's such a kind man he's willing to lead that organization himself.
Obviously someone integrating the chips to train AI, and having the ability to approve/deny his own competition is a huge red flag.
NVidia has a near monopoly on the AI hardware market right now, so some vertical integration of alternative AI hardware doesn't seem nearly as big if a deal if it is needed to fight that current monopoly.
Let's instead go back to when we actually enforced the anti-trust laws that we have. That was nice.
Ideally what we would see is OpenAI investing in a new but independently operated chip manufacturer that makes chips to their standards. Though is it also possible that a chip of such standard would be so specialized that it wouldn't be usable by others? I'm not a hardware person, so that's a genuine question.
That's competition. Apple Silicon completes with Intel at an ecosystem level.
Because anything to do with hardware, particularly anything with silicon, has immense startup costs.
As of today, I say if they go into consumer market they will fail.
If they go into specialised server chips, then they would have a chance, with some sort of accelerator over some arm-based chip - similar to what Nvidia is doing with Grace. Still big money to be spent on supporting the existing ecosystem on their hardware.
a 3mn mask costs what, $20 million each time?
using AI generated vomit for that would get expensive pretty quickly
It doesn't work out all the time, for sure. In fact it probably fails more than it succeeds. But the motivation is pretty clear.
Right now nvidia completely owns the AI market, and is exploiting that absolute monopoly with ever escalating pricing, licensing and restrictive usage models. Everyone keeps trying to escape this -- see Tesla and their super-hyped and now apparently abandoned Dojo thing, while they put in their orders for tens of thousands of H100s -- but instead they keep being beholden to nvidia.
If you have an algorithm that works and need scale, you must vertically integrate to maintain an edge over those using more general compute architectures.
I switched from Mac to Linux precisely for this reason and the biggest surprise has been that whenever I touch a Mac again it feels like poverty.
Not because the UX is bad but because I know how the company behind it operates and I have zero trust for anything that happens on the machine.
> Not because the UX is bad
I don’t see the connection. The first sentence says that walled gardens create bad UX, but Macs are the premier mainstream walled gardens and you don’t find their UX bad.
https://support.microsoft.com/en-us/windows/options-for-usin...
Which is why the EU has decided to break up the walled gardens - although IMHO they could ramp up their efforts a bit.
Using my Apple Silicon Macbook is a way better user experience than my Intel Macbook ever was. It certainly feels like they invested in user experience well after the walls were up around the garden.
Just because vertical integration occasionally works doesn't mean it is actually good for the consumer.
Omg this, can Apple please stop sabotaging Google Maps and Google Photos :(
Source: former Xoogler PM
I fully believe Google holds back differentiating features and DSPA is certainly mismanaged, but I'd hope these basic functionalities are not the ones since there's always the tradeoff between serving 50%+ of your mobile users and trying to differentiate Android/Pixel...
Apple does shit like make it impossible for Google Fi to set up easily on iPhone, so I just assumed most of the UX idiocy came from Apple's anticompetitive review process.
the eu disagrees in that it views itself as representative of the consumer. similar to how us states set laws that may be more strict than others.
get a large enough government of a populace with a large enough portion of the sales, and companies can be made to act. which can be good, but isn’t a guarantee.
just because companies can be forced to act doesn’t mean the forced actions are actually good for the consumer.
(plus a decent amount is political theatre.)
Don't like the way OpenAI treats your data, or how you can only run it in the cloud and not on an on-premises server? Or what dataset they used for training? You're out of luck!
But if the market were more modular, and lots of small companies could use the same hardware in their products, you'd have something to choose from!
If Nvidia, Amd and Intel were in a battle for offering the best VFM, none of these companies would be hopping into hardware. Apple's strong commitment to chip making coincides with years of stagnation from Qualcomm and Intel.
From my experience, companies love nothing more than a 3rd party that solves your problem for you, better than you and at a price that's easily cheaper than what I'd cost to build it in-house. This is especially true when the 3rd party product is an internal spec (gpu, cpu) rather than a competing platform (android auto)
There is a reason car companies don't build their own speakers or tires....but still try to build their own UI (no matter how bad)