Surely GPUs or similarly very parallel machines for things like ML training are very needed and will remain very needed. Seeing as firms such as Groq have done fine, surely Intel can do something of that sort without much difficulty?
Since their GPU business has been unsuccessful perhaps they can go for whatever makes sense, as there's nothing of this sort that they can release that will compete with their own products.
Of course, there's the elephant the room for general purpose tensor machines, which is CUDA - famously closely guarded Nvidia. But with the new wave of "above-CUDA" APIs like TensorFlow, PyTorch, and Keras, there's an opportunity to skip the CUDA layer altogether.
The businesses that should are the startups and businesses who are helping define the model, like ChatGPT and Microsoft, but the adoption is still too early. If I was McDonald's or Wendy's and I wanted to use AI to help promote sales to customers I would need to be able to grab demographic data which may not be appropriate PII data.
All of the lawsuits happening right now for data collected without permission of the data provider is going to all change the landscape.
Stable diffusion models are awesome and super useful. I don't necessarily mean the simplistic text->image stuff (though it's clearly useful)-- but denoising, in painting, animation from stills, 3d from stills.... style changes, etc.
LLMs are at least somewhat useful.
We can radically improve image and video compression with performant enough AI.
Plenty of other applications of AI right now are already useful or would be useful if inference were made more efficient or more local (thus private). No more data needed.
You don't need some grand vision of an AI future to get a total addressable market for high performance AI that is in the same ballpark as high performance GPUs for non-AI usage.
In practice AMD and Intel GPUs should be suitable for machine learning but the software story isn't good. I know I can buy a NVIDIA card and know it's a good investment because everything worth with CUDA and be training models in less than a day. If I went with some other brand I'd expect to put 6 man*months into figuring out the software story, and that's a lot more than the price difference in the cards.
I've been wondering about the soundness of Intel's software strategy in that OneAPI ("One" is a bad smell in marketing speak, the only premium product in my mind that has "One" in the name is "Purina ONE" pet food, the XBOX ONE is an astonishing own goal of a name because you'd never get your mom to understand that an XBOX ONE is better than an XBOX or XBOX 360) is based on OpenCL and the one thing I know about OpenCL is that I don't know anyone who likes coding for it. The idea that you could code for FPGA and GPU out of the same API also seems delusional because FPGA is (mostly) about latency and GPU is about throughput. That is, FPGA can do certain small operations insanely fast and avoid the overhead of 16-bit math when 13-bit math will do, GPU is all about doing large operations in bulk.
Several companies are spending >$10B annually on AI compute. There are a lot more than 6 man-months of savings in it for them...
A company like OpenAI that is mostly concerned about pace might see going with the #2 GPU vendor is risky, even if it has the possibility to save money.
Saying OneAPI/SYCL is bad "because OpenCL" is the same as saying OpenCL is good "because CUDA" (since the NVIDIA implementation of OpenCL is called CUDA and uses the larger CUDA stack).
As for the FPGA/GPU code sharing, iirc FPGA vendors including Altera/Intel and Xilinx/AMD support OpenCL as well. The idea doesn't sound that crazy to me: GPU code also often uses small data types (fp16, tf32, int4), and both work with data parallelism. GPUs operate in parallel with SIMD vectors, but a compute accelerator FPGA achieves a similar thing by essentially pipelining the program. Effective pipelining requires parallelism too, since a serial FPGA program only has one stage active, similar to how a serial GPU program only has one ALU per compute unit active. As such both are mostly suitable for massively parallel programs.
Thank goodness other people recognize this too. Whenever I see “One” in branding I sense a corporation bereft of creativity and afraid of risk. Alternatively, it might also indicate that were too many executives involved and no single person had the authority to make a decision, so they picked the most bland and uninspiring option they could all agree on.
It’s so lame and milquetoast. However, there is one exception I will make, the radio station 101.1 “The One”. Here it feels appropriate and even a little endearing.
Naive question, wouldn't you need a descent tool chain for inference as well ?
“Closeness to CUDA” is less important for inference because all the experimentation is already done by then, and if need be you could just implement the model using Gaudi ops to begin with, in a span of a few days including tuning and debugging.
"N5, PCIe 4.0, and HBM2e. This chip was probably delayed two years." -wmf
Nvidia ate the whole pie because they were in the right place at the right time with shovels to sell. Getting into it now, in house, would be a huge bet that the pot of gold will be there when they get to the end of the rainbow -- and they have a long way to go.
They say the opposite of love is indifference, and basically everything Intel has sold outside of CPUs and NICs have suffered market indifference. Actually, maybe even the NICs given everyone seems to prefer Realtek NICs over Intel NICs these days.
Nvidia and AMD have it way better because if the market doesn't love them, you bet the market will hate on them instead of be indifferent.
CUDA is an API. OpenCL is an API. Pytorch is not.
Pytorch is very firmly GLUED to CUDA. It will probably NEVER support anything else beyond token inference on mobile devices. The only reason Pytorch supports AMD at all is because of AMD's "I can't beleive it's not CUDA" HIP translation layer.
OpenCL is a real cross-platform API and with 3.0 it's finally "good" and coincidentally, intel is....half-heartedly interested in it, except they're shooting themselves in the foot by trying to also cover useless CPUs for inference/training and spreading themselves too thin (OneAPI). Because all intel can think about are CPUs. Everything must drive sales of CPUs.
At this rate just about the only think that might save us from CUDA is rusticl. If a real, full-fat, high-quality openCL 3.0 driver sudently popped into existence on every GPU platform under the sun, maybe pytorch et al could finally be convinced to give a shit about an API other than CUDA.
It is a tiny thing, but Intel really seems to charge a premium for PCIe lanes. Maybe if they sell a lot of dGPUs they’ll be less stingy with the sockets to plug them in to…