eGPU: A 750 MHz Class Soft GPGPU for FPGA
arxiv.org
arxiv.org
The mind blowing part of all of this is the fact that they were able to close timing at 771MHz. That is insanely fast for an FPGA. For perspective, most modern FPGAs run their designs at around 300MHz* While most of the heavy lifting in this design use hardened components like DSPs and FPUs, it's still very impressive to see!
What I didn't see talked about much was how memory is loaded in and out of the processor. I'm curious to see what the memory bandwidth numbers look like as well as the resource utilization of the higher level routing.
*For most hardware designs that aren't things like CPUs and GPUs, you don't always need a super high clock speed. You have a lot more flexibility to compute in space rather than in time (think more threads running slower.) The pros and cons of such tradeoffs are a bit of a complicated topic, but should at least be noted.
It's true but I mean this is Intel in-house research right? If they can't get absolute peak fmax on their own parts that would be a really bad look right? Plus these stratix parts have hard FP blocks (not just DSPs) so they're basically mostly scheduling stuff rather building the whole datapath. But admittedly I haven't read the paper...
>Full Disclosure, I work for an FPGA company
I currently do too (as an intern, maybe even the same one as you) and I haven't looked very hard but I'm sure we have similar fmax achieving projects (maybe even GPUs since we're fighting hard to compete with Nvidia...).
IME, until fairly recently at least, Quartus loved to get fiddly on even the most basic designs when using Stratix class parts where you wanted to rely on, say, hyper-registers or retiming. Like a thing that should clearly be pipelined just... doesn't, and falls over from a gust of wind. It seems to have gotten quite a bit better in the last few versions, though, so seeing an unconstrained compile go this high is pretty good I think. Too bad they don't seem to have released their code, though.
i mean they don't say anything about how long they let it run for - constraints are to narrow the search space for human-time-scale benefit right? i'm sure if i let vivado run (i'm not an intel person) until heat death of the universe it would also give me some fantastic fmax numbers too.
> Too bad they don't seem to have released their code, though.
and they never will. i don't know what it is about this kind of research (or how you can even get away with this) but it's almost unheard of for the commercial groups to release their code/rtl/tcl scripts. i have emailed around before and i just continually get shined on - "oh we're planning to release but it has to go through legal". and this is from my capacity as a phd student ie not direct competitor. and it's funny to me because if you're selling these boards/part/fpgas aren't you strongly incenvitized to throw as much sample code over the fence as possible so that people buy your hardware in order to use your code?
I wish... this is a problem for general embedded things. We just recently bought a cubesat bus and the only way to interface our hardware with it is to send CAN bus frames to it. We asked them for an SDK and they don't have it. Instead they sent us a bunch of ICDs with screenshots of readthedocs snippets containing the frame structures.
The unfortunate answer to that question is that unfortunately we don't make purchasing decisions based on whether sample code exists, or detailed documentation quality for niche embedded hardware. Maybe because everyone is not giving those things.
I could not find the datasheet of the exact FPGA they used (did not spend much time) but Google says that Intel Agilex FPGAs (which is what they used) rated at 600MHz+ are available.
I'm not saying that they did something easy but they obviously used a fast FPGA.
I remember synthesizing a 16-bit Wallace tree in a lab exercise back in college. I think that single multiplier used up 70% of my LUTs.
You only will get massive amounts of hardware parallel multipliers if the underlying circuit has a ton of hardware multipliers (Like Xilinx's VLIW SIMD AI chips)
-------
At all computer sizes, a GPU probably will have more multiply circuits than an equivalent cost FPGA, with exception of maybe those AI chips from Xilinx (where the individual cores are basically presynthesized with hardcoded ISA).
Ex: at under 500mW power usage you probably will prefer some ARM NEON SIMD or TI DSP / VLIW. At cell phone levels you'd prefer a cell phone GPU, and at desktop/server levels you'd prefer a desktop GPU.
Very likely yes, but FPGAs often have hundreds to thousands of hardware multipliers, as part of the DSP blocks. Here for example newer AMD FPGAs: https://eu.mouser.com/datasheet/2/903/ds890_ultrascale_overv...
You're giving completely the wrong impression about dsp slices - it is absolutely not 1 dsp slice per FP operator at any precision that you would want to do floating point arithmetic. It's definitely at least 2 plus a whole bunch of LUTs (~500) for FP16 with 4 stages or something like that. And if you want faster (fewer stages) then you need more slices. On alveo u280, which is an ultrascale part, I have never been able to effectively utilize more than ~4000 dsp slices (out of 9024) for 5,4 mults and that cost basically 99% of clbs in SLR1 and SLR2.
And even then, disconnected FPUs are completely meaningless without a datapath implementing eg matmul and boy oh boy do you have no clue what you're in for there.
Takeaway: it's pointless to compare raw specsheet numbers when everything comes down to datapath.
Ain't that the truth.
However: Xilinx's DSP58 blocks (Versal devices), and older Intel DSPs, do integrate floating point into the DSP tiles - which does narrow the gap between the datasheet's DSP count and the number of floating-point operations achievable per clock.
this is correct (i am currently working on designs that target these parts) but those things are more expensive than comparably sized/equipped nvidia gpus so (in the context of this discussion) it's moot.
---------
FPGAs advantage is that systolic array arrangement. I'm pretty sure GPUs will win in a "hardware multiplier" war, but it might be harder to get all the data into a typical GPU
There's a fair number of programs that will never utilize a GPU with any level of efficiency. That's where FPGAs come in, utilization will be better because of LUTs and Routers and all that magic.
But if people are looking for a TFLOP fight with raw numbers of multipliers? My bet is on the GPU and SIMD-processing in general. Those architectures are just crazy.
--------------
BTW: That top end DSP slice is documented here: https://docs.xilinx.com/v/u/en-US/ug579-ultrascale-dsp
That's a 27-bit x 18-bit multiplier per dsp.
In contrast, a 32-bit floating point operation is 24-bit x 24-bit multiplier plus a bit of extra stuff for the exponent bits. I'm pretty sure that the 4096-shader Vega64 actually had 32-bit multipliers but I'm not 100% sure on that.
But the float 24-bit x 24-bit multiplier would need more than one dsp to replicate, maybe two of them. AMD's newest consumer 7900 xt GPUs are 6144-shader... but those shaders can execute _TWO_ multiplies per clock tick for a total of 12288 hardware multipliers operating at 2500 MHz clock.
For instance, in conventional GPUs, there is dedicated hardware for rasterizing the pixels covered by a triangle; there is dedicated hardware for doing texture map calculations and sample filtering. The instruction set disclosed in the paper doesn't contain anything to help accelerate those tasks. With enough code I'm sure one could get it to draw something, but it will be notably inefficient while doing so.
They don't actually explain anywhere in their paper what 'eGPU' means. Is it 'emulated'? I thought circuits implemented in FPGAs are not considered emulation?
I'm tinking the e means embedded
Is a hyphen missing, so it should be "750 MHz-class"? I searched the linked page but the word only appears in the title, sans hyphen.
- No (price or non-price) comparison/benchmark with any physical GPU of any sort;
- Seems to only be able to do FFTs;
- To be truly useful (or "General" - hey, they do mention _G_PGPU), it would have to be able to run OpenCL code;
- And on that note, what's the point, since (many) FPGAs can run OpenCL code directly?
Is there something I'm missing?
While it's true that they're in the dozens, strictly speaking, we've got GPUs with well over 100 now (the RTX 4090, for instance) and 64+ is a pretty common configuration.
Meanwhile if you had 64 passengers cars, then each of them would be able to drive a different route at the same time and if you counted the seats you would actually end up with 320 "Nvidia cores".