Hacker News Reader

Another thing is that you can pipeline, for example, multiplies. So in a CPU multiply you give the CPU inputs, wait a couple cycles, and then get the result. In an FPGA you can build a pipelined multiply. It's built such that you can feed it input every cycle and get an output every cycle. The only caveat is that the outputs are delayed relative to the inputs. i.e. you may give it (2, 3) to multiply on one cycle, but you won't see the result (6) of that particular input on the output until a couple cycles later.

[Yup, "pipeline" here is the same term used to describe how modern CPUs get their performance. They, too, are pipelining their instruction execution so that many instructions can be in the process of executing at the same time. Though what I describe is a more extreme and specific kind of pipelining.]

This is kind of like having a bunch of multiplies in parallel. But it's useful for, for example, real-time calculations. You get to perform all the calculations you need every single cycle, no matter how complex; your results are just delayed by X cycles. Pipelines are usually also more efficient than straight parallelism (i.e. an 8 deep pipelined multiplier uses less silicon than 8 individual "serial" multiplier units).

Another interesting thing: In a previous life I built an FPGA based video processing device. It sat in an HDMI chain, so it had to be real-time. If that had been built with a GPU, most engineers would build the system to buffer up a frame, perform the processing, and then feed the processed frame out. That results in at least 1 frame of delay. In contrast, because we used an FPGA, it was simple to just pipeline the entire design and thus only needed to buffer up the few lines that we needed. This meant A) we needed no external memory (cheaper) and our latency was on the order of microseconds. In my travels with that job I ran into tons of other companies building video processing devices. They _all_ used frame buffers, which made their devices unacceptable for, e.g., gaming.

Thanks.

Just to clarify whilst I have your attention: isn't it a common practice to pipeline frame buffers too? For instance, Android has three frame buffers. IIRC, one is with the user space handing off display lists, one used for composition, and one is used by driver for rasterization?

In that case, how good, you reckon, would the perf be compared to FGPA?

by straight having a frame buffer vs having a couple lines buffered, I'd imagine it's still the fpga, except now the GPU needs even more memory to do the same task

> Another thing is that you can pipeline, for example, multiplies. So in a CPU multiply you give the CPU inputs, wait a couple cycles, and then get the result.

Just to be clear: CPUs will pipeline high-latency execution units. Most arithmetic operations, vector, integer, or floating point, will have a reciprocal throughput of 1 (i.e., you can issue a new operation every cycle) even if their latency will be 3 or 4 cycles. See Agner's optimization tables for actual counts. The major exception to that rule is division, which has a reciprocal throughput of ~5 cycles or so and a latency ~3-4 times that.

> One clarification on the comment about latency. FPGAs are typically clocked much slower than a modern CPU. Typically, they run somewhere in the low 100's of MHz, whereas an Intel CPU clocks in at around 3GHz last time I went to the Apple Store. With a typical x86 multiply instruction having a latency of about, say, 3 cycles, putting that workload on an FPGA would result in a ~10x slow-down!

No, not that latency. Latency from external input to external output.

Like the latency of receiving an ethernet frame to the point you can decide what to do with it, forward, modify, etc. Or reacting to sensor input.

CPUs process interrupts slowly: X86 CPUs running a non-realtime OS can take sweet 50 microseconds+ just to get to run interrupt service routine let alone doing something useful with the data. Oh, and generally no FPU, SIMD, etc. is available in this ISR [0]. So ISR generally queues the event (on Linux a Bottom Half and on Windows a DPC, deferred procedure call) to actually process the event. The queued events might execute tens of milliseconds later in bad cases.

Microcontrollers can sometimes process external interrupts pretty quickly, it's not impossible to be done in 100 nanoseconds. But it really depends, and you usually have to be pretty careful to achieve this. And even then you often have significant jitter in the processing time.

FPGAs can start to handle an external I/O event on the same clock cycle, and might very well be driving I/O output pin a clock cycle later, if no pipelining is required.

So an FPGA might be done in mere 10 nanoseconds in some ideal cases. With little to no jitter.

[0]: Well, you can do FPU/SIMD in interrupt service routine, but need to handle all register state storage on your own, and if you mess up, it'll cause very interesting bugs in the userland application software.

FPGA latency is also much more predictable. Since you have cycle level control, you can know exactly how long any given operation will take. In a multi-threaded CPU, it's likely that the operation may be interrupted or delayed because of another process using the same resourt.

It's also worth noting that FPGAs are often much closer to the I/O than a CPU on a motherboard. Pin to pin latencies are much reduced in a situation where direct access to I/O is available. An Ethernet stack can sit on the same silicon without having to serialize data between the CPU and the north bridge.

>In a multi-threaded CPU, it's likely that the operation may be interrupted or delayed because of another process using the same resourt.

If you're considering an FPGA, isn't the alternative likely bare-metal CPU programming? In that case, you have just as much control over thread scheduling as you do over FPGA timing don't you?

Not since Ring -1 came to existence.

Throughput and latency are opposite ends of the same tradeoff. FPGAs enable cycle perfect timing control, while for bare-metal CPU programming.. even if you just have an infinite loop running bare-metal on one CPU, looking at the assembly can't tell you anything about the timing. Modern CPUs have multiple layers of caches with penalties for a miss coming in at hundreds or thousands of cycles. They do parallel and speculative execution of instructions. Instructions cost variable amount of cycles and take variably long depending on instructions before and after.

And then, with the SoCs that are everywhere now.. you are poking some memory mapped register, say you toggle a GPIO output value.. how long does it take to change on the actual physical pin? Is the GPIO peripheral part of the processor or some IP they bought in and then interconnected over an AXI bus? Is it buffered? It's all entirely impossible to say.

I feel like people are taking the hybrid approach of having a dedicated microcontroller on the same die as their CPU, like the PRU on TI's ARM chips: http://processors.wiki.ti.com/index.php/PRU-ICSS

This by no means replaces an FPGA, but if you just want your pins to flip and code to run at a predictable interval, this gets you that. (As for reading pins, they have another peripheral on the die that will just timestamp when an event occurs. This is much more consistent than dealing with your OS's interrupt handler, but again, it's no FPGA.)

Yeah, the TI PRUs are what you get approaching this topic from a processor manufacturer standpoint, the XILINX Zynq is how a FPGA manufacturers would do it.

Just to add Microchip with "Configurable Logic Cell" and Cypress with "configurable blocks".

SMM code from your firmware can still take over and add random latencies at the worst times.

For large calculations, the magic of an FPGA is in its throughput. Imagine that you have some mess of addition, multiplication, ... The time for the first calculation hardly matters. Even if the FPGA is slower getting through the first calculation that took 100 clock cycles on a CPU vs 10 on an FPGA, what happens on the next clock cycle? The FPGA cranked through an entire second iteration while the CPU is a few steps in. Next clock cycle? Now the FPGA has pushed another whole calculation through the pipeline.

My favourite personal example of this is using a GPU to perform an all-pairs nearest neighbour lookup in an image (for all pixels, find the nearest keypoint). That's something like 2 billion comparisons per image. A decent CPU (parallelised) took minutes to do that, at the time.

By far the simplest solution was to brute force it on a GPU. Probably took longer "single threaded", as there was no optimisation at all, but over O(10e6) pixels with a list of O(10e3) keypoints in shared memory it was basically instant.

It was a great lesson in premature optimisation. I could have spent days tweaking the CPU method with heuristics, sorting the inputs, etc. In the end it was less than 100 lines of OpenCL I think.