Parallel Programming for FPGAs | Hacker News Reader

Parallel Programming for FPGAs

github.com

80 pointsbjourne32 comments

When joining my current research lab, this book was a god send. For context this book is directed at users who are interested in high level synthesis which at the simplest level let's you write C++ code to convert to hardware constructs in Verilog or VHDL. HLS is really powerful and have allowed us to develop complex hardware like deep learning accelerators much faster but at the cost of poor tooling and many hours of debugging broken tools. Documentation is "good enough" with non existent support beyond your lab mates and your experience. I hope that the eventually support from industry tools will get better, open source work on this area will be developed more, or labs are more willing to share their knowledge and "secrets" with other on how to get the most out of these tools.

How does one actually program on an FPGA? I know for GPU, you can buy an nvidia graphics cardm, rent an aws gpu machine or use google colab. Is there some consumer facing equivalent for FPGA programming?

There are some development boards like the Terasic DE10 Nano. It's sometimes a bit hard to come by because it's used by Misterfpga (a retrogaming fpga based simulator) but it's perfect to learn to develop on.

Other than that I really recommend 8bitworkshop and the Designing Video Game Hardware in Verilog. It's a great way to get started without an actual fpga board to test with.

You could just download the Vendor tools (Xilinx, Intel, Lattice) and start with simulations. They are usually free of charge for private use. Often including their smaller chips. For Xilinx you could, for example, get the Digilent Arty-A7 board, which is based on Xilinx' Artix-7 chip and has plenty of space. There are also free tools for Simulation like GHDL+Gtkwave, but i am completely unsure if there is any that lets you do HLS

FPGA hardware is expensive though

There are some FPGAs that have been _fully_ reverse engineered and now have open source toolchains, such as Lattice iCE40 and Lattice ECP5(-5G). Others are coming along, such as Xilinx 7 series and Lattice's Nexus FPGAs (CrossLink and Certus products).

See: https://f4pga.org/

The open source tools operate at an RTL level (register transfer language, such as SystemVerilog) though, I'm not aware of any open source HLS tools.

Problem at the moment is that FPGAs are being hit really hard by the current global supply chain problems. They're almost impossible to get unless you're a billion dollar corp and the prices are going through the roof for the available supply.

As far as boards go, there are many, many out there. Some of my favorites are:

Alchitry Au: https://www.sparkfun.com/products/16527

Olimex iCE40HX8K-EVN: https://www.olimex.com/Products/FPGA/iCE40/iCE40HX8K-EVB/ope...

ECP5-5G evaluation kit: https://www.latticesemi.com/products/developmentboardsandkit...

Defintely should also have a look at Ice Studio: Visual editor for open FPGA boards. https://icestudio.io/

There are a few niches where FPGA make sense, but for things like DSP they naturally parallelize ops well.

The LCMXO3D-9400HC-B-EVN demo board from Lattice can still be found for $60, and Verilog is the most common dialect to learn.

If you are from the sequential CPU/MCU world, than expect to see some very different paradigms. The language define a logic circuit where states propagate in parallel (within physical limits)... latch-up avoidance and clock domain crossing must be handled properly.

If you are good with hardware, there is also the $20 EBAZ4205 (Xilinx ZYNQ 7Z010 fpga has a cpu that runs 32bit linux) since the crash of various crypto currency trends.

T[p: the next generation of Desktop CPUs are supposed to include configurable logic similar to current ZYNQ chips. We shall see what this means when the other boot drops... Good luck... =)

Do you have any information on the next gen of desktop CPUs including configurable logic? I'm in the FPGA space and have never heard of it.

I would recommend watching the TechTechPotato youtube channel interviews. They cover a lot of domain specific hardware. ;)

You aware of the AnandTech channel and their podcast?

https://www.youtube.com/channel/UC6Qjj9Jpww3iQarwt8HOcDw

I could give you a list of channels that do similar content to TechTechPotato.

I searched for FPGA in his videos but nothing came up. Could you link me the relevant video? I'm interested in it!

Sorry I can't help you more, but if I recall correctly it was in a summery at the end of an interview with one of the major players. Likely AMD or TSMC related given Intel/Apple/Qualcomm is more secretive about future products, and I seem to vaguely remember something about ML accelerators and chiplets internal bus bandwidths as some of the subjects. However, as I tend to binge watch the episodes once in awhile, and the set never notably changes much... it is difficult for me to narrow down when the episode was published. Good luck. =)

You can use a simulation program. One of the professional ones that I like is ModelSim.

You design your chip in VHDL or Verilog, compile it, and then make a (virtual) testbench in ModelSim. Then you run your design in ModelSim.

You can have a look at signals while it's running the simulation in simulated oscilloscopes or you can script a better view in Tcl/Tk or you can have the testbench assert stuff. The latter lets you do something like unit tests. I do that a lot.

Note that simulations are lies. So it's helpful to find obvious bugs--but in the real world all the signals run IN PARALLEL, and they can grow phase differences depending on the routing (which is generated). The simulation cannot test all the combinations (but of course the compiler tries to make the gate to gate timing work on its own).

Open source variants exist. I like https://github.com/nickg/nvc.git . In includes a compiler (from VHDL) and simulator. You can have nvc generate waveform outputs and look at those in gtkwave. It's neat. But it's not as full-featured as ModelSim.

Eventually, you can buy an FPGA development board like the Terasic DE10-Nano and download the program Quartus Prime and actually synthesize your design and load it on the board. You need a REALLY beefy computer to do the synthesis in finite time.

You use a tool to convert your HDL code (Verilog, VHDL or something else) to a bitstream which then needs to be sent to the FPGA to configure its components. Most of these tools and the bitstream formats are proprietary. So, for Intel you want to download Quartus, and for Xilinx you want Vivado. Both are free but I have only used Vivado, which you only need to buy if you want to use it with very expensive FPGAs. After the bitstream has been generated, you need connect your board to your computer to send the bitstream to it, and the board handles the FPGA's interface to actually program it. As a consumer you can buy bare FPGA chips and make your own board, but for learning and prototyping you probably want to just buy a development board. Digilent has some very nice ones with various peripherals.

There are places you van rent an fpga, it looks like azure offers this for example.

For programming, you need a language (eg. Verilog, VHDL), a toolchain (most commonly the vendor one), and a target chip on some kind of board, either rented or on your desk.

> How does one actually program on an FPGA?

Traditional: as others have pointed out, create design in Hardware Description Language (HDL). Simulate / test. Use tool(s) of choice (or usually no choice) to synthesize. Resulting bitstream written to FPGA. Ref https://en.wikipedia.org/wiki/Field-programmable_gate_array#...

Somewhat newer: High-Level Synthesis (as detailed in the OP). Similar to HDL, but more abstracted design. Ref https://en.wikipedia.org/wiki/High-level_synthesis

> Is there some consumer facing equivalent for FPGA programming?

AWS F1 Instances (https://aws.amazon.com/ec2/instance-types/f1/), or dev boards (https://digilent.com/shop/boards-and-components/system-board...)

Slightly off-topic: I see lots of learning resources for FPGAs appearing that explain how to "think parallel", but are there good resources that explain how to deal with practical problems such as badly-documented primitives (looking at you, ECP5) or how to debug a non-working design (1), or how to find what causes the synthesis tool to segfault?

1 involving multiple primitives and you can't attach a signal logger to the signal between them because the FPGA doesn't have a path there to use.

The situation somewhat reminds me of the earlier LaTeX days when every tutorial started with "I'm assuming you have already installed LaTeX" as if that wasn't the biggest problem for most users, and then continues explaining the easy stuff that you can easily learn by trial and error.

> how to debug a non-working design

In LiteX, an open source toolset for making FPGA designs, you can embed an oscilloscope into your FPGA: https://github.com/enjoy-digital/litex/wiki/Use-LiteScope-To...

How does it solve the problem that the FPGA doesn't have a path from the signal-to-observe to the logic blocks?

You can specify the list of signals to observe, which is built into the gateware.

As memory is also limited, you specify a condition like if signal A becomes 0x1234, then log the aforementioned signals for N cycles.

I have to admit I still don't get it. When I try to log the signal using my own signal logger (which I guess is somewhat like that "virtual oscilloscope"), the synthesis tool rejects the design and tells me that I cannot use that signal as an input because the FPGA doesn't have a path from there into the logic blocks. What does LiteX do differently to observe a signal that my own gateware cannot access?

What kind of primitives are you talking about here? In my experience you don't really have to manually instantiate anything other than a PLL or maybe a multiplier unless you're doing something very specialized.

I was specifically thinking about DDR3 related primitives.

Only simulator could check signal to which could not connect physically.

But, you could split project into small parts, connected by some interface (could use triggers or 2-way ram cells), and debug them separate.

Sure, you will loss some capacity, but this is life.

This is not equal to test real solution, because physical arrangement matter, but most probably, you will find most obvious mistakes.

>1 involving multiple primitives and you can't attach a signal logger to the signal between them because the FPGA doesn't have a path there to use.

Does the problem not arise in simulation?

I don't have simulation models for the primitives, and I can't write them due to the lack of documentation. One of the downsides of working purely with the free / open-source toolchain, I guess.

edit for clarification: The lack of simulation models is due to the free toolchain. The documentation is fully available, it's just so bad that you don't even know what some of the primitives are meant to do.

You're not gonna like this answer but every time I've had some issue like that at work the answer is: file a ticket with your local Xilinx field applications engineer.

Are you a hobbyist? Ha, have fun. Digging through the Xilinx forums is also productive.

Edit: I see you're strictly working with only free/open source tooling. I commend your patience. I also don't see the point. It's not the right choice for either learning or getting things done at the current moment.

> You're not gonna like this answer but every time I've had some issue like that at work the answer is: file a ticket with your local Xilinx field applications engineer.

Basically this. I've been using the Intel OneAPI tools to write for their FPGA's in SYCL, and seem to have run into a large number of fundamental bugs (including plain routing errors) that makes me wonder how many other people are actually using the tools.

Well, now you know the answer is almost nobody :).

Xilinx and Altera have been promising efficient high level C++/openCL/CUDA -> FPGA compilation for decades now and almost everybody I know has been burnt by it at work, so thanks for confirming my bias, I'll avoid OneAPI too for the present. Guess VHDL and Verilog and the old standard tooling will live for another 10 years.

Everything is always amazing for the first 80% but the last 20% is impossible to fix and you end up not saving any dev time at all.

> I'll avoid OneAPI too for the present.

I wouldn't write it off _completely_, for everyone. Compiler bugs notwithstanding, it seems to work reasonably well if you don't mind extremely long compiler cycles for running on the hardware. It gives you pretty reasonable emulation tooling to work out how well it runs, whether things are pipelined and, efficient if you know what you are looking at - but it doesn't seem to save you actually having to learn how the FPGAs work and model data flow.

Of course, you are also beholden on drivers/adaptors being written to actually be able to use them.... which at this point is two chipsets, the newer of which has just been deprecated. Evaluating Xilinx HLS is next on my list, at least it seems to have the benefit of being around for much longer, and we have collaborators who have had success with the tooling.

It's not so much that no one is using it. It's more that the only people who are using it have dedicated engineers in the compiler team developing the tool who will dig through the compiler and figure out/fix bugs. Literally even within Intel, internal customers would have to fund head count in the compiler team in order to have a dedicated engineer who could support them in figuring out how to tailor their code for the compiler or fix the compiler to work with their code. And since that process involves hand-hacking the compiler it introduces far more bugs than it fixes.

> Edit: I see you're strictly working with only free/open source tooling. I commend your patience. I also don't see the point. It's not the right choice for either learning or getting things done at the current moment.

A bit late, but my original point was that so many of the tutorials appearing these days that claim to explain FPGA development, especially to people with a software background, don't deal with the practical problems that a SW dev has with FPGA development.

BTW this doesn't only apply to the open toolchain. The segfault problem specifically appeared with closed tools from the FPGA vendor. From a SW dev experience this is definitely new territory because things like javac, tsc, React, Hibernate, and log4j don't segfault if you look at them the wrong way, and even if they did, you can look at the source code and find what is causing the problem. None of this is possible with a closed, vendor-specific synthesis tool nor with a blackbox FPGA primitive with close to no documentation, and while it is often suggested that "FPGA development is hard because you have to think parallel", none of this is even remotely related to that.

Reply on news.ycombinator.com