Compiler and Runtime Specializations for Accelerating Managed Languages on FPGAs

I work in this field of research, and right now, there are many people trying to reach the ultimate goal of "anyone can take their existing code, and my tool will run it on an FPGA and will be 100x faster / 100x more efficient". I don't know if this is the right goal to reach or the best way to get there, but many players are highly interested (academic and industry).

On the hardware side, FPGAs are both a blessing and a curse. They are complicated. Building medium-large designs using existing tooling always takes on the order of hours. EDA tools are hard (open-source tools like VPR help but don't solve all the problems), but some more traditional software engineering investment in these tools could lead to huge improvements. One solution to avoid long compile times is making semi-generic "overlays" designs which are built once and moving the application configuration to runtime with some flexibility and performance tradeoffs (ex. OverGen). There is also some work in compartmentalizing the code into HW compilations units for better parallel compilation by existing tools (ex., RapidStream). You can also try to innovate the hardware itself (ex. monolithic 3D FPGA, CGRA, ...), which I think has the most potential for novel advancements in performance.

On the software side, LLVMIR and MLIR are the tools of choice, with examples such as high-level synthesis tools and also deep-learning compilers. High-level synthesis has huge potential, but most implementations limit themselves to subsets of C++ with pragma hacks for hardware optimization directives. You can stack tools on top of HLS that transpile high-level languages to HLS and then compile the HLS (ex. PyLog), and these tools can integrate directly into specific ML frameworks (ex. ML4HLS). In my work, I have gotten pretty good a building simple one-off Python-based end-to-end transpilers for subsets of deep learning architectures and can prototype FPGA demos within a week of work. Ultimately, these approaches are becoming outdated and not sustainable in the long run, leading to the exploration of more generic approaches. Ideally, I am interested in people trying to build new HLS tools (ex. XLS) or making first-class support for FPGA backends in an MLIR flow.

At first, I was really excited, because I really wanted this pipeline to work well. I've wanted an automatic FPGA workload offloader for awhile. Then I read that their "JIT" FPGA bitstream still took 2 _hours_ to build because it still used the built-in slow-as-molasses FPGA manufacturer bitstream assemblers. That's not a timescale that really supports a real, dynamic workload, unfortunately. Still, the work is neat and gluing together these parts in a way the works is really cool - what is essentially a Graal FPGA backend is pretty neat, if unlikely to be on its own paradigm shifting. Still, maybe the existence of something like this can prod Intel into making a fast mode for the bitstream assembler.

The section on memory management describes a lot of work the runtime has to do to track ownership and copy memory around. Would shared virtual memory greatly simplify this?