A Cloud-Scale Acceleration Architecture: FPGAs in Microsoft’s Datacenters
microsoft.com
microsoft.com
Microsoft uses FPGAs. Google uses ASICs. MapD uses GPUs.
FPGAs are more flexible. ASICs are faster.
https://www.mrcy.com/products/ensemble-fcn8213-server-class-fpga-processing-blade-advancedtca/
A nice little tidbit from the talk. They were able to translate all of Wikipedia from English to Russian using 90% of the deployed FPGAs in 0.1 seconds.
Microsoft/Google/whoever is going to use what makes the most sense for a given set of applications/etc they have, whether it's CPU's, GPU's, FPGA's, ASIC's, or massive armies of intelligent pigeons.
I'm not sure why one would think all of them don't do all of them :)
(in fact, i'd pretty much guarantee it)
Intel agrees with its purchase of Altera.
FPGAs are about as expensive as high end CPUs, can you afford to buy a high end server and burn half the cores on your OS?
On the flip side, if Microsoft can really eat that cost and offer FPGA space as a public cloud service, as an FPGA dev, I'd rather pay to play than bust 5K on a high end dev board.
Exciting opportunities ahead for those that can take advantage of hardware like this.
Look up systolic array architectures for matrix multiplication if you're not already familiar. They play nice with FPGAs (which are design space limited in comparison to ASICs). And can efficiently implement algorithms that are prone to combinatoric explosion by amortizing expenisve off chip memory accesses over multiple operations as the data passes trough the sys array.
If the FPGA manufactures want to open the advantages to all comers they'd make their design software free (as in beer, not freedom) an pAy for it with silicon sales.
<nitpick> Those were PALs, basically fuse-programmed logic-in-tables. Even a single of the later CPLDs could already replace a board or two filled with SSI logic. FPGAs took that to replacing an entire cabinet.
http://exablaze.com/exanic-x10
http://exablaze.com/exanic-x40
We also build switches this way too:
http://www.silicom-usa.com/pr/server-adapters/programmable-f...
Edit:
It looks like they are also allowing applications to use the FGPA-FPGA communication layer to reduce latency for distributed systems.
> "All network traffic is routed the FPGA, allowing it to accelerate high-bandwidth flows. An independent PCIe connection to the host is also provided, allowing the FPGA to be used as a local accelerator..."
> "By enabling the FPGAs to generate and their own networking packets independent of the each and every FPGA in the datacenter can reach every other one (at a scale of hundreds of thousands) in small number of microseconds, without any intervening. This capability allows hosts to use remote FPGAs for with low latency, improving the economics of the deployment, as hosts running services that do not their local FPGAs can donate them to a global pool and value which would otherwise be stranded."
1) As a local co-processor, they have been deployed to accelerate selected Bing search ranking operations. This is described a little bit in the white paper, but I think the most interesting parts are going to be proprietary.
2) As a network accelerator, because they are inline with the server network interface. In the white paper using the FPGA for crypto acceleration is described. This is cool. The CPU overhead of encrypting 40Gbps is significant and managing keys for this many hosts is probably non trivial. Moving encryption to another layer would simplify things significantly I think, let app developers not think about it and let datacenter peeps manage keys independently of whatever is going on in the host.
3) As a large scale distributed processing grid. Because the FPGAs have their own network interfaces, they can be used independent of the host. Regular host network traffic gets passed through unaffected while the FPGA is simultaneously running distributed compute tasks from the grid. The white paper described training a DNN, but also describes their Hardware As A Service delivery model, meaning developers I guess have access to the grid and can deploy whatever they want. I would guess there is a lot of machine learning and map reduce type tasks going here, but who knows. The whitepaper also tantalizingly contains the phrase "cross-datacenter" which implies a globally distributed network of these things. Rad.
So that's a long way to say that I think there are very many use cases for this kind of thing. Coupling the FPGAs with regular compute nodes is operationally beneficial I think - they can be co-processors or they can be part of the grid. Microsoft is doing some pretty cool shit.
Edit: This whole thing is called Project Catapult. There are some other good papers here https://www.microsoft.com/en-us/research/project/project-cat...
Network planes are done with FPGA and ASIC since a long time. Microsoft is late to the game.
Where I see FPGAs shining on the cloud is if vendors started putting them online so that clients could program them for their own applications. But I don't know enough about such things to know if there is yet a market for that.
Please enable chisel and put it in your Azure cloud. It will be a great platform for custom software.
Anyway -- doesn't Chisel just generate Verilog, like basically every other alternative HDL does? What's the problem with just plugging that resulting code into your synthesis tool and going at it?
Granted, the "plugging" part is the annoying bit, since most EDA toolchains seem to be fundamentally hostile to the outside world. But it doesn't really seem like Chisel should be the limiting factor here, at all. (As an alternative case study -- it was very easy for me to integrate a 3rd party HDL (Clash) with tools like IceStorm, but it did require some automation boilerplate)
And things like Chisel and Clash - despite my affections - are still actively evolving research projects, and they only concern themselves with one part of the overall process. They still leave many things to be desired -- for example, I'm still working out the best ways to integrate Clash with the formal verification tools of Yosys, in a way that doesn't just require me to shim some Verilog code. That might require improving the language/libraries to allow the notion of things like SystemVerilog properties. I don't know what it would look like to integrate multi-clock verification with the type safe cross-domain clocks of Clash, etc. There's a lot of things I still can't do with these tools alone, and because they're moving targets, that means I generally have to pick up the slack. They can certainly offer many advantages in terms of time-to-iterate and time-to-fabricate, but they aren't magic wands.
Furthermore -- and I really don't mean to be flippant -- if you're the kind of group who would benefit from FPGAs being deployed to accelerate your workload, but you're blocked on things like gluing Chisel code to your EDA toolchain -- well, you're going to be in for a very rude awakening when you realize that's an incredibly tiny, easy part of the overall process.
If these aren't a challenge for you, and FPGA development would actually pay off economically -- you're already in a decidedly different market than the vast majority of Azure customers.
Even then, there's also no indication it would actually be profitable for them - FPGAs are actually quite niche in their applications and require extensive development cycles in order to utilize to their full potential, especially for complex tasks. Microsoft and Azure employees can justify this because of economies of scale as of right now. It's a labor intensive process -- there's no indication a vast majority of applications would find a good use case for developing their own fabric, one that would offset the costs.
What's more likely is that people like Microsoft and Google will start offering you serves that are powered by this stuff under the hood, to give their platforms a competitive advantage. A better SQL server enhanced with custom logic is far more valuable to the vast majority of their customers than being able to program some custom fabric themselves, in today's world.
Microsoft already has custom fabricated Xeons with unlocked silicon for example, giving them private functionality to accelerate things like SDN workloads, and those Xeons power Azure -- why aren't you being given access to those? For the same reasons: those features only make sense to support, develop and utilize at a certain scale for the involved parties -- and it gives them competitive advantages when used to accelerate traditional workloads (that can only be realized with a substantial amount of work on part of the vendor, but is "free" for the customers.)
The venn diagram of "People who have a problem which FPGAs would solve" and "Customers who desire commodity cloud computing on Azure" is extremely small.
That could change one day, but it doesn't look like it's set to change anytime in the near future, especially given the landscape of EDA tooling, and the fundamentally different cost models associated with hardware vs software development.
We haven't reached a stage where fpgas make economical sense for small customers. GPUs were in the same stage 5 years ago. I hope Microsoft becomes a trail blazer in bringing fpgas on par with GPUs. They have one of the best tool-chain teams working for them, so it is doable.
I've only heard about chisel briefly, used in reference to a risc design I think.
I've considered making a lisp like language for design on a few occasions. But maybe chisel solves the pain points?
> I've considered making a lisp like language for design on a few occasions. But maybe chisel solves the pain points?
To be honest, I've only been doing FPGA design for a few weeks but, like: basically anything is better than VHDL/Verilog IMO. :)
Personally, I prefer CLaSH as opposed to Chisel. (Chisel is a DSL built using Scala - Clash is actually a compiler, not a DSL, from Haskell to Hardware. I think CLaSH conceptually is very nice.)
But honestly it really does seem to me that anything is a vast improvement over the current languages, for the most part. Your life is simply vastly improved by being able to do things like use a REPL in Scala/Haskell, or having cosimulation support in your toolchains (so you can run your designs as cycle-accurate software simulations, with no code changes, before doing synthesis).
Even then -- the intuition to use, or build something like Lisp isn't necessarily misplaced! Hardware description and functional programming share some interesting commonalities, it seems.
My point wasn't really about state machines or whatever. Languages like Verilog and VHDL are absolutely terrible at things like abstraction capabilities. You don't have modules, you don't have a REPL, you don't have any kind of abstraction facilities almost at all, on top of the languages being ridiculously verbose. They sit in some bizarre uncanny valley where anything takes a surprising amount of code to write, yet is terrible in terms of reuse, modularity, understandability, etc.
Example: A few weeks ago I ran into a case where I couldn't easily wrap a cell library primitive by wrapping it in a Verilog module, and then re-use that module a few times. The primitive was a PLL on my board. Oops, can't do that -- because the synthesis toolchain disliked a module around the PLL primitive, as the primitive had some arguments which were required to be constants (despite the fact all call sites to the wrapping module had constant arguments). So I had to use a Verilog macro which generated a Verilog module, and I had to instantiate that macro like 10 times to 10 different modules, each one corresponding to a different clock speed, with no reuse outside of a Macro. Because some parameters had to be constant. If Verilog metaprogramming wasn't just a rip off of C macros, it might not be so terrible. But shit like this is just tedious.
I can't even do things like use higher order functions, or strong typing capabilities, or things like leverage a real module system (with boundary and abstraction capabilities) -- even when I am completely in control of their use and completely understand how they will synthesize. Tools like REPLs vastly improve time-to-iterate with an interactive environment. Cosimulation being a primary feature of these tools means I generally get fairly accurate software simulations directly from my HDL itself, independent of any EDA toolchain etc.
The languages are just bad languages. Can you get work done in them? Yes, and everybody does. Do they accurately reflect the fact you're declaratively describing a hardware circuit? Yes. That doesn't mean Verilog/VHDL, as languages, aren't objectively terrible.
I understand why this is. The cost models associated with these tools -- and process -- are fundamentally different (for hardware, it's not about "determinism" like software programmers care about -- but things like MTTF, lead time, and optimizing around that), and Verilog is really a small portion of the overall lifecycle of developing a chip (you still need post-synth testing, possibly do gate-level debugging, post-synth modifications, on-board testing, etc). And tools like Clash/Chisel are not perfect by any means, and I still have plenty of slack to pick up by myself, nonetheless.
The languages are still bad, despite all that.
A lot of the trouble with FPGA development in my opinion isn't the HDL, but the toolchains themselves. I haven't met anyone that likes FPGA toolchains. Things like PLL instantiation problems are likely to do with the tool and not the language.
A lot of people (I'm not saying this is you) coming from a software background tend to approach FPGAs as if it were a software problem. They seem very similar, after all. You write some code into a window, hit a button, and bam, you have some functionality. To some extent, I think it leads them to tend to shove a square peg in a round hole. Just like there are programming paradigms governing software design, there paradigms governing FPGA design as well. Trying to force certain abstractions can lead to headache.
Abstractions are very important in any design process, be it analog hardware, digital hardware, software, bridge design, etc. Software tends to favor abstractions because they are powerful, but also very cheap. One can afford to waste some CPU cycles if it reduces development time and the occurrence of bugs. In FPGA and ASIC design, abstractions are just as important but one can't afford to waste clock speed or chip area to get it. FPGAs are expensive and ASICs even more so. The design paradigms of HDL exist for that reason, as frustrating as they may be.
In a sense, I've just grown accustomed to Verilog's warts. I was appalled at the FPGA development process when I first started but at this point I just take it for granted. I think thing will get better but it will take time. Even so, I think that it is best to stick with Verilog/VHDL for the time being. Just like if one wanted to web development, one would have to learn javascript, which is also considered terrible.
If you do it for your job, perhaps. However, if you're a hobbyist, that doesn't make as much sense. As a hobbyist you have the luxury of being able to use the tools you like the best. In any case, FPGA design shouldn't be closely coupled to any one language, they're just a tool that allows you to express the design you have in mind. If Clash or Chisel can help you express that design more succinctly and clearly than the competition, there's no need to stick to the other tools. The need only exists for professionals, as changing to languages with lower market share carries risk.
Generally, these tools are going to expect some level of familiarity with the base language, and most of the material around them is going to be rooted on that assumption. So if you've never done FP, or Scala, or Haskell, it'll likely be more difficult. Although I don't think its insurmountable.
I've been thinking recently it might be fun to write an article introducing Clash from a hardware design perspective, for people without prior Haskell/FP/etc experience. It certainly seems there isn't much material out there along these lines, IMO.
Lisp seems to be where I've settled in my programming. Mostly clojure, but I've been working through some common lisp as the JVM is a pain point for me. Inspite of all the libraries and ecosystem it enables.
I think I will perhaps continue with my idea of a lisp based hardware design language, as Scala is very definitely not appealing to me. I will take a look at clash though as Haskell was very compelling.
Thanks
If you're interested in a higher level hardware language, I'd highly recommend MyHDL or PyMTL. Both are much closer to Verilog (for better or worse) and give you higher level semantics.
I would think that the toolchains would be the biggest complaint.
If one wanted to do web development, javascript knowledge is important. It is the same with FPGAs and HDL.