Graphics Chips Help Process Big Data Sets in Milliseconds | Hacker News Reader

Graphics Chips Help Process Big Data Sets in Milliseconds

technologyreview.com

41 pointsrobabbott44 comments

It's cool that it's essentially a consumer-participatory GPU database, but every time I read about GPU crunching I'm underwhelmed. The power waste and cost is absurd. FPGAs have become as capable, or more, than GPUs at a ridiculously higher level of efficiency. I understand that this platform is about technology accessibility... but having casual computer users burning their graphics card 24/7 seems cute but irrelevant to current methods of supercomputation. I find a lot of people who care about efficient natural resource usage don't think twice about very wasteful computing. Those watts are a whole lot of coal.

I agree. It only makes sense for certain types of workloads and for systems that are constantly in use (like supercomputers). This has been borne out through the movement of Bitcoin miner rigs from GPUs to FPGAs to ASICs. The power consumption and comparative performance of the GPU cannot win out over these devices.

You're right, I think BTC mining is our current best example of the trajectory of HPC. It proved from an economic point of view that GPUs are a kludge at best. BTC has really spread awareness about FPGA and ASIC technology. Highly capable FPGAs (not just the standard glue logic type) have mostly been the provence of HFT and the NSA until now. And finance has only really been hip for the past 3-4 years as they chase ultra-low trading latencies.

And now a MicroZed costs $199 and Digilent is releasing the Zybo at $149! Technology sea change, anyone? It's going to make coding REAL interesting again...

GPUs are no more a kludge or waste of power than CPUs. They just provide access to a different point on the efficiency vs. ease of programming graph. FPGAs may be getting cheaper but they're still much more difficult to program than GPUs. The sea change will come when someone makes an FPGA that's an order of magnitude easier to program.

Totally agree, that's why I mentioned this is about technology accessibility and really not much else. That's what I mean by kludge- it's far from ideal, but it "just works." And it's funny you mention CPUs- Von Neumann Turing machines are looking like a real kludge to me at this point as well... In my view SOCs like the Zynq are the real news, and those don't get talked about much on HN. Sequential PLUS parallel with high efficiency. So what if it's difficult to understand? Trying to really flexes the brain, and it's been worth it for me. But not easy.

That's why I wanted to start this conversation, I think it's germane to GPU efforts and people may not be aware. I would be more excited about a project like this spreading FPGA tech, and more effort being spent there. My company is doing it and we want a bigger open-source community with us. (We're not in a position to release code yet, but we will.)

Effort is energy. I'm hoping mentioning these technologies will give people pursuing projects like this some options on where to spend their efforts. There's a finite pool of people who are working in this space and I hope it grows.

Agreed that accessible FPGA SoCs are super exciting and I want to learn more about them. In the past when I've looked at these I've always been disappointed in the raw FLOPS available vs. GPUs, though it's difficult to compare directly, and I'm totally unfamiliar with the FPGA world. If I buy e.g. a ZYBO, make something cool, and want to scale it up to beat e.g. an NVIDIA GTX Titan in raw performance (4.5 TFLOPS), what are my options?

I guess it really depends on the application space you're interested in. We are building a cloud-FPGA platform but the barrier is just as you mention- the user being able do the programming. The idea is more people could scale their projects using the platform, but holy crap do we have a ways to go. We're going to lease out algorithmic services, basically, since getting a customer's design to work on our gear is more than difficult, there's way too much hardware variety. Portability is practically nonexistent, and that's what we're specifically trying to address.

Our approach for scaling is a monster FPGA board we're developing. It has a Zynq on it. It will be extremely inexpensive in terms of value next to, say, a highly-capable production server.

What kind of projects would you be working on if you bought a Zynq board?

Cloud FPGAs sound awesome. I'm interested in computer vision and machine learning, specifically "deep learning" neural nets. The current state of the art in neural nets is GPU implementations, and I think it's the right time to start thinking about FPGAs and eventually ASICs.

For deep learning, high performance in a single machine is important, as it's very difficult to distribute learning across multiple machines in an efficient way. A more powerful machine can train a bigger network in the same amount of time, and a bigger network can learn more complex things, with no known upper bound.

That's a great usage of FPGAs, and part of the platform will enable you to choose what algorithms you want to boot on your cluster slice. (In the FPGA space these pre-built algorithm circuits are called "IP blocks".) we will definitely be offering machine learning algorithms, with an eye towards minimizing data storage as much as possible. That may sound counter-intuitive, but there's a method to the madness (hopefully!)

I can imagine an fpga-cloud with an attached marketplace: you need performance for a certain algorithm, hardware engineers will compete and customize to your needs. I can think of quite a few engineers who'd love to participate.

That's the idea. I've just put my email address in my profile, please contact me about this if you like.

Your email is not visible, can you add it?

Actually I just reread and saw you used the word "compete", I had read it as "complete". Now that's a cool idea you have there.

Will this be similar to Parallella at all?

Parallella is an incredible project that I'm rooting for passionately. I am really looking forward to contributing to it, but we're more commercially oriented. Parallella is an Arduino/RP kind of project, and I have a feeling I will learn things from it. I, personally, still have a lot to learn in this space. Actually it feels kind of like a boundless area for learning. I really believe this is the next phase of computation, and that programming for non-Turing machines is the step that will prepare us for spintronics and quantum. I can't imagine Windows for Spintronics, I just can't. Our current software platforms, especially OSes, are going to have to change, and we want to start now. But the path is unclear- the full potential of reconfigurable computing has not yet been realized. That's why I find it exciting.

Do you think that there will be a computer in the near future that optimizes frequent workloads onto FPGA circuitry? Kind of like the storage optimization that occurs in certain SAN hardware now? This would require an OS component capable of identifying candidates from the workload and dynamically porting that to a hardware layer on the system.

Absolutely. That's a major part of what we're working on. The Zynq, specifically, allows for this more than has been possible previously. When your ARM(s) and your programmable logic are on the same wafer, you can kick tasks over more quickly. And shared resources are a big boon. This obviously is a gross oversimplification, and not straightforward from a x86 development point of view. The ARM runs Linux, or you can roll your own bare-metal OS for your application, and it plays the role you mention.

Now you're just making me jealous... ;)

I really love the idea of cloud-fpga solutions, but I can't begin to fathom how this could work without doing the fpga "coding" in a high level language. The simulation speed of verilog is too painful...

That said, working with the GPU requires a very specialized set of high level constructions as well; I imagine that most workloads could have significant parts of their computational path abstracted out to pretty common easy to port fpga processes.

If you ever launch a beta...let me know!

You're right on the money... our platform will be exposing high-level languages to the user, specifically, functional, dynamic, interpreted ones. That's our ace in the hole. I'm intentionally not mentioning our company name because it's not my intention to promote here; that would pollute the conversation. That being said, I've written your username down :)

The sea change will come when someone makes an FPGA that's an order of magnitude easier to program.

I think it's not as much a matter of making an FPGA that would be easier to program as it is a matter of coming up with an order-of-magnitude of improvement in programming techniques. What you're demanding is not far removed from saying that better programming models require improved transistors.

That would be true if FPGAs were just arrays of undifferentiated transistors, but that's far from reality. FPGAs have DSP blocks, embedded memories, and other hardware elements that make a big difference in how they are programmed. It's not ridiculous to postulate that a hypothetical order-of-magnitude-better FPGA programming environment might benefit from or require hardware changes.

I don't think power is relevant here, because the GPU is being used to reduce latency when running queries interactively, so it's only on for a small fraction of the time.

You can use OpenCL, which is normally used for programming GPUs, on FPGAs http://www.altera.com/products/software/opencl/opencl-index....

Also see http://www.anandtech.com/show/7334/a-look-at-alteras-opencl-... for more information

Someone correct me if I'm out of line here, but I think it would be appropriate to mention your personal interest in FPGA's (which I discovered from your comments lower down).

On the point itself, "accessibility" is probably understating the nature of the problem. Actually being able to program for a platform, and how easily you can do it, is a huge issue. Issues of "accessibility" apply as much to huge companies with giant server farms (note how little even GPU computing is used by them) as they do to tinkerers.

You're not out of line, and I haven't tried to hide the fact that I'm now professionally interested, in fact that's all I'm blabbing about below. But that's no coincidence- this is why I got into FPGAs: the problem spaces that are like what this cool project addresses. But I rejected GPUs as a platform completely for the reasons I list. And that's how I came to be interested in FPGAs. I am not an EE and it was a hard decision to make the switch to pursuing this, and I'm sharing that experience in the hope of inspiring/meeting others. As well as trying to open myself up to input/advice from people more experienced than I.

Fair point, as often happens I forgot to consider the possibility that your preference for FPGA's caused you to get a job selling an FPGA platform and not vice versa.

In any case, my own viewpoint is that CUDA and OpenCL are the best that has been produced by a huge effort by well financed and technically sophisticated groups. GPU computing offers advantages in flops/watt and flops/dollar, and yet is still not widely utilized because of the increased programming complexity.

Given this, I think it will be very hard for a small group to compete using a completely new architecture. On the other hand, you are starting with a blank slate and you are controlling the entire stack, so I hope you can use that to your advantage.

It's silly to make blanket statements like "FPGAs have become as capable, or more, than GPUs."

It depends on what you're trying to do! If you want to multiply big matrices, GPUs are great for that. A lot of scientific applications work well on GPUs. And obviously, GPUs are great for graphics.

Also, FPGAs are no longer blank templates on which you can stamp any design. The new ones all come with built-in "IP blocks" which you can't change. So you get a bunch of gates you can modify, but also perhaps dozen CPUs and a bank of memory, or so. Maybe someday GPUs and FPGAs will converge-- the former are getting more flexible, and the latter are getting less so.

The biggest problem with contemporary GPUs is that they're I/O-starved, which I don't see anywhere in your comments. PCI-e is just not enough bandwidth. That is why GPUs are a sideshow in big data. It doesn't matter how many cores you have if you're sipping your data through a straw.

The power consumption argument seems like a strawman. Replace a few incandescent lightbulbs in your home with LED ones. Congratulations. You can now run your GPU 24/7 and come out ahead in power consumption.

>> It's silly to make blanket statements like "FPGAs have become as capable, or more, than GPUs."

Umm, you took "at a ridiculously higher level of efficiency" out of my quote in order to call it silly? C'mon man.

If you've ever diligently crunched numbers on FPGA versus GPU monthly electricity costs at scale, I don't think you'd be capable of saying power efficiency is a strawman. If the idealistic efficient-resource-usage aspect is not convincing enough for you, we are talking millions and millions of dollars here (actually more). Look at Amazon GPU numbers and get back to me. We have detailed breakdowns of price comparisons that have been worked out in depth using the same algorithm. This is nothing like swapping out lightbulbs, that's an actual absurd and uninformed statement that can be verified and quantified as such.

And I do mention both IP blocks (even in quotes as you do) and how SoC's CPU-PL communication is much faster from being same-chip-integrated, hours before you did, so I'm not sure what you mean.

I'm also not getting why you use hard IP blocks as an argument that FPGAs are getting less flexible. They are helping to advance FPGA accessibility and flexibility a ton. Why do you think they're there? The development time savings are huge and the testing and optimization is by nature better.

The 7000 series has pretty heavy options, you can get a lot PL all your own on them. We've been specifically talking about an ARM+PL SoC, are you trying to say that an FPGA like a Virtex or Kintex is somehow not a "blank slate"? You want no standardization of any kind? The lack of that is a huge problem, and it's getting solutions.

I wasn't trying to be negative about FPGAs. I was just pointing out that there are good reasons to use GPUs in some scenarios. If your company can put FPGAs within reach of more people, that would be awesome.

I used to work at a startup company that had a plan to reduce power consumption in data centers by spinning down hard drives when they weren't in use. Too late, we learned that there isn't a lot of money in saving power. The average data center in the US might spend $500k/year on electricity. That sounds like a lot, until you consider that the fully loaded cost of a good engineer will be around $200k/year. Power becomes a factor only when you hit a wall in terms of how much can be delivered to the data center.

I was an electrical engineer in college, and I learned how to write Verilog and do register-transfer-level design. It's something that I don't think most software engineers have the background to do. Currently, I work in the area of big data, on Hadoop.

I cannot deny that Hadoop is not very power-efficient. But it offers some things that are more important to customers: infinite scalability, the ability to run on commodity hardware, and the ability to interface with the system via normal-looking software.

Actually, even writing Java code is too difficult for many Hadoop users now. They prefer to write SQL. (Even SQL is too hard for some, and they use automated tools to generate the SQL and produce charts.)

As you talk about your background more I'm getting where you're coming from. GPUs are obviously more powerful and capable for high-end graphics disregarding power efficiency. And if you're doing Hadoop big data your perspective makes more sense to me, because storage-centric applications really have no use for FPGAs. Maybe they'll be useful for querying one day, but they sure can't store large datasets. I get your point that FPGAs are not a universal panacea, but in the case of GPUs for supercomputing, and not graphics tasks, I think they are.

In the past, GPUs weren't that practical for big data, since they offered a lot of CPU power, but not much in the way of I/O. It's possible that PCI-e 3.0 and 4.0 will change that, since they supposedly offer up to 15 GB/s and 31 GB/s bandwidth.

Big data is more than just querying. There are people running fancy machine learning algorithms. Those are the people most likely to use the flexibility of having a Java (or other programming language) interface.

I'm curious why you think FPGAs will win over GPUs in HPC. I struggle to imagine academics writing Verilog or VHDL. I can just barely see them sending in legions of grad students to try to write CUDA or OpenCL, but RTL design seems a bridge too far. I also haven't heard of any of the big FPGA companies trying to make a splash in HPC, whereas NVIDIA has been very active with Tesla and its other high-end offerings.

I wonder how it compares to: http://wiki.postgresql.org/wiki/PGStrom

(in terms of architecture, not performance benchmarks)

In addition to GPU, Cray has recently added the Intel Phi coprocessor to its XC30 Cascade supercomputers (http://investors.cray.com/phoenix.zhtml?c=98390&p=irol-newsA...). I think that this supports the argument that certain problems are better handled on traditional processors than on GPGPU platforms.

The Xeon Phi resembles a GPU much more than it does a CPU. The original project, Larrabee, was at one point meant to be a GPU that competed with NVIDIA and ATI.

It's a multicore x86. It resembles GPUs only in the sense that it sits on its own PCIe board and has a lot of cores, but the programming model is just x86. It would have been very different from the NVidia/AMD competition if it had ended up as a GPU.

One of the biggest obstacles to GPGPU exploitation is being at the mercy of each vendors proprietary software stack and the resulting fragmentation & lack of openness. It's like the pre-PC era, without Unix... Larrabee might have helped. Now Xeon Phi is a niche product.

It is an x86 with very weak in-order cores, which instead have very large (512-bit) vector units. Very much like an NVIDIA "symmetric multiprocessor". If you try to program the Xeon Phi anything like a general-purpose CPU, you will not get within 5% of its power --- the programming model is essentially that of a GPU. The one thing that makes the Xeon Phi more general-purposy than current GPUs is the cache-coherence across cores.

They are programmed the way shared memory parallel machines have been used since time immemorial in the HPC world. Hundreds of cores is normal there. The same Intel software stack (OpenMP, Threaded Building Blocks, parallel Fortran etc) is used, same one that Intel markets for regular HPC.

Yeah, you have slower cores and the SIMD is twice as wide, but its main selling point is that you get to keep the regular programming model - unlike with NVidia or AMD. Along with benefits that come with it (mature software, openness, etc).

See eg. Intel's Xeon Phi Programmin Guide for an intro: http://software.intel.com/sites/default/files/article/330164...

Has the guy released any code yet?

I haven't seen any releases yet. One article says he plans to open source it within the next year.

Hey, MapD author here (Todd Mostak). I'm working on a release now - hopefully it will be out by the end of the year! Would really appreciate help from anyone interested in the project - todd@map-d.com

Todd - Would love to help you out. I will send you my email.

That sounds really interesting. I will send you an email.

As a side note, I was excited to see that there was some geospatial support in there. That's certainly piqued my interest! How much of a focus was spatial support?

Reply on news.ycombinator.com