Comment by spangry | Hacker News Reader

Comment by spangry | Hacker News Reader

If Nvidia are looking to supplant Intel, I think they key, weirdly, is i/o. In a sense, GPU acceleration of general purpose computation is just a specific case of the more general case of distributed computing (albeit the computers are extremely 'close'). For embarrassingly parallel workloads, i/o is not an issue; the data and computations can be batched, distributed and no further communication required (until you collect the results).

But for mixed, semi-sequential/semi-parallel workloads, I/O becomes the bottleneck. Having to retrieve intermediate results, combine and then redistribute can eat up your gains from parallelisation. It will be interesting to see if Nvidia start pushing for a new standard to replace PCIe, and maybe invest in low-latency, high throughput networking R&D. Who knows? Maybe they'll just build some kind of networking functionality in to their GPUs directly, so data can be transferred NVRAM -> NVRAM without having to travel up and down the stack (until they need to interact with userspace; once on the way in and once on the way out).

Given there are RDMA adapters that can manage ~20ns latency already, it would be interesting to see what could be achieved before we start pushing up against the laws of physics. Even at the physical layer, my understanding is that most networking fibre optics manage about 60% the speed of light.

So, just to make a wild prediction: within the next year, Nvidia will acquire a HPC network adapter company (e.g. Mellanox).

Nvidia already has high speed GPU-GPU links:

http://www.nvidia.com/object/nvlink.html

They aren't widely deployed yet - the DGX-1 uses it, though.

I assume it will be infiniband that will be the commodity hard adopted to distribute Deep Learning. However, Nvidia are currently trying to segment the market by providing GPU->GPU direct access via infiniband on only the Tesla cards - not the commodity ones (titanx, 1080 gtx). That strategy will probably be the death of them in Deep Learning.

Or it will make buttloads of money until they have to move it downmarket when serious competitors arrive. Money they can use to lap competition by investing in better cudnn and hardware.

Full disclosure: I am (and have been since I found about about cudnn) long nvidia.

That's how I see it playing out too, assuming Nvidia pull it off. By my reckoning, it roughly goes:

HPC -> Mainstream servers (and niches like ML) -> 'Professional workstations' -> Consumer desktops

Intel's response (at least in part) seems fairly sensible: throw money and developers at open-source and open-standardisation efforts to try and rally the rest of the market (e.g around stuff like OpenCL). Otherwise network effects from stuff like widespread CUDA usage will become too strong to overcome, at which point Nvidia will have a monopoly.

However, I think they're missing an opportunity when it comes to high-speed networking. Specifically, I think they should be looking to release cheap, simple and open-standards based networking gear without trying to cross-sell Xeons or whatever at the same time. I realise that this means cannibalising revenue from their CPU market, but at this point someone is going to do it so it may as well be them. Hell, they could probably even get away with tying the functionality in to some fairly widespread Intel CPU instruction set (say, from Ivybridge onward). However, anything past that and it's a non-starter for consumer segments (which, in my mind, is also the developer segment).

10gbit RDMA is actually sort of affordable nowadays (at least if you pick up gear from eBay). But it's currently a complicated mess of competing and incompatible standards. Some SFP+ modules are even vendor-locked; if they sense there's a different SFP+ branded module on the other end they stop functioning due to an 'unsupported configuration'. These kinds of things make it impossible for a decent sized consumer market to form. If Intel could just accept that the world is moving on with or without them and get a proper consumer market going, at the very least they'd keep the market open (and at the very best the might even tilt the odds in their favour).

AMD, on the other hand, I have no idea wtf they are doing (and neither do they, by the looks of it). It could just be my biases, but I really think the best strategy for them is to basically bet the house on open-source and really up their engagement with Linux core dev. While I get it's attractive to target the gaming market because it's profitable in the present, it probably means death in the future. Actually their interests are pretty aligned with Intel's, and they have highly complementary work forces. It sounds crazy, but I wouldn't be too surprised if Intel were to acquire AMD at some point, or at least buy out their GPU division...

Why are you assuming Infiniband is the winner? There's Intel Omni-Path and there's Ethernet.

(Disclaimer: I'm the system architect of InfiniPath, which is one of the things that evolved into Omni-Path.)

Ah, just the man I've been wanting to talk to then :)

As I mentioned in another post, I really think Intel are missing an opportunity when it comes to high speed networking. Why don't they drop all the crazy server market lock-in vendor shenanigans (e.g. brand to brand SFP incompatibility, 'call us' pricing etc.) and just jump straight to the consumer market? I know this means cannibalising CPU revenue, but that's going to happen anyway.

Developers are consumers. In many cases the direction that their skills develop in is largely determined by the consumer hardware they're running at home. The same seems to be true for Ops folks, who all seems to be running vSphere homelabs atm. Based on what others have said here, it sounds like NVIDIA have chosen to cross-sell, and work their way down from HPC -> ... -> Consumer, presumably to extract maximum rents, despite not yet having strong enough network effects o lock out competition.

This seems like the perfect opportunity for Intel to cut the legs out from under that strategy, by aggressively pushing widespread consumer (or at least 'power-user'/developer) take-up of high speed, low-latency networking. And hasn't iWARP been endorsed as an IETF standard (or is on its way to be)? It seems like there's a window of opportunity here, albeit one that gets smaller and smaller as Nvidia slowly work their way down market.

Well, Omni-Path has a 48 port switch ASIC. So that's it for consumer accessibility.

But if you want to buy the PCI Express cards, Newegg has 'em. Intel has a great channel organization.

It's certainly impressive tech. But, and bear in mind this is just from a consumer perspective, here are the reasons I wouldn't buy it:

- ~$8000 USD for a 24 port switch is way beyond my price range. I doubt any consumer market could support this (http://www.newegg.com/Product/Product.aspx?Item=9SIA6ZP53G35...)

- ~$550 USD for a single port adapter is also way outside of my budget, especially since I'd need to buy at least half a dozen of these for my home lab (http://www.newegg.com/Product/Product.aspx?Item=N82E16833106...)

- It's very difficult to find solid information on the product. The Ark page doesn't tell me anything (http://ark.intel.com/products/92007/Intel-Omni-Path-Host-Fab...). The link to the whitepaper (which I'm assuming is actually a marketing brochure) 404s.

- So even if I could afford this as a consumer/dev, I probably wouldn't buy it due to the uncertainty around total cost of ownership. Will it accept non-Intel branded QSFP transceivers? Same question with optical cabling.

- Given the product brief mentions 'fabric performance [will] scale automatically with ongoing advances in Intel Xeon processors...', it leaves me wondering if this is somehow dependent on some specialised Xeon CPU instruction sets, making it useless for most home labs (which, at best, will be running older generation Xeons from second-hand servers, but more likely CPUs from Intel's consumer line-up).

On costs: If these are just reflective of the cost of production, then fair enough, I guess I'll just have to wait (for either Intel, or another manufacturer, to devise a cheaper production process). If it's not, then I think you're missing the opportunity outlined above, and you'll lose the HPC fight given Nvidia occupy the high ground here.

On compatibility: If this is just some idiot 'multi-channel cross sales' strategy that some genius with an MBA has cooked up, you're going to have a bad time. It falls apart the second someone releases a commodity adapter (which may already exist, for all I know).

Oh and to answer you question about Inifiniband, I think the previous poster might be referring to this: http://i.imgur.com/2ciM4qy.png

It's also why I figure Infiniband would be a likely acquisition if Nvidia wanted to head in that direction (i.e. GPU to GPU networking a.k.a NVRAM RDMA).

Does Nvidia already have NVLink for this purpose?

They do, but you can't run NVLink to the CPU unless you're using exotic IBM POWER8 systems.

x86 systems like the DGX-1 can only run NVLink between the GPUs, and still rely on PCI-E for the CPU connection.

Reply on news.ycombinator.com