Summary:
- 28nm
- looks similar to P100 (interposer with HBM)
- 55 teraops/s performance
- custom number format (variable length fixed point?)
- simplified memory architecture (no cache?)
- no info about power consumption
No automatic cache hierarchy. There is local memory, which you manage explicitly. More info here:
https://www.nervanasys.com/nervana-engine-delivers-deep-lear...
Ha, ha. Yeah… no. For one thing, there's no mention of its power consumption, no CUDA support, questionable memory design (sure, you can get a million TFlops without cache, now try to get that chip to do anything useful), etc.
Intel probably bought 'em to work on integrated GPUs or Xeon Phi or something.
Ha, ha. Yeah… no.
> there's no mention of its power consumption
Can't be tremendously higher than Pascal for reasons of physics.
> sure, you can get a million TFlops without cache, now try to get that chip to do anything useful
"Without cache" is certainly an exaggeration. It won't have a globally coherent cache hierarchy in the style of CPUs. It certainly will have various on chip memories to hold intermediate results. Neural net workloads are incredibly predictable and homogeneous and are essentially the perfect scenario for hand optimization of data flows to beat automatic caching.
> no CUDA support
You're just being silly now. CUDA isn't a standard, it's proprietary to NVIDIA and this isn't a general purpose processor anyway.
I would love to agree with you but the reality is that CUDA is the de-facto industry standard e.g. [1].
> and this isn't a general purpose processor anyway.
The "GP" part in GPGPU would like to disagree... [2]
[1] https://en.wikipedia.org/wiki/Comparison_of_deep_learning_so...
Case in point, GPUs are standard (for now) for training DNNs, but the big server farms that actually do the scoring run on Xeons.
However, Intel dominance is not to be underestimated, they definitely can make industry wide impact quickly. Just saying you can not easily draw parallels here