Comment by wolfgke | Hacker News Reader

Comment by wolfgke | Hacker News Reader

Parent Full thread

> I thought intel, after a decade of market domination, was sitting on piles of options and brains. But AMD made them drown (on the PR side) way faster than I expected. The only thing they had was Premium 10cores models. No innovative design or idea.

Intel has AVX-512, which is in my opinion one of the greatest innvoation in the x96 instruction set for a long time. The problem is that Intel wants to introduce AVX-512 only very slowly into consumer CPUs (slowly beginning with Cannon Lake, which will only be available in small quantities for some time if rumors are to be believed). Even more: For some parts of AVX-512 (e.g. 4FMAPS and 4FMAPS which are very useful for deep learning), Intel seems to be willing only for special expensive accelerator cards (Knights Mill) to include them to segment markets even further.

Similarly it is often complained that Intel offers no ECC support for the Core i... series of CPU (IMHO this complaint is rightful). Again: Intel does offer it, but only in the much more expensive server/workstation CPUs (Xeon).

So in my opinion Intel is sitting on a lot of options and brains - but they seem only willing to sell this in the really expensive CPUs for much higher prices than lots of market segments are willing to pay.

I think they are waiting for 10nm in order to reduce massive power consumption/downclocking needed when one runs AVX-512. So in a way their manufacturing process is holding them back. AMD completely avoided these headaches by castrating AVX in Ryzen, allowing much lower TDP (65W).

I am also curious why would 4FMPAS be useful for Deep Learning when all the major performance improvements lately come from half 16-bit/quarter 8-bit-precision floating/fixed point math (you don't need precise boundaries).

> I am also curious why would 4FMPAS be useful for Deep Learning when all the major performance improvements lately come from half 16-bit/quarter 8-bit-precision floating/fixed point math (you don't need precise boundaries).

This is a claim by Intel. See for example slide 140 of http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture15...

Intel's TPU2 and 3, as well as NVidia's V100 series get great tensor performance on FP16 and FP8. There are many implementation and algorithmic risks associated with using such small FP sizes, especially if you are working on problems related to convergence, and you have significant roundoff error.

Smaller FP sizes mean you can run more of these in parallel. Smaller FP sizes mean (much) larger roundoff error, fewer bits of precision kept.

Yup, does the lower precision in Deep Learning matter though? You are basically constructing some hyper-dimensional soft boundaries separating classes of objects, and even between training runs on FP64/128/256/512 one can end up with very different boundaries - whether these come from some random training/sample order or lowered precision probably doesn't play a large role, if it speeds up training 10x? If you really want more precise boundaries, you can train on 8/16-bit first and then use transfer learning on pre-trained low-precision model with higher precision as your last step anyway.

The TPU is a google chip, not an Intel chip.

>Intel has AVX-512, which is in my opinion one of the greatest innvoation in the x96 instruction set for a long time

how is AVX-512 innovative when it's essentially SSE with 4x bigger registers? am i missing something?

> how is AVX-512 innovative when it's essentially SSE with 4x bigger registers? am i missing something?

It is much more: Just to give examples:

- The introduction of the opmask registers to mask many AVX-512 instructions

- AVX-512 Conflict Detection Instructions (CD) enables lots of loops to vectorize that could not vectorized before.

Also the fact that now 32 SIMD registered can be addressed does not seem very innovative from the outside, but implies a deep change: Before only 8 (32 bit mode; bite 5-3 of MOD-REG-R/M field) or 16 (64 bit mode; additionally use REX.R field) SIMD registers could be addresses for deep instructional encoding reasons. This also holds when using a VEX prefix. So being able to use 32 SIMD registers requires a completely new prefix scheme (EVEX). This new scheme contains lots of new capabilities (source: https://en.wikipedia.org/w/index.php?title=AVX-512&oldid=841...):

- Expanded register encoding allowing 32 512-bit registers.

- Support up to 4 operands.

- Adds 7 new opmask registers for masking most AVX-512 instructions.

- Adds a new scalar memory mode that automatically performs a broadcast.

- Adds room for explicit rounding control in each instruction.

- Adds a new compressed displacement memory addressing mode.

A prefix that lets you properly pick more registers isn't much in the way of innovation either, no matter how important it is.

I'm really not convinced by this list.

I agree those features sound great.

But it's difficult to practically use AVX-512 when the turbo boost throttling prohibits breaking even with an equivalent AVX/AVX2 workload. I had a small project that would trivially scale up to AVX-512 register sizes. Despite doubling the vector width, it was actually much slower than the AVX/AVX2 version in practice -- just because it had such a low turbo boost ratio when running AVX-512 instructions.

> But it's difficult to practically use AVX-512 when the turbo boost throttling prohibits breaking even with an equivalent AVX/AVX2 workload.

This was also a problem with the first AVX implementations, so this is very typical: In the first generation, Intel makes such an instruction set extension available, so that one can write applications that make use of it (though they will usually not be faster, sometimes even slower). In the following generation this new feature is made fast, so that the newly written algorithms really get to profit from it.

AVX-512 itself isn't really an innovation. It's just making vector units bigger again. At best you struggle to fit two AVX-512 units. The equivalent of four AVX2 units isn't hard to keep fed with instructions, and would actually win in some scenarios due to being more flexible.

Intel having significantly more vector capacity than AMD in each core is just a design decision, not an innovation.

Adding a single-precision FMA instruction is also nice but not particularly 'innovative'.

And it's not like ECC is innovative either. Intel is sitting on options, but it's not sitting on ideas.

> AVX-512 itself isn't really an innovation. It's just making vector units bigger again.

See https://news.ycombinator.com/item?id=17264520

> AVX-512 itself isn't really an innovation. It's just making vector units bigger again.

No, AVX-512 is not only about doubling everything and calling it a day. I think the main innovation in AVX-512 is making it an easier vectorization target for compilers.

https://software.intel.com/en-us/articles/the-intel-advanced...

Name me a usage scenario, that AVX-512 is applicable to speed up in 80% of the market. I cant even think of one.

Intel's 10nm HVM isn't coming until 2019. That has been confirmed in an investor meeting when they were pushed to finally give an answer. So Intel has options? yes, lower Pricing, new SKU to muddle things up, more tricks like these Overclocking to damage AMD buyers expectation. Show unrealistic Cinebench that now totally flavours them. More Intel Inside tactics, changes of LLVM / GCC general code to force slower compile on AMD CPU, bundle their SSD with their CPU and force any vendor to get off AMD.

Oh yes, lots of options. None of them are technically superior though.

And btw, lots of the best Intel engineers are no longer with them. Apple, Qualcomm, Samsung, Nvidia, Telsa are all hiring their talents out of Intel Campus. If TSMC had Fabs in US I would bet many Intel Fabs engineer would have joined TSMC as well.

> Name me a usage scenario, that AVX-512 is applicable to speed up in 80% of the market. I cant even think of one.

The problem with this kind of statement is that many people by their PC/laptop just for surfing and office applications. Especially for the latter application even a 7 year old PC currently suffices. But it is in my opinion easy to come up with ideas how applications that really need this kind of speed will profit from AVX-512. Additionally keep in mind that people who run applications that benefit from more speed will much more be prone to buy new PCs than people who just do office and surfing.

Reply on news.ycombinator.com