Hacker News Reader

Comment by dragontamer | Hacker News Reader

Its an interesting design for sure.

4x Stacks of HBM2 means 1TBps memory bandwidth at 16GB. Only 60-compute units enabled (maybe 4-CUs are expected to break during manufacturing? Its a weird number for sure...).

Since it shares dies with the MI50, the Radeon VII will have 1/2 speed double-precision, making this the cheapest high-performance double-precision card in existance.

------

FP16 compute is supported at double speed, but there are no tensor cores. So FP16 matrix multiplication / tensor ops are still a major benefit to NVidia.

But the memory size and bandwidth is quite salivating. That's a lot of bandwidth, and a number of problems are known to be memory-bound. Deep learning enthusiasts probably will stick to NVidia cards, but other compute problems may want to start playing around with this thing.

------

Video gamers seem meh about the specs. But I think anyone looking at this card for its compute performance would be impressed.

View on HN

> Deep learning enthusiasts probably will stick to NVIDIA cards

We've been building the wrong hardware for ML for a while now.

A TPU doesn't delivery supremacy for problems over GPU hardware. It's a Google senior engineering retention and PR project.

Exceedingly few problems resemble image recognition. You wouldn't be able to tell from the research, because due to the tools it's sort of the only affordable thing to do.

Playing Go barely looks like it. Or more accurately, Go doesn't look like most other games. Molecular physics and synthesis barely looks like it. Neural programs/neural Turing machines that matter (converting complex sim code like game rules and physics into neural networks via learning) may never achieve the accuracy or performance/watt necessary to really compete with just making a bunch of CPUs or straight up dedicated hardware.

Word embeddings are a disaster. People keep trying to do innovative stuff with it, and it's 2018 and we've just just recently got entity names from Wikipedia. I think Peter Norvig, having basically only machine translation to point to, is really going to eat crow when non-perceptual AI tasks sort of have nothing to do with stuff Google has developed. But what do I know.

I'm sure someone's going to trot out some obscure TPU or GPU-driven thing. It's been years! Awesome innovations in hardware really do lead to obvious, immediate crazy cool stuff. I'm really talking about the Kinect as an example here, in that it was really hyped and not much came of it. The TPU, and ML-specified hardware targeted to today's problems, is a slower-motion Kinect of our time.

Mining cryptocurrency, the most recent innovation, is the opposite of cool.

I'm confident these tools exist to capitalize on the subsidy from the gaming industry. AMD is wise to not chase ML features.

What is the right hardware for ML?

Probably Larabee. Too early and too expensive for its time.

The Japanese PEZY machine looks incredibly interresting. Too bad it is only used in that one super-computer.

Its overall design sits somewhere between a GPU and CPU. Like a GPU, it supports 8-way hyperthreading (GPUs typically support 10. CPUs usually only support 2... maybe 4 for Larabee). Like a GPU, its programmed through an OpenCL-like language (I know you don't like that, but its how it works).

However, the overall layout is very CPU-like as well: a "City" is composed of "Villages", which is composed of multiple "Processing Elements". Each PE is scalar like a normal CPU, although 8-way SMT. So its not like a GPU's SIMD or SIMT architecture in this regards.

--------

See information here:

https://en.wikichip.org/wiki/pezy/pezy-scx/pezy-sc

It really is an interesting design that toes the line between CPU and GPU.

Is PEZY still a going concern after the arrests?

So that's why Xeon Phi has been so successful!

Real talk, though. Why do you think larabee is the "right" hardware?

I'd assume because of the extra serial processing power of a CPU.

x86 isn't the right choice though. As the chips get smaller, all the x86 cruft starts to become a larger portion of each core.

RISCV with 40-ish instructions plus a SIMD extension would probably be ideal. They've already proven that those designs use less space per core than ARM.

The truth is that for many types/sizes of models AVS2 or AVS512 in Xeons is as fast and GPUs.

I am interested in seeing benchmarks regarding this. I believe this is true for models with truly huge embeddings, but otherwise if your models are even a little compute dense then GPUs are faster.

My experience is that you need the additional caveat of a streaming fairly homogeneous and highly parallelizable approach or the memory transfer, branching and communication overheads will eat away nearly all the GPU gains.

I've also noticed that a lot of the time, people are comparing GPU to naive implementations (and sometimes, implementations written in dynamic languages) instead of to highly tuned BLAS or MKL implementations. For a large swathe of problem types, using the fastest math library will reduce the CPU/GPU gap to less than an order of magnitude or less.

> comparing GPU to naive implementations (and sometimes, implementations written in dynamic languages) instead of to highly tuned BLAS or MKL implementations

In your experience, are these comparisons using naive GPU implementations as well? If so, that still seems like a valuable comparison.

No, they're not. Straight front the horse's mouth:

https://software.intel.com/en-us/mkl/features/benchmarks

In no case will you see it get close to 10TFLOPS. GPUs easily do this, and can approach 100TFLOPS with tensor cores.

I don't dispute this. If you can remove memory transfer overhead and meet the requirements I mentioned, then GPUs will be much better. But there are many problems that do not fit that (SIMD per warp) regime. In short, GPUs are no panacea and have their own trade-offs and bottleneck sensitivities, just like everything else.

I agree that they are no panacea, but we also have an example of the Intel Xeon phi, which did have high speed hbm memory. That too did not compete with GPUs. I think the majority of it has to do with the GPU chip itself being extremely simple, with no Branch prediction, no branch difference, very simple caching behavior, etc. They are able to allocate a much larger portion of the die to just throw fma units at it.

Well, I'm speaking from personal experience, for many (but not all) text-classification IO-intensive models with large embedding tables Xeon on TF compiled with AVX2 or AVX514 and FMA is as fast as GPU. Obviously these models need to be on the lower spectrum of computational complexity.

This is not true. I've benchmarked it, and you can see yourself by looking at the MKL benchmarks versus cuBLAS. cuBLAS is always significantly faster.

That will certainly depend on the kind of ML you want to run. What is the right hardware for computing?

Seems like what you need for inference is usually very different from what you need for training.

They absolutely need tensor/matrix multiply cores. Incremental fp32 improvements aren't impressive.

> talking about the Kinect as an example here, in that it was really hyped and not much came of it

The underlying technology was acquired by Apple and is used on iPhones for FaceID, https://www.theverge.com/circuitbreaker/2017/9/17/16315510/i...

> PrimeSense was never solely responsible for the technology in Microsoft’s Kinect — as evidenced by the huge improvements Microsoft made to Kinect 2.0 on its own — and it’s also obvious that Apple is doing plenty of new software and processing work on top of this hardware. But the basic idea of the Kinect is unchanged. And now it’s in a tiny notch on the front of a $999 iPhone.

Matrix multiplication is the bread-and-butter of ML (both training and inference), for fairly obvious and solid reasons. Luckily for us, it's also very important in all sorts of domains, including graphics, rendering and gaming. That's all there is to it. There's no real "subsidy from the gaming industry", only a cross-subsidy situation in which common problems that are shared across domains can take advantage of the increased scale that comes from sharing the GPU-compute platform.

I'm not sure what you mean by "ML features", but AMD/ATI's dismissive attitude towards the ML and GPU-compute communities is looking quite silly by now.

I'm not sure what you mean by "ML features", but AMD/ATI's dismissive attitude towards the ML and GPU-compute communities is looking quite silly by now.

I disagree, ML computing has tons of competition and its margins will be quickly diminish. I think they are right to focus on gaming.

> Exceedingly few problems resemble image recognition.

It makes it extremely difficult to take any of what you just said seriously knowing that you made this statement.

I don't know dude, if there weren't such lucrative jobs for academics at places like Google and pharmaceutical research companies, the discourse wouldn't be so narrowly focused on doing blah blah neural networks on huge libraries of images and molecular hashes.

People with academic backgrounds would be less afraid to point out how much vaporware machine learning has produced. They'd take stock in that their tenure, or six year PhDs, give them an opportunity to not chase trends so aggressively. They'd shill online less, for some giant faceless corporation like Google.

I'm not going to shit on a bunch of great stuff people have made. But the scary thing is that Noam Chomsky is probably right, that AI science hasn't advanced much at all. That we're still in an AI winter where it matters, the science, despite all the money and attention.

> the discourse wouldn't be so narrowly focused on doing blah blah neural networks on huge libraries of images and molecular hashes.

Do you mind clarifying what exactly your issues are with these specific applications of machine learning? Is it that you think there isn't really any meaningful value that can be gained in working within these areas? Or is it just that you want people to spend more time researching a different topic that you think would be more interesting/compelling?

Care to expand? Prima facie this is a very reasonable stance. Further, image data has a lot of structure not present in other data and it's far from obvious what to do with ML techniques in other contexts.

This is coming from a physicist starting to investigate ML tools in my own work.

Some of the greatest advancements in research and application of machine learning have been within image recognition in the past few years. The inflection point and subsequent boom of deep learning circa 2012 is basically marked by the ImageNet competition of that year.

I think we are at a point where the cool hardware will actually be low power inference (deployed on basically the class of models we have now).

Perfect dies (64 CUs) get sold as a Radeon Instinct MI60 at a much higher price.

Good point, and that makes a lot of sense.

Do you have any estimate what that higher price will be?

I'm pretty sure MI60 is north of $5000. Maybe $8000 or so.

There's no guarantee this won't have gimped FP64, Hawaii for example had 1/2 rate FP64 but the consumer cards were 1/8 speed. But even 1/8 would leave the 1/32 NVIDIA cards in the dust.

If they don't gimp FP64 it won't leave a whole lot of reason to buy the MI50 for probably 10x the price.

> If they don't gimp FP64 it won't leave a whole lot of reason to buy the MI50 for probably 10x the price.

MI50 has ECC RAM support. That's absolutely necessary for high-end scientific compute.

Not at a 10x price difference it isn't, at least if you're at all price conscious and looking for workstations / small clusters. This makes it far cheaper to just re-run simulations as necessary to get the same statistical guarantees with regards to memory corruption errors.

What AMD needs right now is marketshare so more stuff gets written in something cross-platform instead of cuda. I'd imagine that 1/2 has at least been discussed at great length for this card.

It's looking like that "something cross-platform" will be cuda, given that AMD's latest approach is to implement a compiler to convert cuda to "hip."

> There's no guarantee this won't have gimped FP64 [..]

It's not gimped according to Ryan Smith from AnandTech: https://twitter.com/RyanSmithAT/status/1083959608371175424

Which means it destroys it's only real competitor for consumer FP64 - the Titan V - which costs 3000 Dollar.

If it could somehow actually sell for the MSRP (which of course it won't), it is a better gaming card than the RTX 2080. The 2080 costs more, and it's flagship feature is something you'll almost certainly want to turn off for gaming since it kills your frame rate and doesn't provide anything most people will notice while playing a game. Screen space reflections and cubemaps are convincing enough when you're zipping around a game world.

$699 matches the MSRP for the RTX 2080: https://pcpartpicker.com/products/video-card/#c=427&sort=pri...

But you aren't usually going to be finding them that cheap. That will probably be the case with the VII too, but we don't know yet.

Well, that's why I linked to PCPartPicker, to show that RTX 2080 models at MSRP do exist. They may have been scarce at launch, but they can be purchased today less than 4 months later.

Since AMD will sell the Radeon VII directly at first, I think early adopters will probably face limited quantities rather than price markups (not counting unauthorized resellers). After that, we'll have to see how the situation with retailers and later on non-reference models shakes out.

Would it make sense for them to make a 8GB variant as well? 16GB make it very expensive and actually for gaming it's most likely not needed yet.

It looks like one of the card’s headline features, 1TB/sec of memory bandwidth, was only achievable with a 16gb design (basically a doubling of channels/memory over the previous Vega 64 design, with which this card shares many similarities).

Why can't they use 4x2GB and provide the same bandwidth? Or it's some limitation of HBM2?

yes, apparently. though thry could probably get it done eventually, probably not worth it.

https://news.ycombinator.com/item?id=18868617

HBM2 has minimum 4GB in each stack IIRC

Reply on news.ycombinator.com