Hacker News Reader

Comment by drewg123 | Hacker News Reader

My biggest concern in moving to AMD is support for profiling and system visibility. On Intel, there are many more performance counters exposed, and Intel themselves make great tools (Vtune, Intel PCM).

AMD has a profiler, but it is nowhere near as capable as Vtune.

AMD has ways to do some things that Intel's PCM tools do (like monitor memory bandwidth), but they are exceedingly awkward (eg, only option is producing a CSV, no live output). And there are lots of things which are just missing (like support for monitoring NUMA fabric bandwidth, PCIe bandwidth, power, etc). I gave a talk at AMD in Austin last fall where I beat them up about some of these things.

Oh, and I'm up to over 260Gb/s of 100% TLS traffic from an AMD Rome :)

View on HN

I think I see a comment to this effect on every post about AMD, that their software ecosystem isn't as advanced.

But, isn't it a pretty narrow market that benefits from Intel's tooling here?

Startups running their entire business on JITed runtimes on top of multiple layers of virtualization in cloud providers aren't using this. Enterprises with enough resources to revise their entire stack to work on ARM (Cloudflare, Microsoft, Google, Amazon) likely have the internal resources and skills already to build the missing tooling and profile statistically using massive fleets of servers.

How large is the target market of Intel purchasers that are large enough and performance sensitive enough that it's profitable to micro-optimize assembly, and yet not large enough that it's a rounding error to build similar tooling for AMD x86-64 or ARM or otherwise?

Its not a question of just building tooling. It is a question of AMD having fewer performance counters. So even organizations like Google with tons of SWEs can't do anything if there are no performance counters, or if the performance counters are not exposed.

And we (Netflix Open Connect) kind of fit that middle ground that you're talking about.

> Its not a question of just building tooling. It is a question of AMD having fewer performance counters. So even organizations like Google with tons of SWEs can't do anything if there are no performance counters, or if the performance counters are not exposed.

Have you tried JTAG?

Any examples of missing counters that are essential? I am curious because I rarely see people care about things further than TLB, L1, L2 misses (which exist on AMD).

If you have a large enough problem space that you have the capacity to develop a 'bench' of people to tackle deep analysis, you probably should do so, rather than relying entirely solely on tools to tell you what's going on. It's not enough to know how to do something, you have to have done it routinely enough that when it really matters you don't treat it like a rehearsal.

> How large is the target market of Intel purchasers that are large enough and performance sensitive enough that it's profitable to micro-optimize assembly

I'd like to think if a large customer picked up the phone and called Intel and asked why the new processors they got are slower than the old ones, that asking that they send someone to take a look at it would not be outside of the realm of reasonable requests. But not being in a position to buy millions of dollars worth of gear from Intel so I'll have to wait for someone to tell me what happened.

If they bother to send someone out, that person would probably like to have - or even insist on - some decent tools. Reliable and informative, if not necessarily friendly enough for consumers.

I wonder to what extent Intel's tools prevent such customer visits, versus being made to give small and medium sized customer a feeling that they have some secret weapon using Intel, rather than the reality of one. A pretty collar for the pig to wear to the Fair.

Same applies to GPU debugging tools, and that isn't a narrow market, with HPC, FinTech, game engines, machine learning.

Compared to both Intel on the CPU side and Nvidia on the graphics side, AMD has always been lacking in the software department. Hopefully they can fix that now that they're making some money.

AMD is a classical hardware manufacturer in that they don't respect software as an equal partner in modern hardware development, despite the fact that their core competency is making general purpose computers. It's not a capital issue, it's a cultural one.

That said, AMD (and Intel) have reinvented themselves several times with pretty big cultural shifts so I'm hopeful they get someone like Jim Keller with the authority to shake things up.

>so I'm hopeful they get someone like Jim Keller

Who do you think designed the Zen architecture? He as since left AMD, went to Tesla, and is now at Intel.

Why do you think I brought him up? AMD needs someone like Jim Keller but for software.

Who the hell is the Jim Keller of software? I don't think Dave Cutler is the right comparison, although they're both pretty fucking awesome.

Jeff Dean?

> Why do you think I brought him up? AMD needs someone like Jim Keller but for software.

If there will be a JK for software, I'd say if he lands in a typical dotcom, half of the dev team with careers typical to Facebook/Amazon/Google will be relieved of their duties in a week.

They'd never hire him. Too old.

Jim Keller didn't design Zen.

> I'm hopeful they get someone like Jim Keller with the authority to shake things up.

I's say it is even less likely if Jim will be on the board

Despite this, I find it monumentally easier to get drivers and software of AMD's website, compared to Intel.

And then there's also "Intel is Removing End of Life Drivers and BIOS Downloads"[0]. Great, if I didn't save them, I need to go to a 3rd party site, run the risk of getting malware, have to compare hashes, etc, etc, etc... This is, in my opinion, an absolute no-go as a vendor, and instantly makes me want to drop them. Which I did, all the PCs my clients get from me are AMD systems now.

AMD is a serious vendor, you can even get the old ATI RAGE drivers off their website.

[0]: https://www.bleepingcomputer.com/news/hardware/intel-is-remo...

EDIT: The linked tweet[1] in the article mirrors my sentiments on this topic almost perfectly:

> "DEAR COMPANIES THAT MAKE HARDWARE: YOU'RE SUPPOSED TO HOST YOUR DRIVERS UNTIL THE SUN EXPLODES OR YOU GO OUT OF BUSINESS, NOT UNTIL YOU GET TIRED OF HOSTING 20 MEGABYTES"

[1]: https://twitter.com/Foone/status/1172237142485078016

20mb? My latest nvidia 'driver' download was over 500mb and prompted me to log-in to something before installing.

What? I expect to get drivers even after the sun explodes

YES!

Are you talking about a Linux version of Vtune or are you running your datacenters on Windows?

I know Vtune for Linux exists but I've never heard much a bout it, everyone seems to be recommending "perf". Is there more to measuring memory bandwidth than multiplying last level cache misses with cache line size?

I think where Intel does best is their open source drivers, tools and contributions, though AMD has improved a lot in that respect too.

>Oh, and I'm up to over 260Gb/s of 100% TLS traffic from an AMD Rome :)

Hold on, is that 30% improvement from 3-6 months ago? I thought it was 200Gb/s last time and it was sort of pushed to the limit with memory bandwidth bottleneck? Do you think you can push it even further something like 300Gb/s?

Seriously this is mind boggling.

This is a motherboard designed for Rome, so it has 3200 memory, rather than 2933 (which was OC'ed). Plus a few improvements here and there in FreeBSD, minus the fact that we're using 2 Gen4 x16 NICs, so there is some cross-domain IO going on.

I believe AMD has been contributing upstream to intel-cmt-cat, and Intel has accepted those patches.

How is that possible? Raw Bw of PCIE is around 200Gb/s.

Pcie4 (of which rome is) can do about 220Gbps realistically per card. Multiple cards, which I'm sure Netflix does, can go higher.

True but usually it’s difficult to actually achieve that speed on the bus. Isn’t the NIC a bottleneck? Usually they aren’t capable of wirespeed due to shallow buffers.

It's doable. We do 100Gbps on pcie3, and AMD presented at the dpdk conference last year showing 200gbps: https://dpdkna2019.sched.com/event/WYAs/dpdk-and-pcie-gen4-b...

I'm curious, why encrypt Netflix video traffic at all?

(Not at Netflix) So that ISPs don't interfere with it by blocking, inserting ads, etc.

can you expand on the TLS bit?

Transport Layer Security, which used to be known as SSL (eg, https). It uses up nearly half of our CPU time and our memory bandwidth to read data from memory into the CPU, encrypt it, and write the encrypted data back to memory.

If we were serving plain http without TLS, we could probably reach 400Gb/s from this box (based purely on back of the envelope math)

You can't use any kind of TLS offloading accelerator?

Are you asking what’s most cost-effective?

I'm asking what best meets the requirements, which may be a balance of cost and performance.

Looks like it's in progress: https://2019.eurobsdcon.org/slides/Kernel%20TLS%20and%20TLS%...

There are 2 classes of accelerators -- lookaside and inline.

The lookaside accelerators (like QAT) sit on the PCIe bus and DMA plaintext into the accelerator, and DMA encrypted data out. This offloads the CPU, but it does not help with memory bandwidth. Since memory bandwidth is our biggest bottleneck, they don't help.

Inline accelerators are "smart nics". The NIC DMAs a pre-formatted TLS record in plain text inside a (series of) TCP packet(s) from the host, encrypts it, and sends it in an almost stateless way. This is what we call "NIC TLS" in FreeSD. Sadly, the inline accelerators we've tried all have fatal flaws which prevent us from using them. I cannot get into those flaws for NDA reasons, but we're hopeful that there will be one that we can use soon.

Good to know, thanks. I'm curious what the fatal flaws are when you can eventually reveal them.

The first kind can't be made to work with P2P-DMA?

I did not mean what TLS is, I meant the context of the high performance numbers. Using what software, etc.

Running on FreeBSD, I did a quick search, turns out I submitted it on HN [1] exactly 5 months ago. And it was only running 200Gbps then

[1] https://news.ycombinator.com/item?id=21043765

Reply on news.ycombinator.com