Hacker News Reader

Comment by nerdbaggy | Hacker News Reader

The Graviton are also real cores in the VM, not like the Intel/AMD that have hyper threading.

There is not some force of nature that makes 1 thread per core "real". On pointer-chasing workloads a hyperthread is as real as anything. HT is basically another way to exploit instruction-level parallelism that your compiler left on the table. You gotta pick what suits your program.

Or to put it another way:

A hyperthread logical processor is a way of obtaining better efficiency by using surplus resources left on the table by the "real" logical processor that otherwise would remain idle (read: waste).

Considering hyperthreading as a way of obtaining an extra "real core" is a gross misunderstanding of what hyperthreading is meant to achieve.

> Considering hyperthreading as a way of obtaining an extra "real core" is a gross misunderstanding of what hyperthreading is meant to achieve.

It's as real as a Bulldozer CMT thread, and that was widely considered a "real core".

An integer ALU being pinned to a particular thread doesn't make it "real" especially when it comes at the expense of shared frontend resources like a decoder that has to alternate between servicing each thread on alternate cycles (which, as Agner Fog's Microarchitecture notes, massively bottlenecks both threads). And obviously if one thread has ILP that can be exploited, and the other "core" isn't using its ALU, it would be better to share that! And when you allow that, what you get is SMT.

At the end of the day that's all CMT is - SMT with inefficiently allocated (pinned) resources, and it had even less dedicated resources on the frontend as well. And people will absolutely die on the hill that bulldozer was a "real core". There are probably some scheduling advantages to doing CMT instead of SMT, but also performance costs as well.

So what is a "real core" anyway in this context? Is physical (unsharable, unchangeable) division of resources "inherently better" (or even inherently different) from logical/software-defined division of resources?

Then you've got this whole thing from IBM recently... and leaving aside the fact it's cache, the question IBM is fundamentally asking us is, why not just allocate more resources to "cores" that need them? Why not execution units as well, why wouldn't that be better? Why is hardware defined core better than software defined core? https://www.anandtech.com/show/16924/did-ibm-just-preview-th...

And when you look to how you would implement that for non-cache resources, isn't the simplistic answer something very similar to SMT? Not sure POWER9 is all that far off base with runtime-configurerd SMT4/SMT8 and a big fat core, maybe that's how Intel can make better use of some of the gigacores they've built. Sure it can run one thread really fast but why not 8 threads on the same resources? Or you can go the other way and have one thread issue onto multiple cores, as long as there's ILP to cover it and the performance impact of crossing cores is not large, does the difference really matter?

And yeah maybe that's still one core... but then so is a bulldozer CMT module too lol. The whole "what is a core exactly" is kind of trite, it doesn't really matter.

> Considering hyperthreading as a way of obtaining an extra "real core" is a gross misunderstanding of what hyperthreading is meant to achieve.

That’s cool but when a vcpu is a hyper thread (rather than a physical core with two hyper threads) that’s the reality of your cloud experience.

If you really need or want an entire physical CPU core, you pay extra for physical hardware instead of a virtualization.

Yes, it's expensive, but the performance of your server is a simple question of how much money you're willing to part with.

The point is that you can get that with graviton too, which is a much smaller transition.

If you're got a workload that isn't characterized by missing cache and waiting around for main memory fairly often then usually SMT is going to be pretty inefficient. As a crude rule of thumb, the single thread performance you get is going to scale as something like the square root of the number of transistors you throw at the problem. So if you want two threads of a certain performance you can either use one bit core with SMT-2 or two cores each a third or so the size. And the two cores will tend to be more power efficient two with less data moving long distances through the caches.

Now, if you are hitting main memory often it makes sense to go wild with thread and use SMT-8 like IBM's POWER cores or Sun's SPARC cores did.

And if you mostly just care about maxing out single threaded performance for user responsiveness then you do indeed have lots of unused resources most of the time and you might as well add SMT for when you're more throughput bound.

But while the design and transistor costs of adding SMT to a design are very modest, everything I've heard about the test and verification of it seems pretty hairy.

Especially because a hyperthreaded core can usually run 2-4 of any particular instruction in parallel, and 6-10 total instructions in parallel. Even if your workloads never stall, they can still get a reasonable core worth of resources each, just a smaller core.

Terminology nit: Hyper-Threading(tm) is the Intel-ese marketing term for their implementation. The computer architecture term is SMT (simultaneous multithreading).

SMT is synonym with multi-processing and applies to full cores as well as shared cores. AFAIK, there is no non-marketing term for Hyper-Threading, it's a very specific architecture.

You're probably thinking of SMP or CMP? SMT is specifically about multiple threads on a core.

Ouch, yes, I was thinking of SMP, not SMT.

TIL that there is a general name.

That's a pure AWS choice (you can use them only as real core) as is the fact that AWS has decided to provide large discounts for their graviton instance as a way to limit portability to other clouds.

All major cloud providers have arm64-based instances available now.

Graviton 3 is currently way ahead of the other offerings. The others are closer to Graviton 2.

No, its a generation behind the YT710 instances at alibaba, which are now in preview even in the US.

https://www.alibabacloud.com/product/ecs/g8m

"As the first instance family that uses ARM v9 architecture CPUs,"

and "Arm-based Alibaba Cloud T-Head Yitian 710 Crushes SPECrate2017_int_base"

https://www.servethehome.com/arm-based-alibaba-cloud-t-head-...

The gravaton3's are V1's (ARM v8.4), so yes AWS is ahead of the companies using the N1 cores from ampere. But, its a bit of a US centric view.

It doesn't appear to be in preview in the USA yet. It appears only to be available in the Hangzhou-I zone, and you can only obtain access to a single instance.

> AWS has decided to provide large discounts for their graviton instance as a way to limit portability to other clouds.

It this a negative aspect?

Absolutely yes.

So they're evil because they're offering things at low prices?

Are you familiar with the concept of predatory pricing?

Yes, and dumping.

However, you're making a bold assumption there. You don't know their cost structure. Their CPU time could literally be just that cheap for Graviton.

We know enough about Apple Silicon to know that high-performance ARM chips aren’t cheap to manufacture.

Until AWS breaks out their margins between X86 and ARM (not going to happen) or total AWS margins start to go down, we don’t really publicly know if they are “dumping” or if they are just passing on the savings to the consumer.

I’d bet that it’s a little of both

It could go a third way, where they use them internally, too, and they're cheaper to operate.

So a certain percentage of the costs could be amortized through internal usage.

Edit:

> We know enough about Apple Silicon to know that high-performance ARM chips aren’t cheap to manufacture.

Oh, and for this :-)

We do know that Apple's per unit costs must be cheap, since the iPhone SE 2022 retails at $429 for an entire phone, and it comes with A15, which is quite competitive with Android flagship chipsets from 2023. Since they can squeeze in an entire phone + profits in $429, the chipset itself can't be that expensive.

Yes, R&D is probably very expensive but they can spread that around to millions of units, just like Amazon :-)

Of course, Apple could also be dumping chipsets, in which case, I don't know ¯\_(ツ)_/¯

The cost of the CPU is small part of a server TCO. Graviton instances could be cheaper because the platform is cheaper, uses less power and needs less cooling - I think we know from Apple Silicon that ARM chips can have these advantages.

Disclaimer: I work for AWS but I don't have any internal knowledge about Graviton pricing and non-public performance data.

> We know enough about Apple Silicon to know that high-performance ARM chips aren’t cheap to manufacture.

We also know that Intel server CPUs aren't cheap to buy and include a significant profit margin for Intel.

More to the point lots about Graviton seems to be focused on keeping the total cost of ownership down - clock speed for example to limit power consumption.

Perhaps AWS is pricing aggressively to drive adoption. I'd be astonished if they are offering a service like this at scale at below their required return on capital.

> Apple Silicon → ARM not cheap

Well, maybe not cheap, but certainly cheaper.

"Apple’s move to M1 chips will save $2.5B this year, estimates IBM exec"

https://9to5mac.com/2020/11/18/apples-move-to-m1/

Apple even passed some of those cost savings on to consumers, most of the comparable M1 models were ~ $100 cheaper than the much slower Intel models they replaced. And Apple being Apple, if they passed on that much, their total cost savings almost certainly were significantly higher.

My point was more about `limit portability to other clouds` than the pricing issue, even if it is known that Amazon used is economic power in the past to push competitors out of the market.

How does that limit portability?

If you have been able to convert your deployment from x64 to aarch64, you can do the same in the other direction, or choose another cloud provider that provides aarch64 stuff. It is really easy to build container images in multiple architectures nowadays.

Reply on news.ycombinator.com