Hacker News Reader

Comment by zdw | Hacker News Reader

I see this going a few ways for different players:

The perpetual architecture license folks that make their own cores like Apple, Samsung, Qualcomm, and Fujitsu (I think they needed this for the A64FX, right?) will be fine, and may just fork off on the ARMv8.3 spec, adding a few instructions here or there. Apple especially will be fine as they can get code into LLVM for whatever "Apple Silicon" evolves into over time.

The smaller vendors that license core designs (like the A5x and A7x series, etc.) like Allwinner, Rockchip, and Broadcom are probably in a worse state - nVidia could cut them off from any new designs. I'd be scrambling for an alternative if I were any of these companies.

Long term, it really depends on how nVidia acts - they could release low end cores with no license fees to try to fend off RISC-V, but that hasn't been overly successful when tried earlier with the SPARC and Power architectures. Best case scenario, they keep all the perpetual architecture people happy and architecturally coherent, and release some interesting datacenter chips, leaving the low end (and low margin) to 3rd parties.

Hopefully they'll also try to mend fences with the open source community, or at least avoid repeating past offenses.

View on HN

> The perpetual architecture license folks that make their own cores like Apple, Samsung, Qualcomm, and Fujitsu (I think they needed this for the A64FX, right?) will be fine

There is one thing they would need to worry about though, which is that if the rest of the market moves to RISC-V or x64 or whatever else, it's not implausible that someone might at some point make a processor which is superior to the ones those companies make in-house. If it's the same architecture, you just buy them or license the design and put them in your devices. If it's not, you're stuck between suffering an architecture transition that your competitors have already put behind them or sticking with your uncompetitive in-house designs using the old architecture that nobody else wants anymore.

Their best move might be to forget about the architecture license and make the switch to something else with the rest of the market.

> Their best move might be to forget about the architecture license and make the switch to something else with the rest of the market.

This assumes that there isn't some other factor in transitioning architecture - this argument could boil down in the mid 2000's to "Why not go x86/amd64", but you couldn't buy a license to that easily (would need to be 3-way with Intel/AMD to further complicate things)

Apple has done quite well with their ARM license, outperforming the rest of the mobile form factor CPU market by a considerable margin. I don't doubt that they could transition - they've done it successfully 3 times already, even before the current ARM transition.

Apple under Cook has said they want to "to own and control the primary technologies behind the products we make". I doubt they'd turn away from that now to become dependent on an outside technology, especially given how deep their pockets are.

It's kind of puzzling that Apple didn't buy them. They don't seem to be particularly aggressive/creative in the M&A department

They have a perpetual license. What value is it for them to buy ARM?

Many of Apple's direct competitors are ARM customers, and US politics is already turning a large anti-trust eye towards FAANG. That might have been a factor.

Apple buying arm would never get approved. They have way to many direct competitors that heavily rely on SoCs that has arm cores.

Apple could transition in 10 years to RISC-V, just like how they transitioned 10 years ago to x86, 10 years before to PPC, 10 years before to .........

They did dump ZFS when they decided they didn't like the licensing terms.

> Apple has done quite well with their ARM license, outperforming the rest of the mobile form factor CPU market by a considerable margin.

But that has really nothing to do with the architecture. They just spent more money on CPU R&D than their ARM competitors. They could have done the same thing with RISC-V, and if that's where the rest of the industry is going, they could be better off going there too. Especially for Mac, since they're about to do a transition anyway. They could benefit from putting it off for another year while they change the target architecture to something the rest of the market might not be expected to avoid in the future.

There is also no guarantee that their success is permanent. They might have done a better job than Qualcomm this year, but what happens tomorrow, if Google throws their hat into the ring, or AMD makes a strong play for the mobile market, or Intel gets their heads out of their butts, or China decides they want the crown and gives a design team an unlimited budget? There is value in the ability to switch easily in the event that someone else is the king of the mountain for a while.

> I don't doubt that they could transition - they've done it successfully 3 times already, even before the current ARM transition.

Just because they can do it doesn't mean it's free.

Really if Intel was shrewd, they'd recognize that they've lost Apple's business already and just sell them an x86 license, under some terms that Intel would care about and Apple wouldn't, like they can only put the chips in their own devices. Then Apple could save themselves the transition entirely and do another refresh with processors from Intel while they put their CPU team to task redesigning their own chips to be x86_64. It would give both Apple and Intel a chance to throw a punch at Nvidia (which neither of them like) while helping both of them. Apple by avoiding the Mac transition cost and Intel by maintaining the market share of their processor architecture and earning them whatever money they get from Apple for the license.

And it gives Intel a chance to win Apple's business back. All they have to do is design a better CPU than Apple does in-house, and Apple could start shipping them again without having to switch architectures. Which is also of value to Apple because it allows them to do just that if Intel does produce a better CPU than they do at any point in the future.

The problem I see with your argument is that it ignores their superior mobile ARM SoC. If they are going to unify on an architecture, it would certainly be easier to migrate Mac to ARM than iOS to x86.

Your assumption is that the architecture matters and they couldn't build an x86 SoC which is just as good, but they could.

And you don't want to move to ARM if everybody else is moving away from it.

Samsung, Qualcomm, and MediaTek all currently just use off the shelf A5x & A7x cores in their SoCs. Unless that part of the company is losing money I don't expect nVidia to cut that off. Especially since that's likely a key part of why nVidia acquired ARM in the first place - I can't imagine they care about the Mali team(s) or IP.

What happens if/when Nvidia launches SoCs that compete with Qualcomm and MediaTek. Will it continue to offer the latest cores to competitors when it will make a lot more money on its own SoCs? This is the reason for the widepsread concern about Nvidia owning Arm.

I don't know if Nvidia is eager to re-enter the SoC market. It wouldn't be a clear path to more money, since they would need to then handle modem, wifi, ISP, display, etc... instead of just CPU & GPU. And they'd need to work with Android & its HALs. And all the dozens/hundreds of device OEMs.

They could, but that's more than just an easy money grab. Something Nvidia would already be familiar with from Tegra.

What seems more likely/risky would be nvidia starts charging a premium for a Mali replacement, or begins sandbagging Mali on the lower end of things. But Qualcomm already has Adreno to defend against that.

Looks like Nvidia never left the SoC market to begin with.

The latest Tegra SoC launched March of 2020.

They released a new SBC aimed at autonomous vehicles. They haven't had a mobile SoC since 2015's Tegra X1. Which only made it into the Pixel C tablet, Nintendo Shield, and Nvidia's Shield TV (including the 2019 refresh)

You're forgetting the TX2 as well as the Jetson Nano

Which ended up in what mobile products?

Mediatek are in the lower end market, something Nvidia's culture doesn't like competing in. Qualcomm holds the key to Modem. Which means Nvidia competing with Qualcomm wont work that well. Not to mention they have already tried that with Icera and generally speaking Mobile Phone SoC are low margin business. ( Comparatively Speaking )

Completely take your point on Qualcomm.

On mobile SoC margins I guess that margins are low because there is a lot of competition - start cutting off IP to the competition and margins will rise.

I suspect that their focus will be on the server / automotive to start off with but the very fact that they can to any of this is troubling for me.

Qualcomm used to design their own cores up until the last generation or two, but you're right they use the reference designs now.

EDIT: correction, make that the last generation or four (oops, time flies)

Qualcomm hasn't used an in-house core design since 2015 with the original Kryo. Everything Kryo 2xx and newer are based on Cortex.

That was a really sad time honestly. QCOMM went from leading the pack to basically using reference designs (which they still arrogantly brand as Kryo despite essentially being a tweak of a reference design.

It all happened because Apple came out with the first 64-bit design, and QCOMM wasn't ready. Rather than deliver 32-bit for 1 more year, they used an off the shelf ARM 64-bit design (A57) in their SoC called the Snapdragon 810, and boy was it terrible.

From what I gathered, they made at least _some_ risky architecture choices in their custom architecture that turned out not to be sustainable over the next generations. Also note that their Cortex core is indeed customized to a significant extent.

"and may just fork off on the ARMv8.3 spec, adding a few instructions here or there"

No, they may not. People keep suggesting these kinds of things, but part of the license agreement is that you can't modify the ISA. Only ARM can do that.

Well, regardless of whether this amendment is kosher or not, AMX definitely exists. Perhaps the $2T tech behemoth was able to work out a sweetheart deal with the $40B semiconductor company.

> There’s been a lot of confusion as to what this means, as until now it hadn’t been widely known that Arm architecture licensees were allowed to extend their ISA with custom instructions. We weren’t able to get any confirmation from either Apple or Arm on the matter, but one thing that is clear is that Apple isn’t publicly exposing these new instructions to developers, and they’re not included in Apple’s public compilers. We do know, however, that Apple internally does have compilers available for it, and libraries such as the Acclerate.framework seem to be able to take advantage of AMX. [0]

my123's instruction names leads to a very shallow rabbit hole on google, which turns up a similar list [1]

Agreed upon: ['amxclr', 'amxextrx', 'amxextry', 'amxfma16', 'amxfma32', 'amxfma64', 'amxfms16', 'amxfms32', 'amxfms64', 'amxgenlut', 'amxldx', 'amxldy', 'amxldz', 'amxldzi', 'amxmac16', 'amxmatfp', 'amxmatint', 'amxset', 'amxstx', 'amxsty', 'amxstz', 'amxstzi', 'amxvecfp', 'amxvecint']

my123 also has ['amxextrh', 'amxextrv'].

[0] https://www.anandtech.com/show/14892/the-apple-iphone-11-pro....

[1] https://www.realworldtech.com/forum/?threadid=187087&curpost...

That's untrue.

(famously so, Intel used to ship arm chips with WMMX and Apple for example ships their CPU today with the AMX AI acceleration extension)

WMMX was exposed via the ARM coprocessor mechanism (so it was permitted by the architecture). The coprocessor stuff was removed in ARMv8.

Now custom instructions are directly on the regular instruction space...

(+ there's the can of worms of target-specific MSRs being writable from user-space, Apple does this as part of APRR to flip the JIT region from RW- to R-X and vice-versa without going through a trip to the kernel. That also has the advantage that the state is modifiable per-thread)

In ARMv8 you have a much cleaner mechanism through system registers(MSR/MRS).

Apple has been using system registers for years already. AMX is interesting because it's actual instruction encodings that are unused by the spec.

That's like saying that my Intel CPU comes with an NVIDA Turing AI acceleration extension. The instructions the CPU can run on an Apple ARM-based CPU is all ARM ISA. That's in the license arrangement, if you fail to pass ARM's compliance tests (which include not adding your own instructions, or modifying the ones included) you can't use ARM's license.

Please, stop spreading nonsense. All of this is public knowledge.

No. I reverse-engineered it and AMX on the Apple A13 is an instruction set extension running on the main CPU core.

The Neural Engine is a completely separate hardware block, and you have good reasons to have such an extension available on the CPU directly, to reduce latency for short-running tasks.

Are the AMX instructions available in EL0?

Is it possible AMX is implemented with the implementation-defined system registers and aliases of SYS/SYSL in the encoding space reserved for implementation-defined system instructions? Do you have the encodings for the AMX instructions?

AMX instructions are available in EL0 yes, and are used by CoreML and Accelerate.framework.

A sample instruction: 20 12 20 00... which doesn't in any stretch parse as a valid arm64 instruction in the Arm specification.

Edit: Some other AMX combinations off-hand:

00 10 20 00

21 12 20 00

20 12 20 00

40 10 20 00

very interesting, thanks!

The AMX is an accelerator block... If you concluded otherwise, your reverse-engineering skills are not great...

Let me repeat this: part of the ARM architectural license says that you can't modify the ISA. You have to implement a whole subset (the manual says what's mandatory and what's optional), and only that. This is, as I've been saying, public knowledge. This is how it works. And there are very good reasons for this, like avoiding fragmentation and losing control of their own ISA.

And once again, stop spreading misinformation.

Hello,

Specifically about the Apple case,

After your tone, not certainly obligated to answer but will write one quickly...

Apple A13 adds AMX, a set of (mostly) AI acceleration instructions that are also useful for matrix math in general. The AMX configuration happens at the level of the AMX_CONFIG_EL1/EL12/EL2/EL21 registers, with AMX_STATE_T_EL1 and AMX_CONTEXT_EL1 being also present.

The list of instructions is at https://pastebin.ubuntu.com/p/xZmmVF7tS8/ (didn't bother to document it publicly at least at this point).

Hopefully that clears things up a bit,

And please don't ever do this again, thank you. (this also doesn't comply with the guidelines)

-- a member of the checkra1n team

You may be correct, but do you really have to be so attacking?

Can you provide a link to the "public knowledge" for those who don't know?

It seems to me that if Apple felt that Nvidia would limit them, they could have outbid them for ARM! So I think you are correct.

Apple would not get antitrust approval (iPhone maker controls all android chips????). So that’s why.

Were it a serious enough threat to their ARM license, they'd find a way to buy ARM and keep it independent.

Exactly, Apple is already straddling the line (and imho way past it) on anti-comp laws.

I'm not defending Apple, just thinking that can't we say this for many of the biggest tech firms? They are way past the line on anticompetitive business.

Yes, Apple is not alone in this. Google is another example, and they are very aware of this and are acting very carefully

What is their monopoly or which of their competitors are they colluding with?

Neither of those are required to violate anti-trust laws.

Such as? https://www.ftc.gov/enforcement/anticompetitive-practices

I think Apple is not committed to ARM at all. Bitcode, Rosetta 2, "Apple Silicon" - it all suggests they want to keep ISA flexibility.

Exactly. Apple’s strategy here is very clear:

Offer customers iOS apps and games on the next MacBook as a straight swap for Boot Camp and Parallels. Once they’ve moved everyone over to their own chips and brought back Rosetta and U/Bs they’re essentially free to replace whatever they like at the architecture level.

In their reveal I noticed that they only mentioned ARM binaries running in virtual environments. It makes sense if you don’t want to commit to supporting GNU tools natively on your devices (as it would mean sticking with an established ISA)

I would be quite surprised if LLVM ever lost the ability to compile C for the platform.

Exactly my point though.

Apple is large enough that if they want to break from ARM in the future they can do so by forking an LLVM backend. That's not a large job if it's a very small change, but once it's been done once they have plenty of resources to provide ongoing support (like they do for webkit).

The dividends of doing so are potentially massive. Given that they've been able to make really large gains with the A series chips to date on mobile (not least because they've been able to offload tasks to dedicated co-processors that general-purpose ARM cores don't ship with), the rewards for having chips that are a generation ahead of other like-for-like computers will outweigh the cost of maintaining the software.

Wow, but that cost - it's not a small thing to transition ISA, and don't forget that this transition is one of the simpler ones (more registers, fairly few devices). The risks of transitioning everything away from arm would be much greater.

I guess they have some ISA flexibility (which is remarkable). But not much; each transition was still a very special set of circumstances and a huge hurdle, I'm sure.

At the low-level driver interface, transitioning ISA is a big deal. But I would guess that, at higher levels, most of the work is just changing the target of your compiler?

As in, most of the work occurs in the low-level parts of the Operating system. After that the OS should abstract the differences away from User-space.

No way; not at all.

First of all: there's lots of software that's not the OS. The OS is the easy bit: everything else: grindy, grindy horrorstory. A lot of that code will be third-party. And if you think, "hey, we'll just recompile!", and you can actually get them to too - well, good luck, but performance will be abysmal in many cases. Lots and lots of libraries have hand-tuned code for specific architectures. Anything with vectorization - despite compilers being much better than they used to be - may see huge bog downs without hand tuning. That's not just speculation; you can look at software that's gets the vectorization treatment or was ported to arm from x86 poorly - perfomance falls off a cliff.

Then there's the JITs and interpreters, of which there are quite a few, and they're often hyper-tuned to the ISA's they run on. Also, they can't afford to run something like LLVM on every bit of output; that's way too slow. So even non-vectorized code suffers (you can look at some of the .net core ARM developments to get a feel for this, but the same goes for JS/Java etc). Webbrowsers are hyper-tuned. regexengines, packet filters, etc etc etc

Not to mention: just getting a compiler like LLVM to support a new ISA as optimally as x86 or ARM isn't a small feat.

Finally: at least at this point, until our AI overloads render that redundant - all this work takes expertise, but that expertise takes training, which isn't that easy on an ISA without hardware. That's why Apple's current transition is so easy: they already have the hardware; and the trained experts some with over a decade of experience on that ISA!. But if they really want to go their own route... well, that's tricky, because what are all those engineers going to play around on to learn how it works; what's fast, and what's bad?

All in all, it's no coincidence transitions like this take a long time, and that's for simple (aka well-prepared) transitions like the one's Apple's doing now. Saying they have ISA "flexibility", like they're somehow interchangeable is completely missing the point on how tricky on those details are, and how much they're going to matter on how achievable such a transition is. Apple doesn't have general ISA flexibility, it has a costly route from specifically x86 to specifically ARM, and nothing else.

Extremely aggressive optimizations are really special though, and they tend to require rewrites when new CPU extensions release (and workarounds to work on older hardware). If you rely on super low level ultra-aggressive micro optimizations your code is going to have a relatively short shelf life, different ISA or not.

The vast majority of the code written for any given computer or smartphone doesn't have this level of sophistication and optimization though. I'd wager that for most code just changing the build target will indeed mostly just work.

It won't be painless but modern code tends to be so high level and abstracted (especially on smartphones) than the underlying ISA matters a lot less than in the past.

That's not my experience. Plain old SIMD code ages fairly well - but if you port to a new arch, you will need to look at it again.

This isn't a question of precise instruction timing, it's a question of compilers being pretty bad at leveraging SIMD in general, even in 2020. Also, while I'm sure lots of projects have hand-tuned assembly, even higher level stuff like intrinsics help a lot, and need manual porting.

Absolutely this - even some aggressively optimised stuff like game engines don't have insurmountable amounts of micro optimisations in them.

Also - why would Apple massively up and change uArchs? Even if they did decide to turn Apple Silicon into not-ARM, I'd wager it would look a lot more like ARM than for example, x86 does.

Do it more and more and they'll have the tools to efficiently manage them.

Also likely the small tweaks they will want from time to time should be "easy" to follow internally, if you can orchestrate everything from top to bottom and back.

I doubt Apple is dumb enough not to have basically a perpetual license for ARM.

ARM was launched as a joint venture between Acorn, Apple, and VLSI. I believe that since day 0 Apple had perpetual access to the IP.

They sold all of the ARM shares in mid 90s to prevent themselves from going bankruptcy.

Not to mention starting a JV has nothing to do with perpetual IP access. You will still have to pay for it.

They can certainly have a contract with Arm that allows them to renew their arch license in perpetuity that nvidia won't be able to void.

I obviously don't know that for sure but the idea that Apple would stake their future on something they don't have a legal ironclad position seems unlikely.

I would also agree, the thing is - businesses breakup and come together all the time. If it makes sense and both parties can agree despite of past disagrements and lawsuits, they will partner.

Just because Apple and nVidia has bad relationship at the moment regarding their GPUs is probably orthogonal to what they'll do with this new branch of nVidia, that is ARM.

What need does Apple have with ARMs R&D going forward? They have their own chip designers, build tools, etc.?

True about frenemies the entire time that Apple was suing Samsung, it was using Samsung to manufacture many of its components.

But if your chip heavily builds on arm's IP you need a license for that at least as long as you can't replace the infringing parts of the design. Which sounds very much impossible if you also want to progress on other aspects of having the best chips.

Apple uses the ARM ISA. It doesn't use ARM IP - as in, the design of the chip. Apple designed their own damn chip!

Since they're not branding it as ARM in any way, shape, or form, and they have a perpetual architectural license to the ISA, I suspect they could do pretty much what they please - as long as they don't call it ARM. Which they don't.

During the design were they careful not to create any derivative works of arm IP and/or not to infringe on any of arm's patents?

Apple co-founded the company. I imagine that they have a right to that IP.

Even if not, then yes, I would imagine that a behemoth of IP design like Apple would probably have considered IP rights during that design...

They definitely call it ARM in developer tools and technical documents.

I doubt that will be true going forward. They first used the “Apple Silicon” branding for the Mac and yesterday they used the branding for the Watch and iPad (?)

>Just because Apple and nVidia has bad relationship at the moment regarding their GPUs is probably orthogonal to what they'll do with this new branch of nVidia, that is ARM.

Yea, they got fucked by Nvidia business practices multiple times. There's a saying about shame on you, shame on me, etc. Unless the entire Nvidia business unit also gets replaced in the same transaction, it doesn't matter how much of a faux separation of concerns they want to market.

https://www.electronicsweekly.com/news/business/finance/arm-...

Broadcom has an architectural license. They do also license core designs.

Really if nVidia locks up the lower end cores, then a lot of stuff breaks. Billions of super tiny ARM cores are everywhere. ARM has few competitors in the instruction set space for low end, low power, low cost cores. AVR, PIC, and MIPS are what come to mind. And AVR/PIC are owned by Microchip corporation.

These ARM chip unit licenses are dirt cheap, there's hundreds of small manufacturers, and their chips go in everything, and in unexpected places. And these aren't just little microprocessors anymore. They're even in SoCs as little coprocessors that manage separate hardware components in realtime.

The amount of penetration ARM has in hidden places cannot be underestimated. And there isn't a quick replacement for them. Not one freely licensed to any manufacturer.

> nVidia could cut them off from any new designs

Why would they do that anyway? The downsides are obvious (immediate loss of revenue), the risks are huge (antitrust litigation, big boost to RiscV or even Mips), the possible benefits are nebulous.

Those who are most obviously at risk are designers of mobile GPUs (Broadcom, PowerVR ...).

If they do it that directly, sure. But on a large enough (time)scale, incentives are the only thing that matters. And they'll certainly think hard about putting their fancy new research that would help a competitor into their openly licensed chips from now on.

Curious as I don't know the terms of a perpetual architectural ARM license. But, is it valid only for a specific architecture, say v8 or v9, or is it valid for all future architectures as well? Or is it one of those things, where it depends per licensee and how they negotiated?

Nvidia mend. Lol

Reply on news.ycombinator.com