Amazon's Secret Weapon in Chip Design Is Amazon
spectrum.ieee.org
spectrum.ieee.org
More like "Amazon's Blindlingly Obvious Weapon In Chip Design Is AWS".
The difference is, Amazon's bet on commodified ML Compute infra largely paid off due to a mix of developer advocacy and the fact that there is a large existing market of users somewhat adept with AWS.
In fact, it could be a case study of how an incumbent can lose the ball - back in the 2014-18 period Tensorflow was THE framework, and Google absolutely could have used it as a killer app to market then new GCP (and they did try), but Amazon was able to outcompete GCP on both Containerization and Cloud ML Compute because of their strong developer advocacy and training programs.
Don't be dismissive about Amazon's historically strong developer advocacy motion. Peak Microsoft, Intel, Cisco, VMWare etc all placed similar bets, and Nvidia has done something similar since the mid-2010s in the ML space. At the end of the day, buyers are somewhat technical.
GTM strategy is just as important as technical and product strategy.
By designing their own chips and partnering with a foundry to manufacture them, Apple can create customized solutions that meet their product's specific requirements, distinguishing themselves from other PC manufacturers that use Intel/AMD chips.
Amazon recognizing it as an opportunity to stand out from the competition in the cloud services market by offering its own chips.
Tbf, Apple transitioned to Samsung by the early 2010s for their SoCs. The MacBook on Apple Silicon was a recent transition after the kinks in the iPad Pro (which is laptop specced) were ironed out.
> Amazon recognizing it as an opportunity to stand out from the competition in the cloud services market by offering its own chips
Exactly, and evangelizing earlier than other cloud providers.
If they're doing better than the others, good for them. It still blindingly obvious that having the biggest cloud is a huge advantage for chip design and to successfully exploit savings because of that scale, not some sort of super amazing secret just now being revealed by IEEE.
Google had the advantage of owning the entire ML and Infra stack (TensorFlow, K8s, BERT, CNCF) and Microsoft had an inbuilt advantage in research communities thanks to MS Research's outsized impact in fundamental ML research.
At the time, the Annapurna Labs acquisition was seen as a massive coin-toss because IBM went down a similar path a decade before and failed.
I thought for a few minutes and I could not come up with an example of an ML technology that originated at MS Research and then spread outside MSFT. Care to give some examples? Thanks!
In the 2010s they were the leader in NLP and the precursor of LLMs like GPT3/3.5/4/4o
Machine Translation with Human Parity (2018) - https://arxiv.org/abs/1803.05567
MT-DNN (2019) - https://arxiv.org/abs/1901.11504
MASS (2019) - https://arxiv.org/abs/1905.02450
VALL-E (2023) - https://arxiv.org/abs/2301.02111
VALL-E 2 (2024) - https://arxiv.org/abs/2406.05370
While OpenAI was the first to monetize an LLM at scale via ChatGPT, it's still the early stages of this field, and there is a lot of innovation that can still be leveraged, especially in non-English language modeling, machine translation, text-to-speech, etc.
It's in this segment that Microsoft Research shines moreso than even Google Research let alone other organizations because of their strong NLP background in Chinese (Microsoft Research Asia), South Asian languages (Microsoft Research India), Arabic (Microsoft Research's older work during the Iraq War), etc.
Ditto for PA Semi, who were bought by Apple.
SiFive changed to being an IP provider, is no longer selling chips.
It feels sometimes like the PC world is in collapse. No one wants to sell chips any more, it's all for cloud or for appliance-ized phone-lile systems. When someone does come along and starts making new chips, they get bought.
I wish the hyperscalers had some interest in keeping a competitive market alive, in supporting independent competition. I respect the desire to have a strong in house team and Graviton 3+ have really been excellent (Nitro is also super interesting deep tech I wish we saw in the world; alas AMD's SeaMicro acquisition took one similar out in the wild super-fabric offering off the table rather than promoted it). But man, the consolidation in chip making has been brutal & it doesn't feel like there's enough folks getting started making cores to keep things healthy. There's some disruption from below with RISC-V starting, but it's been slow & is very down market still (Tenstorrent being the notable exception). The broad ecosystem feels like it needs help, needs new vitality, is ossofying, in large part because of these acquisitions.
Alternatively, smartphones are the PC (personal computer) of choice.
SoC and mobile chips are themselves near desktop level performance. Heck, Apple's A18 can outcompete the M1 in certain benchmarks, and outcompetes a Kaby Lake Intel i5 (2017-19 period) in most aspects.
We are reaching a point where commodity mobile processors have mid-2010s desktop chip level performance but at a fraction of the cost, which opens up plenty of opportunities.
And there has been some progress getting some phones running real Linuxes, with upstream kernels & more regular userspaces. There's some cheating too, using hybrid Android drivers bent to be Linux-y.
But man it is so irritating to me that there's such tight controls on these chips in every way. So many seem to only be available for large devices makers. There seems to be very narrow segmentation. Ideally one would hope a small SBC with say a Snapdragon 7s Gen 3 (SM7675-A), with a single X4 core, would be so interesting as a low cost small board. But instead of the most popular chips on the planet - cellphone chips - being everywhere, there's a whole market of extra special extra old-core embedded chips - the Allwinner and Rockchips and Broadcoms - providing an alternate. One would hope removing display, touchscreen, battery, and case could lower costs, but it's just not done. https://www.anandtech.com/show/21316/qualcomm-intros-snapdra...
And phones, man, so many are locked locked locked down. Many of the Qualcomm Snapdragon X Elite laptops are locked down - laptops - which can by design install no OS but windows. Highly restricted use seems normal in the ARM world, makes these so much less able to be enjoyed.
The other major downside to phones is they have enormously limited io. A single USB 3 port isn't the worst, but it's still a very narrow straw for something like a modern flash drive to squeeze it's data through. Hopefully, again, USB4 improves this, but ideally I'd love some PCIe connectivity and especially multi-port designs (Lenovo has a couple gaming phones with >1 port in some markets, way cool).
I agree with the excitement for mobile. But it's also a dark segment, a pinnacle of consumerdom & regression to the mean versus that brief great amazing age of Personal Computer compatible, which has spawned systems both small and massive and mighty, that we have been able to extend & use however we might imagine. Phones returned us to an age of control, where our species is impotent at using the tech we have all around us, phones are infernal devices trapping us, are the spiritual foe of human spirit.
This is a company that is actively shooting itself in the foot on all fronts and by the time they realize it’s gone too far they won’t be able to turn the ship fast enough.
I have had so many issues with orders from Amazon recently that they told me I’m on a watch list for returns and could have my account terminated.
I’ve been a customer for about 20 years with this account.
I’ll be shopping at other retailers now.
Each had an issue as a result of Amazon’s shady practices. One was returned, sold as new. The replacement was defective.
So I ordered another model. First was damaged in transit. Second was a return that another OEMs charger.
I said “fuck this” and went to Best Buy, paid approximately the same price and had an actual, new, working Chromebook for my son.
My account is not closed (but they "reserve the right to close it" anytime). I am glad I have no kindle or DRM stuff...
I tried to get an explanation, but just got a robot email.
I will never buy there anymore. Even if some stuff are hard to find elsewhere
Amazon has not lost, but it is definitely losing its unique edge.
Those are three examples in the last couple of months…
Do they actually use such direct language? I received a passive-aggressive message from Amazon implying I was returning things too frequently (which was unfair IMHO; I had just returned three items that I had purchased as part of a bundle, but which might have made my return stats for the year look bad). But they haven't escalated it yet for me with more direct language.
I’ve got friends who’ve moved from AWS-centric roles to Azure-centric, prime is a dead product walking, twitch is unprofitable, Alexa is dead, and their original business is squeezing users, drivers, and employees. A strong viable Amazon.com alternative and they are done.
This is my opinion and I'm not trying to convince you of it. Have a nice day.
I make most of my money supporting companies on AWS, but I've stopped recommending it to new shops largely for the same reasons - if you're not big enough for them to knock on your door, they won't answer when you knock on theirs.
I remember one move I made to a new city. I'd downsized a bunch of my stuff and got settled in pretty quickly. Everything was right with the world except my Harmony remote didn't survive the move and my guitar hanger got left behind.
So I looked at Amazon to at least get a baseline price for these two things, and they offered to deliver both of them to my door in a couple of hours.
I sanity-checked the prices and they were fine.
"What is this wizardry," I thought to myself, when I had both items at my door in a couple of hours.
There was a time before that when they had a better web presence for selling regular hardgoods than Amazon (which was still mostly known as just an online book store), and the Sears Parts website was the very first place to look online for manuals, diagrams, parts, and standard accessories for any random household thing (including shop vac attachments).
They also co-founded the Prodigy network back in the 80s when home computers were still very novel and people weren't broadly sure if the concept would ever catch on.
Sears could have been Amazon if they kept their customer obsession.
[1] https://en.wikipedia.org/wiki/Sears,_Roebuck_%26_Company_Mai...
and
https://www.waymarking.com/waymarks/wm4RCM_Sears_Tower_Seatt...
The Sears Catalog was the Amazon of the early 20th century. So we must ask ourselves "Why did Sears discontinue its catalog service in 1993?" One of the answers to that question is the rise of retailers like Walmart where people could walk into a local (or at least closer to them than a Sears location) store and buy all sorts of things that they otherwise would have had to order. The catalog was inconvenient with other alternatives available.
The concern with Amazon is with its delivery ability - sure, for now, their unsustainable model that burns out drivers and pays them a pittance is working. Should that slip where they cannot deliver same-day/next day/day after reliably then that's an opportunity for other retailers to do to them what Walmart did to Sears.
You can even see it in this discussion - Walmart's online component directly competes and with many, many more local retails locations than Amazon and can often either have the items ready for pickup the same day or even deliver the same day.
So yes, there's a precedent for this, and if Amazon is focusing more on AWS and the buckets and buckets of money there but starts neglecting the retail part then that leaves a massive opening for competitors.
Unless I want to buy something weird, in which case there's also a good chance delivery fails entirely for some reason and the order gets cancelled on the other side.
Also didn't have very good experiences in the UK (Cambridge and Portsmouth).
In 2023 I placed exactly 100 orders and this year I'm probably going to hit the same number. I've had very few problems over the years (my first order was March 29, 2000). During that time, I've lived in a bunch of different cities across three states.
Our apartment building has a package room with a camera and my packages have never been stolen from it, and the delivery instructions in my Amazon profile say to use that package room including how to get there and that Amazon can tap into the room. Often times delivery drivers don't do that and just leave the package on a shelf out in the open next to the mailboxes. I cannot even count the number of times someone has ripped into the package to see if it contained something interesting only to leave it behind when it's boring like pencils or cleaning supplies.
Amazon refuses to take any responsibility for packages stolen and has in the past denied refund requests without a police report on the incident (which I have filed multiple for just to get a refund though the package thefts are never investigated so I'm not sure what filing a police report is meant to do here).
Their delivery is inconsistent so I just don't bother. Though that does mean that I don't order as often from them as perhaps they'd like - if I can pick it up locally after work then I'll do that rather than order it and wait for it to be present and unopened only to be disappointed yet again at the Amazon driver who didn't read the delivery instructions which state multiple times not to leave the package next to the mailboxes because it will be destroyed or stolen.
I don't think they have the best name reputation. At least people my age, WalMart is synonymous with cheap and poor quality. But that was before the age of dropshipping and Amazon, where now Walmart's offerings are probably actually better quality than a lot of stuff. IMO they probably should have leveraged Jet there instead, but it is what it is.
What are we comparing to here?
Google Cloud? Azure? Linode? Openstack? OVH? Oracle Cloud? Hetzner?
They all suck too, just in different ways.
I can think of a dozen quotes related to the decline of a thing being "impossible", and then sudden.
Enron for example had it's best year, right before it suddenly stopped existing.
"The candle that burns twice as bright, burns half as long", and "The night is darkest before the dawn" both relate to things appearing one way because the end is more likely.
Amazon is mostly inertia, they are user hostile to a fault, which is a far removal from what made them huge in the first place. Normally when there has been such a reversal in a companies principles: it's the beginning of the end.
I'm not desperate for it, I couldn't care less. But what we will witness now is the power of capital to beget capital, don't be fooled into thinking Amazon is innovative.
Don't anthropomorphise the lawnmower.
One point that is not mentioned is that Amazon is making boatload of money by advertising. Their ad revenue is 50 billion dollars and rapidly rising. This make them look invincible and total shit at the same time.
There are countless examples of companies that seem to have a lot of money then die suddenly.
There are plenty of zombie companies (IBM) too, hard to know what this will be, but I wouldn't take financials as an indicator of anything really.
For profit businesses exist to make money. That is all. Any other claim is fairy dust and bullshit. IBM may not be making waves anywhere but they are still pulling at least 10 billion in profit so they are doing a hell of a lot better than most other companies.
The intended purpose of a corporation is to do something. Profit shouldn't be a goal in of itself, it's a consequence of successfully doing something.
Revenue is good, as it employs people and keeps society churning, but this slavish devotion to the idea that companies only exist to make profits is alien; it's the personification of greed.
Many companies exist because the founder had a passion for actually doing something, and they can make money doing it. If they chased only profit, their life would be miserable.
> If they chased only profit, their life would be miserable.
Get some hobbies and don't make work your life.
It appears to be a reasonable strategy, as seen with Apple's successful introduction of the M1 chip, which set them apart from other PC makers.
They have the highest prices of any cloud. What happened to “your margin is my opportunity”?
And, as far as I know, customers are unable to allocate a VM with fewer than eight A100, H100, or H200 GPUs. (Please tell me how if I’m wrong.)
So, customers are incentivized to use other cloud products for GPUs in the short term.
They seem to be heavily invested in their own chips in the medium term.
I thought Jensen recently said he does not want to offer their own cloud offering. He instead wants to focus on creating ready made solutions for cloud vendors to purchase & re-sell services with.
It takes more effort to virtualize SXM5 than individual PCIe cards, so you find availability in 8s in VMs too regardless of provider
Also, in case you didn’t get the memo GPUs are hard to come by most places these days.
Edit: Misspellling ‿
Will probably take them to Small Claims Court. Worst Customer Support ever.
Such a scam.
My wife is happy with them though.