Amazon Will Spend Nearly a Year of AWS Revenue on AI Investments | Hacker News Reader

Amazon Will Spend Nearly a Year of AWS Revenue on AI Investments

nextplatform.com

49 pointsrbanffy79 comments

I left AWS Professional Services in 2023. Being in ProServe, I interacted with a lot of different service teams (teams that work on various AWS services) and I got a first hand view of their messaging and strategy.

Internally, Amazon/AWS has always been a shitty company for employees. But they always were somewhat innovative and long term focused. I saw things going down hill by the end of 2022. They are definitely a “Day 2” company now and not out in front of trends

As if the original RTO policies weren’t bad enough for instance, now they are forcing their “field by design” workers to be in the office 5 days a week when they aren’t at a customers site. These are the consultants (full time employees), sales, SAs etc who spend most of their time interacting with the customer and where you rarely have your internal teammates in your same office.

Before anyone asked why I worked there if I knew it was shitty. I worked remotely the entire time, it looked great on my resume and I like money. It definitely opened some doors

Were you one of the many overemployed people that also worked at AWS?

I am 50. I have always had a strict policy of never doing two jobs or a “side hustle”. Whatever we can’t do from my working one job, we don’t do.

That means no open source work worrying about my GitHub profile, no BS “thought leadership” posts on LinkedIn. Nothing.

When I get off of work, I shut my computer down and live my life - exercising, travel, spending time with friends and family, etc.

That's a wonderful attitude. I'm the same. The stuff that's on my GitHub is there for me, I needed it and/or I wanted to contribute to it. It's not for visibility.

But with that I do feel that I've had a couple of jobs slip through the cracks. Not enough exposure.

The truth about not doing open source work and a personal GitHub profile is kind of lie.

The department I worked for at AWS had a very easy to use open source approval process to open source work we did from scratch for a client as long as it didn’t have client information we could put it here after approval.

https://github.com/aws-samples

I was able to legally/ethically fork all of the work I submitted (MIT Licensed) to my own profile after I left. But, no one cared or even looked at my open source profile.

However, I was also a major contributor to a popular open source “AWS Solution” that was big in its niche.

https://aws.amazon.com/solutions/

That did lead to two interviews and one job offer in 2023 year.

Almost all of my projects and the “solution” are obsolete now. I was able to use most of them last year with some tweaking.

Imagine if Amazon acted the same way- oh we already have 1 customer, we won’t bother with the next 6 billion because we have one shittty one already.

What exactly is your point? I should be part of the “hustle culture” and work my main job and a have a side gig and sacrifice my free time?

I’ve heard most of the AWS services that are actually profitable are the old ones, and that all the new ones barely make money. Is that true?

What do you mean by making money? That’s not a simple question to answer. At the end of the day, all of AWS runs on a bunch of VMs that are spun up with a certain AMI depending on what the workload is. I’m sure AWS is charging enough to make the price more than the compute costs except for services that are completely free like CloudFormation.

If you mean do they cost more to develop than they are currently making for income, that’s how tech companies work - high fixed costs for initial development and low marginal costs that are spread across the customers.

They have started canceling projects where the cost of continued development doesn’t make a direct or indirect profit and don’t add to the ecosystem. The two I can think of most recently are CodeCommit (a very bad hosted git repository) and Cloud9 (hosted development environment).

I mean that only a few of the services generate all the profit in AWS, and the rest are currently priced at a loss.

Surprised to hear they’re canceling some of them. That usually scares away developers.

I mean it’s literally one command to retarget the remote url of your git repository and Cloud9 was just a remote hosted VSCode with some AWS extensions you can get locally

Which business unit were you in?

WWPS

This seems like a fine decision by Amazon?

If there is a ton of demand for shovels it seems fine to make more shovel factories?

There is a redflag here. Their CapEx is growing faster than revenue. And there is no apparent direct correlation between AWS growth, percentage wise, and CapEx surge...Using CapEx makes profits look higher today...

Did we read a different article? It seems like the claim is that it pays for itself in under 2 years.

> For AI servers and the datacenters that wrap around them, as we have shown here, for every $1 you spend for an AI cluster and its datacenter (including its power and cooling), you get 87.8 cents back every year that you rent that AI cluster out to customers, assuming a reasonable mix of on-demand and reserved instance hours and a $9.40 per GPU-hour rate. In 410 days, you get your bait back on the entire AI datacenter investment if you rent all the GPUs out. The model we created (riffing on an Nvidia model) shows that for the three years after that, the 87.8 cents per year goes right into your pocket.

I am discussing this in the context of their disclosures for Q4 and historical reporting. The "pays itself in 2 years" is intuitively wrong and on the basis of 100% usage estimates.

CapEx exploded while depreciation barely increased means Amazon pushed more spending into assets that won’t be depreciated immediately. This inflates short-term profits. In 2024, CapEx surged to almost 120% of operating income. They are spending more on investments than they earn from operations. I dont think it's sustainable.

If this is a good decision or not all depends on what the ROI ends up being.

They are paying an opportunity cost to have access to future growth.

What’s the alternative? Not have the capacity when needed?

If it’s clear there’s a shovel bubble, though, then it’s a bad idea to over-invest in shovel factories.

Agreed, investing in platform facilities and AWS capability seems like it cannot be a terrible idea.

This seems like a fine decision by Amazon? If there is a ton of demand for [dog shit tacos] it seems fine to make more [dog shit taco] factories?

Welcome to capitalism. And remember that we're the ones paying for the dog shit tacos that Amazon sells to the companies that then stuff them down our throats, whether we want them or not.

Is it us, or is it VCs?

For the foreseeable future at least.

Financially liquid companies locked into feeding AWS (with their 50%-ish profit margin) should buy Nvidia/TSMC/ASML/(AMD) stock to get some of their money back?

Totally not a bubble.

I've heard arguments that you should buy NVDA as a hedge for losing your job to AI. Not sure what to make of it.

If AI actually starts replacing jobs, society is going to collapse/restructure to the point that risk management no longer applies.

> If AI actually starts replacing jobs, society is going to collapse/restructure to the point that risk management no longer applies.

Once more, with feeling: automation has continued to replace jobs. That's why you aren't currently toiling in a field to eke out a meagre subsistence living. If AI starts to replace jobs (a statement made true if a single job is replaced by AI) then likely the good or service that fronts that job will become slightly cheaper, or the same.

What if AI can replace 100% of jobs that don't require manual labor?

I imagine we'll invent new jobs that we want humans to do. Why do people pay thousands for mechanical watches when quartz ones are objectively better?

It is replacing jobs

> It is replacing jobs

Prematurely, in my opinion only. A lot of those will be coming back unless there's a major breakthrough soon.

Which ones? In reality, I mean, not executives’ fantasies.

I spend 90 minutes in a meeting with our CEO and two board members and the one guy went on almost the entire time about how amazing OpenAI Operators was and how we should find a way to replace most of our people with it. All because it was able to buy a plane ticket after he give it his login credentials.

Perfect example of a fantasy put forth by an executive.

these kind of comments on HN show me how many people lack creativity. wherever you work(ed) I am sure you met myriad of people and thought:

- they don’t do shit

- I can do what they do in my sleep

it is these types of jobs that are being replaced and are easily replaceable with a little bit of creativity and imagination…

The parent comment asked (very specifically!) for real examples, not C-suite fantasies. They received a reply literally describing the fantasy of a c-suite individual. I thought that was funny.

How my one sentence reply pointing out the irony of the reply led you to believe that I (we? lots of HNers?) lack creativity is beyond me.

If management knows that underperformers are underperformers in order to fire them in favor of LLMs then management could do it already in favor of better workers.

You can basically string replace "replace underperforming humans" with "replace humans". That is their dream

I'm dealing with this as well. CEO basically soiled himself with excitement in a group chat, some toadies joined in, most were silent. Because how else can you receive that other than "think of all the money we'd have if we could get rid of those needy workers"

The workers on whose work the thing was trained, too

You didn't notice a bunch of layoffs in 2024?

Even if the technology of AI ends up driving a huge economic value there is no guarantee that will translate into NVIDIA stock performance. The expectation that NVIDIA will capture a lot of this value is already priced in to the stock, which is a major reason why the current price is so high.

What I can't wrap my head around is the fact NVDA almost is a representation of "AI infrastructure" in the narrative. It is weird now that everyone is saying the focus is be in inference / test time scaling compute: inference should be easier to offload to chips from other manufacturers.

Maybe I'm too dense to get it, but that doesn't make sense to me.

I think there is a money to be made in making huge slow inference chips

It needs,

* Does the very best model fit in ram

* Does it eventually produce an answer

I'm completely ok with waiting a few minutes for an inference answer if I can avoid spending $250k for an 8x

Me too. Just as most problems won't need "phd-level intelligence" (whatever they might want you to think, 10x devs don't require phd skills nor research abilities / test time compute for being good at their job), most problems won't need subsecond latency or response time.

I'm fine waiting, not sure if others really need the best now or ever.

A better hedge might be private prison corporations and private military contractors.

For prisons see GEO (up 100% since the election [0]) and CXW (Up 50% [1])

gov't contractors are in a bit of a bind given the treasury chaos (consider the current agenda to cut waste and inefficiencies)

Looking around for private security I don't see anything publicly traded but I did learn there is an ETF named GUNZ [2] and one might check out the companies they invest in.

[0] https://finance.yahoo.com/quote/GEO/

[1] https://finance.yahoo.com/quote/CXW/

[2] https://finance.yahoo.com/quote/GUNZ/

AWS has a lot lower margins that most enterprise IT companies.

No.

To everyone going for the shovel metaphor: Samuel Brannan cornered the entire shovel supply of San Francisco before running down the street shouting "Gold! Gold on the American River!". He'd have gone broke if he had just done a big capex to offer Shovels as a Service. He still died without enough money to pay for his own funeral.

https://en.wikipedia.org/wiki/Samuel_Brannan

Really looking forward what all that GPU compute is going to do for non AI stuff once it quiets down a little.

With that much power, we’ll be able to reliably predict a hundred thousand years into the future of our societies and shorten the period of barbarism after the fall of the galactic empire.

I was wondering all the crappy simulation software used in engineering that is still CPU-bound.

Does this mean that I will be able to use my Echo Dot for something other than asking what the weather is?

My daughter routinely asks hers to tell jokes.

Amazon requires employees to work five days a week from the office without paying them extra for it. Even if paid extra for it, it is not healthy considering the number of contagiously sick people at the office, risking a persistent inflammatory load on the body. I don't see good workers wanting to work for Amazon, and I don't see good outcomes ahead for Amazon. It will remain a non-innovating laggard that is trivially outcompeted. The only reason AWS even profits is because of its outrageous egress cost.

All of the BigTech companies are forcing RTO. Hybrid work is just as bad RTO. Either way I can’t make an above local market wage and live in a low cost of living area.

Anytime any company reaches out to me and if I ask is it fully remote and they say “no” the conversation ends. I almost ended my initial conversation with AWS in 2020 when they reached out to me about an SDE position that would require an eventual relocation until they suggested I interview a ProServe (no longer there),

I’ve since stopped conversations for a similar position at GCP once they said it would be hybrid. Not that I would ever work for a large company again unless I didn’t have a choice.

Hybrid work is just as bad if you live well outside of a commuting distance. If you live within a commuting distance, it is somewhat acceptable if it's about two days a week with a decent salary.

Good remote jobs are very difficult to get as they are very competitive, so hybrid jobs fill that void for those who can commute.

5x/wk RTO is nothing but a means of growth control and cheap layoffs.

That’s very true. When I was looking for my jobs in 2023 and again last year, it’s a shit show out there trying to find remote software development jobs if even all you are looking for are regularly old enterprise CRUD jobs. That was my “Plan B” both times.

I was lucky that even before COVID around 2017 my goal was to work in strategic cloud consulting focusing on app development (ie not staff aug) because that was the only way I could see making above local market wages without having to commute and even back then most of those jobs were remote with travel.

But then again, if you don’t have to worry about moving somewhere with a “good school system” because you have kids, why not move somewhere closer to the job?

It's OK. The revenue is from incompetencies of engineers and enterprises that can not switch into siginificantly cheaper cloud vendors.

Where are these significant cheaper cloud vendors?

Oracle. I like bare-bone ARM VM and bare-bone volumes.

Going from the embrace of a semi-domesticated mostly aloof wolf to that of an understimulated psychotic tiger.

Uh... Good luck?

… Everywhere ?

OVH, Hetzner, Cloudflare, Vultr, Linode, Scaleway, Equinix (ok Equinix this can be pricey at lower levels)

A million other regional VPS providers, Colocation Hosts, etc ?

Most of these are what I'd call "cloud in name only" providers - everyone uses the term but you would have significant challenge moving a cloud workload that makes use of the higher layer abstractions to these.

There are very few workloads that require more than what you can accomplish with a handful of VMs. Using tools like Terraform makes it a lot easier to abstract away the specialised services.

Yeah, you haven’t worked with any large companies if you think that.

What’s the largest implementation you have been responsible for?

I work with a couple workloads that can’t even be completely deployed in cloud environments. Those aren’t common.

The vast majority of companies can get along with a Google office license and a Wix website.

Not everyone works at a company with hundreds of thousands of employees and hundreds of millions of users.

I agree RDS, Aurora, Big Query, S3, load balancers, declarative security policies, artifact repositories, managed auto-scaling clusters and so on are very convenient, but they aren’t a requirement.

You “worked with a couple of workloads” and that’s what makes you an expert on infrastructure and architecture at scale?

Your LinkedIn profile is your HN profile. I see you have worked at some large well known companies. How can you possibly not have been exposed to some large deployments?

These are three companies that you have worked at

https://www.workday.com/en-us/company/partners/amazon-web-se...

https://aws.amazon.com/solutions/case-studies/zendesk/

https://www.spartasystems.com/resources/the-honeywell-life-s...

And none of those is “the simple case” I alluded to. The vast majority of businesses need, perhaps, email, file sharing, instant messaging and, perhaps, a website. They won’t train their own ML models, nor have parallel sysplexes of mainframes spread across multiple datacenters.

So, in the grand scheme of things, how much revenue do you think all of those small businesses combined make compared to just the combined business revenue and compute needs of just the large companies you have worked for?

It surprises me that you have blinders on to an industry where you personally have worked for companies with large enough implementations that AWS has felt the need to brag about

In what does that change my initial observation that the vast majority of companies don’t need more than the most basic cloud services?

Who are these vast majority of companies? The second company I worked for between 2000-2008 that I spoke about here:

https://news.ycombinator.com/item?id=42985966

With 20-30 people on staff if I were responsible for that architecture today. I would need:

A MySQL instance with read replicas (we had that back then onsite)

SQL Server for a some legacy projects - we had those too.

File Transfer family for FTP transfers and some automations around that.

Web servers and load balancers.

SQS and probably Lambda. Back then we used MSMQ and later MQSeries and a home grown application servers that took care of asynchronous message processing.

Web servers - a couple of EC2 instances and a load balancer and these days because of how the internet is probably WAF.

We would have needed something to orchestrate our ETL jobs. Back then we ran on 15 physical computers we would probably use something like AWS Batch today.

And of course S3.

You see how quickly your needs escalate once you are doing any real workloads?

The next smallest company I worked for had 50-60 people this was between 2018-2020. We sold access to aggregated publicly available health care provider data as well as some other health care related data. Our micro services were used by large health care companies as the backend to their websites and mobile devices and one new customer could increase the load on services they subscribed to by 20%.

Here we also needed multiple MySqL databases, CloudFront, WAF, Cognito, ElasticSearch, Redshift for large analytical loads, EC2 for some legacy software, S3, ECS for the microservices, Lambda/SQS and step functions for some ETL jobs that scaled from 0 to hundreds of thousands of transactions, Cognito for authentication, etc

You might not remember. But around March 2020, health care providers websites were being hit hard because of a little virus that was going around, the scalability that we put in place came in handy then.

Do you propose that we should have hosted all of that on some VMs?

You need to find the right balance between an expert IT team and cheaper employees. Using pre-baked cloud services is always easier, requires less management, but the operational expenditure is higher while the staffing cost might be lower. Where the company is based will also impact the professionals you'll have access to - there are places where you can have highly skilled people, and places where you'll struggle to even fully staff your IT operation.

> Do you propose that we should have hosted all of that on some VMs?

What do you think Amazon uses to run the services you pay for? Unicorns?

You really think that all people do with AWS is host a bunch of VMs and they manage all of the services (databases, storage, big data services, messaging apps) themselves?

Drill down in each category

https://docs.aws.amazon.com/whitepapers/latest/aws-overview/...

I’ve used most of those services and know well that outside of DynamoDB, EC2/Lightsail, S3, DataLakes (s3+athena) and AWS Lambda.

Most of aws services have sub-par performance for insane prices, for components that can easily be replaced at scale with open source more mature components.

Programmers have just lost all of their sysadmin skills these days and will use everything to justify the need for these insane , low performance abstraction abomination that Netifly, AWS RDS, AWS RedisCache, AWS TimeseriesDB, etc are.

Whatsapp used to process billions of users and exited for a billion dollar+ valuation and exit, with just a few FreeBSD running VPSes running their code, and a 20 person employee team…

Most people will never touch stuff in scale of whatsapp, stack overflow, instagram.

and they did it all just fine. Programmers should be mandated to learn Linux and Sysadmin skills again instead of re-inventing entire bullshit services for what is essentially often systemd replacement or a crontab/fcron replacement.

Maybe all of these companies know something you don’t know?

For every WhatsApp that you cite used their own infrastructure, I can site companies like Netflix and Apple that decided to use the public cloud and AWS in particular

> Programmers have just lost all of their sysadmin skills these days and will use everything to justify the need for these insane , low performance abstraction abomination that Netifly, AWS RDS, AWS RedisCache, AWS TimeseriesDB, etc are.

Does it add business value as programmers to spend time babysitting infrastructure? Does it “make the beer taste better”? Does it help to move faster? I can set up an entire highly available data center with all of these services you mentioned with a few lines of yaml/HCL.

Spoil from incompetents is the good strategy and all too much nerd engineers should follow the strategy.

Reply on news.ycombinator.com