Hire-to-fire at Amazon India? | Hacker News Reader

Hire-to-fire at Amazon India?

leetcode.com

962 pointsbobjones334552 comments

Former AWS engineer here.

I worked on a pretty critical product in AWS (big AWS service with lots of traffic) and I can safely say that it's totally up to your manager and pre-existing conditions which make up the job. My manager was great as a person but would always lack in my career-oriented goals (bigger projects, promotions, etc)

But what really sucked for me was the pre-existing conditions. Our on-call was pretty bad (40-60 tickets a week) and there was very little investment being put in to improve it. We had a lot of little scripts here and there which would solve extremely specific situations but no focus was ever put on in building a general framework or trying to reduce the ticket count. This often led to engineers taking the day off after their on-call due to the load and honestly it made people quite grumpy. And upper management was always much more interested in feature delivery since the focus was always on promotions and the more you delivered the better it looked for your manager. So now you have engineers with such a terrible on-call load along with pressure to deliver new features and projects within the atrocious tight deadlines that would be set. It was, to be blunt, a shit show.

Code quality was atrocious. We had one enormous Java method (>1000 lines) which would take care of nearly every single request coming into our service... With only about 7-8 unit tests. It was so difficult to get even basic things done to the point where any ticket that needed to be done would take a minimum of 4-5 days regardless of complexity. And of course managers and senior engineers would estimate small tickets to take around 1-2 days and then be shocked when 2 days later it's not even close to being finished. I will give Amazon credit that they do grill design reviews pretty harshly so those are done well in general. But code reviewers didn't care about quality or best practice. If it works then ship it.

I'm just not 100% sure about the whole PIP scene. Our service was extremely critical and we were extremely understaffed. So I don't think it applied to anyone in our org but I know of other teams who would have no issues in taking in a fresh college grad, making them do work for 6-12 months and then just randomly putting them on PIP. Sad but I've seen it happen a few times in my time there.

I'm glad I got the Amazon stamp on my resume and left. When I left, more than half my team and my manager quit around the same time too. It was definitely a wild experience.

> Code quality was atrocious. We had one enormous Java method (>1000 lines) which would take care of nearly every single request coming into our service... With only about 7-8 unit tests

AWS is pretty reliable for the most part so I am pretty surprise that the code quality is that bad.

I think one thing I learned from AWS is that there's so much hidden away from the customer. There definitely were (and probably still are) many issues which the customers won't actively experience. Reliability doesn't necessarily equate to good standards and good practice.

But yes, from a customer point of view, AWS is pretty nice.

Reminds me of that old quote by John Godfrey Saxe:

”Laws, like sausages, cease to inspire respect in proportion as we know how they are made.”

"I just had one for breakfast." -- Jim Hacker

I think that highly depends on the service. The new App Runner service for instance is a wild ride of buggyness, lack of testing and incorrect documentation.

> AWS is pretty reliable for the most part so I am pretty surprise that the code quality is that bad.

I'm not totally surprised because of two factors: very stable product definitions and lots and lots of users.

A number of years back, I was talking with people at a famous and popular site with a broad audience. I asked them how much unit testing they did. They said that particular isolated pieces sometimes had tests. But most of the user-facing stuff didn't because they had one-button rollout and one-button rollback. Instead of bothering with unit tests, they'd just frequently release changes, watch the metrics and the customer support queue, and quickly roll back if they'd introduced a bug.

For very, very popular services, a second of being live will exercise more code paths and edge cases than even the most dedicated testing team could ever dream of.

We hear a hell of a lot about testing but the most fundamental piece of software quality nowadays is the release strategy: running on tee'd live production traffic, canarying, metrics and alerting, quick roll backs, etc.

Depends on if you are serving ads over cat pictures or routing air traffic. Different solutions for different problems.

It's a very short-sighted view on testing, although I'm not surprised SREs would say it. The biggest problem with software deployment is that it is owned and managed by people who have no vested interest in developer productivity, including devops engineers.

A major goal of any org should be developer productivity; otherwise you are just hemorrhaging money and talent. When I say developer productivity, I mean: How confidently and quickly can I make a shippable, rollback-free change to a unit of software?

If you are the dos equis man of testing, "I don't always test my code, but when I do, I do it in production", then you can't confidently make any change without risking a production outage, so you play lots of games, like you mentioned, around canarying, rolling out to a small percentage of users, etc., but at the end of the day your developer productivity has absolutely tanked.

The goal of any system maintenance should be that a developer can quickly make and test a change locally and be highly confident that the change is correct. The canarying, phased rollouts, and other such systems should not be the primary means of testing code correctness.

Yeah, I really appreciate excellent rollout strategies, although I suspect a lot of them are more developed out of self defense by SRE teams. I see it as a series of safety nets: I'm still going to write tests for my code so that I don't have far to fall if I make a mistake. But I also want a safe rollout so if I miss the first net I don't splatter on the pavement.

And I totally agree with out about developer productivity. It's just not a consideration in most places. For example, in a factory or a restaurant, meetings are things that happen rarely and in constrained time slots, because everybody realizes that production is primary. But in most software companies, actually getting work done is second priority to meetings.

Agreed. I was an SRE for over a year and the philosophy is that anything that is shipped can be broken. SRE is all about detecting, limiting, and mitigating damage. I think this is the right philosophy for SREs but should not be the total picture in the org.

I am also agreed in that I anecdotally often see a disregard for automated testing. I am still trying to understand how to eliminate this tendency. I know that in every software project I've had a major hand in building, I've helped ensure automated testing, with a heavy emphasis on unit testing, become a major part of team culture, and I've always felt the tests more than paid for themselves over time, even in the relative short-term.

For sure. I'm still trying to understand it too. Two things that have helped me:

To contain time pressure, I like a kanban board with small units of work. If the team has a history of steady delivery of small lumps of useful stuff, managers are more willing to trust that we know what we're doing.

To mitigate the natural human lack of humility, I start with the rule that every bug requires a test before fixing. People may act as if they won't make mistakes, but it's much harder to claim that when we have a live bug. And then I like to introduce pair programming. Collaboratively adding failing tests and fixing them (as with ping-pong pairing) makes it fun.

What approaches do you use to shift things over?

I find it's much easier for greenfield projects than existing projects because you can lead by example. Also, with these projects there are opportunities to define team rules and culture for the project from the beginning.

Perhaps the greatest inertial force against improving test automation is the truth that any change to a in-production product incurs risk, coupled with the fact that adding tests often requires refactoring to make the code more testable. Techniques like writing tests as you refactor never get employed because there is resistance to any refactoring occurring at all in the first place. The general principle followed is: "make the smallest change possible to accomplish the objective".

I feel like overcoming these forces requires bravery from management or the team, coupled with a longer term vision for the project. For some projects, which are limping along on life support, it may not even be worth it to improve code quality. However, for most other projects I believe there is a better balance to be had between not unnecessarily breaking the product and not being afraid to make relatively risky changes to improve maintainability of the product.

Makes total sense. I do most of my work on greenfield projects for that reason.

But I think you're right. There's really no low-risk path when you have a poorly tested code base. You can keep letting productivity decline, which guarantees eventual project failure. You can do a giant rewrite, which is hugely risky. Or you can gradually dig yourself out.

There's still risk there, of course. But it's in smaller, more manageable lumps. It seems like the clearly superior path to me.

If the release/rollback process is fast enough, and your detection of anomalies is fast enough, you can still have great productivity, and few relevant outages, when testing in production. Hell, there are situations where testing outside of production is never going to cut it, as generation of sufficient load of the right shape would take you a whole lot more of engineering time than the consequences of failure.

That said, the tradeoffs are different for different companies, and different services in the same company: Within the same team at $large_company, I owned code where testing in production, via deployments and an amazing feature flag system, was better than unit tests, while there were other areas where the build system would dedicate many CPU-hours to testing before any release. To be able to have that flexibility though, you need to know your systems, know your problems, and have great tooling for both testing in production and extremely parallelized test suites. Small and medium sized companies might not have either alternative, and we had both!

So what I'd say is that any general rule on what should be the primary means of testing code correctness is going to not lead to optimal productivity, and even more so if you don't have top quality of tooling across every possible dimension. It's perfectly OK to argue about specific examples, but without judgement of this kinds of things without having the entire story of what's there is just hubris.

I unfortunately can't agree with that sentiment. If I have to rely on production traffic to test my feature, then, at minimum, my feedback cycle is:

1) Make change. Think really hard about it to make sure it's correct.

2) put out code review.

3) get approval. Merge change.

4) CI pipeline builds and deploys to prod.

5) absent of alerts must mean it works?

Even if you have no QA environment and nothing between you and prod, I've rarely seen deployment to prod take less than 30 minutes. That's an hour feedback cycle. Contrast with:

1) write code.

2) write unit tests.

3) run tests locally.

The feedback cycle, especially when you get iterative, can get as low as single digit seconds. I run my tests and see a bug. I fix the bug, then re-run the tests. Similarly, for a more complex feature, I can break the feature down into multiple cycles of build, test, verify.

And that's not even accounting for the overhead of managing feature flags, which is not free. In the best case, you need to at least release a second PR to remove the feature flag when the feature is successful. At my previous employer, this step was often forgotten and resulted in real, consequential technical debt as it became harder to figure out how the product behaved based on which feature flags were turned off or on.

If you have an experience leads you to believe that production testing can more productive than local automated testing, I at least have never seen it occur in and I find it difficult to even imagine it being true.

That's an overly general statement. Can you do that for front-end code that stores all of its state elsewhere? Sure. Can you do it for a storage system? Absolutely freaking not. If you introduce a bug that loses or corrupts data, there's no going back. You will have committed the worst sin that somebody in that specialty can commit. Better to test as much as you can, at every level. Other kinds of code are often somewhere in between.

Also, even if it's true that being live will exercise more edge cases etc., it's a terrible way to test changes during early development. For one thing, there's no isolation. It becomes harder to determine which of several recent changes caused a problem, and that burden unfairly falls on the person who's on call instead of the person who introduced the error. And decent unit/functional tests allow "dumb" mistakes (we all make them) to be caught earlier than waiting in a deploy queue, allowing faster iteration. "Most recent change probably caused the problem" is a very useful heuristic, but the more low-assurance changes you allow in the less useful it becomes.

To drive the point home even further: I have found data-loss bugs in focused testing that didn't show up in prod for months. I know because in many cases I was able to add logging for the preconditions when I fixed the bug. No logs for months, then some completely unrelated and completely valid change by another engineer tickles the preconditions and BAM. That would have been an absolute nightmare for other members of my team, possibly even after I was gone. Based on those experiences, I will never believe that foregoing systematic early tests can be valid. The systems most of us work on are too complex for that.

"Test in prod" only works for trivial code and/or trivial teams. Not in the grown-up world.

Everyone should be testing in prod, in that you release code and see metrics and monitoring to show that everything is working.

Testing in production is not going “let’s see if this will work” it is “we will release and validate that everything is working as expected”

People need to get over the old school cowboys who jump on prod to see if something works.

Yes, everyone should release code and watch metrics etc. but I think that's at the very edge of what "testing" encompasses. Between model checking, traditional forms of testing, and shadow-traffic testing (which can test higher per-server load than prod), finding something after deploy should be like a parachute failure. Yes those happen, yes there should be a reserve, but if it happens more than once in a blue moon you have a process problem somewhere (quite likely between teams/services but still).

Tangential, where can I learn more about shadow-traffic testing? Books, blogs, tools etc.

Cindy Sridharan often writes well about these topics.

https://copyconstruct.medium.com/testing-in-production-the-s...

For this one I'd start about half way down, under the heading

"Shadowing (also known as Dark Traffic Testing or Mirroring)"

Unfortunately the terminology is a bit fragmented - shadowing, mirroring, teeing, dark traffic (ick), ad nauseam.

Whatever you call it, it's often pretty high overhead to add the infrastructure, unless you're already using some sort of "service mesh" (Envoy/Istio/Caddy/whatever) that supports it. Even then, if you're dealing specifically with a storage system then there can be some thorny issues - idempotent vs. non-idempotent vs. destructive requests, requests which require other objects (files/objects or directories/buckets) to exist or be in specific states, etc. I'm not going to pretend it's easy.

If you can do it, though, it can be an incredibly valuable tool. There ain't nothing like the real traffic, baby. ;) My favorite feature, which I alluded to earlier, is that you can shadow traffic from a larger production cluster onto a smaller shadow cluster and give it a serious stress test. All sorts of bugs tend to fall out that way. The one thing you can't really catch, even with a good shadow, is interactions with other services - including things like permissions or quotas. But if those are the only things you have to shake out in true production, you're doing well.

Teeing/dark launch/dual write strategies solve most of the issue for databases. Sure you run into concerns when changing the framework that manages that, but that's usually a far smaller surface area than your entire storage layer.

That said, you should have tests anyway.

Tell that to the millisecond of testing in production that makes the MRI fry the patient's brain, to the one that trades one trillion instead of one thousand naked puts, to the nuclear armaggedon launch check that canaries humanity...

>For very, very popular services, a second of being live will exercise more code paths and edge cases than even the most dedicated testing team could ever dream of.

Most of the code we care about is to handle anomalous situations. That AZ going down a week or two is a good example. It's when stuff like that happens that a bunch of code springs to life to keep things running. And indeed, things didn't exactly roll over just fine for us.

> AWS is pretty reliable for the most part

In telecom or traditional mainframes, for example, the compute unit itself was expected to be reliable. Individual elements of AWS are not pretty reliable in that context. Check out the single host EC2 SLA.

However, today most large or even medium scale software assumes unreliable individual elements and has redundancy at the program level. For that purpose, AWS core services are pretty reliable.

Way back I used to work in telecommunications at a place that provided POTS service. They are two very completely different worlds. Software engineers act as if 5 9's is a badge of honor, when really it isn't. When you are responsible for something that people use to dial 911 and can make the difference between life and death a few minutes of downtime doesn't cut it.

Right, and even five nines would be impressive compared to:

AWS will use commercially reasonable efforts to ensure that each individual Amazon EC2 instance (“Single EC2 Instance”) has an Hourly Uptime Percentage of at least 90% of the time in which that Single EC2 Instance is deployed during each clock hour (the “Hourly Commitment”). In the event any Single EC2 Instance does not meet the Hourly Commitment, you will not be charged for that instance hour of Single EC2 Instance usage.

This essentially forces the use of distributed computing for even small businesses.

EC2 is absolutely not meant for this, though. Use an abstraction layer like Heroku if you're going to not understand what you're getting into.

The amount of times I've had to 'advise' small businesses that are somehow running their small business site off a single EC2 instance's ephemeral boot volume is atrocious.

I don’t have any experience with huroku but what most small businesses need is a (perhaps simulated) reliable box on a fast network. As glorious as the paxos based present is, it’s overkill to the point of distraction for most businesses. The whole attraction of the cloud for them is not needing to hire sysadmins. Replacing that requirement with needing a devops team is even worse.

For something heavily used, like the EC2 and load-balancing, perhaps, but I am still experiencing PTSD from my last CloudFormation encounter.

AWS has the advantage of having so many engineers behind the scenes available for firefighting with a culture of pushing more than is reasonable that as a customer, that would sort of disappear. They simply have to occasionally make trade offs between unrealistic development/feature request goals and firefighting whenever the firefighting is needed. This also acts as another form of pressure to work even more to meet timeline goals.

Don't let your developers know that you're expecting them to always be behind, infinitely queued up with work, and constantly in emergency mode and they won't have much time to think about what's really going on and how efficiency is being pushed at the cost of their sanity.

This is indeed surprising. Any time we have slowness issues the usual recommendation would be to throw resources at the problem; increase cpu, add more memory et al. We used to lament that we should spend time debugging the problem and fix the actual issue. We then used to say that probably at places like AWS and the other biggies they'd be following some excellent best practices and we should also strive to reach that level of excellence.

AWS is very big, culturally, on making sure that all the bugginess from shitty code is not shown externally to the customer. Externally it might look like everything is fine to you, but internally AWS is a massive, leaky cargo ship with thousands of engineers running around 24/7 with duct tape and band-aids to plug the leaks.

Yeah there's no way I'd put up with this. Trillion dollar company and still operating like it's amateur hour on many teams it seems.

The org is getting what it wants out of the team; why would they fix it?

As long as you've got meat for the grinder, you can keep making sausage.

Also: the cruelty is the point.

I'd disagree with that.

One doesn't need to be cruel to be callous or oblivious.

One can intentionally remain oblivious, but that doesn't imply that the cruelty is an actual goal.

See also people who happily eat meat (myself included) who are still troubled (as a 5th order concern) by the process.

American capitalism is built on employee heroics, but at least you can get the AWS stamp on your resie, which, let's be honest, will make you the "it" girl of job hunting.

I think you overestimate the appeal of a handful of large companies, taken by itself, on a resume if only because they come with downsides as well.

The level of abuse and mismanagement I put up with before I got older is pretty shocking. Young people have a much higher tolerance for BS, in many cases because they don't have enough experience to know that their workplace is broken.

Or worse, they may internalize it as normal and unreflexively continue breeding the problem. Those olderish managers of young people were young people themselves, once.

Yeah, it's much easier to end up in a malfunctioning workplace than a good one. The number of healthy workplaces where everything seems to just work is shockingly small, and you are actually lucky to end up in ONE through an entire career. And then you can never go back.

This works when the company is hot, but eventually it's not. When the last time you saw HP, or IBM positively on a resume?

Big companies breed big company problems that can leave engineers poorly equipped for the rapid delivery world of startups/scaleups.

"HP" or "IBM" carry a lot more weight on a resume than something like "BF Consulting Ltd"

As someone with one of those two on his resume, I'd say you are not wrong.

That said, it's a bit more nuanced. It still looks good if you can share experiences that prove why it's good.

Ex. "I learned what it's like to be closer to the cutting edge in some respects, but I also learned how that should be secondary to delivering a good product and good customer service due to issues X, Y, Z observed. Also, using technology A is great, but it might not be worth your investment at current stage."

This kind of statement illustrates that you learned multiple things of value, and hopefully avoided bad habits and are pragmatic and worth having. It's possible they can leverage your experience to avoid making mistakes in the next growth stage.

So, yes, it doesn't always look good immediately. It's up to you to prove to a questioner why it was good and hopefully they also agree.

"Big companies breed big company problems that can leave engineers poorly equipped for the rapid delivery world of startups/scaleups. "

Why are big company employees not suited (or trained for?) to startup like environments?

There are several reasons big company experience doesn't translate.

1. Legacy tech, "cool" big companies have the latest stacks but over time these stacks become old and the dev practices atrophy. The biggest cases of this I've seen are engineers unable/unwilling to do their own QA as its "not their job". Or unwilling to adapt to new technology the startup may be using.

2. Politics: Big tech companies have major politics leading to pathological "not our problem" conditions. Efficient and successful startups/scale-ups need to minimize politics, and some employees from big tech companies will have found this to be one of their primary skills.

3. Ambiguity/Timeline constraints/dealing with crap: At a big tech company everyone is expected to be at the top of their game and the company rarely faces deadlines that are not self-imposed. An engineer may expect 100% test coverage, crystal clear product requirements, and no risk of failure. Dealing with sub-optimal conditions is common in startups/high growth companies.

4. Definition of success: A big company may strongly value marginal contributions as prized wins. Shaving ms off of a frequent call can drive real monetary improvements when the company has hundreds of millions of customers. Making engineers marginally more productive has huge benefits when you have 50k+ engineers. Startups often just don't care about these things and simply won't value the skills necessary for this work in most cases.

On the flip side there are great engineers/managers who learned what made the big company successful and how to navigate internal obstacles. These employees are likely to be gold to a startup, but they are also gold to a well capitalized company with vastly more data on just how effective they are than a startup has. Odds are, startups are interviewing/hiring the employees the big companies don't care about - something the hiring firm often implicitly knows.

but what's the point?

Don’t conflate corporate size with engineering competency. It’s counter-intuitive, but oh so common to see this. That’s why so many engineers want to work for start ups!

> Don’t conflate corporate size with engineering competency. It’s counter-intuitive, but oh so common to see this. That’s why so many engineers want to work for start ups!

I thought start ups were all about letting tech debt pile up like there's no tomorrow, since there might literally be no tomorrow for them.

The truth of the matter is that I've never seen anywhere where this isn't the case to some degree. Long term thinking is rare.

I think it's very safe to assume the level of technical rigor in a given undertaking just falls to the minimum required unless there's a very strong force keeping it in a higher state. Maybe places like NASA JPL or Apple manage to float above the minimum because of a really unified and powerful culture, but outside of that I'm thinking it's more or less universal. e.g. the 737 MAX debacle illustrates Boeing's stochastic search for the lower bound of technical rigor when it comes to flight control software quality.

> I think it's very safe to assume the level of technical rigor in a given undertaking just falls to the minimum required unless there's a very strong force keeping it in a higher state.

It really depends on who's setting the tone. If it's an owner or manager taking an active interest, I think your observation is true.

> Maybe places like NASA JPL or Apple manage to float above the minimum because of a really unified and powerful culture, but outside of that I'm thinking it's more or less universal. e.g. the 737 MAX debacle illustrates Boeing's stochastic search for the lower bound of technical rigor when it comes to flight control software quality.

IIRC, Boeing has made a years-long effort to cripple their unionized engineering workforce. I don't remember exactly where I read this, but for a long time they had a very effective, rigorous organization, but management (from McDonnell Douglas. IIRC) make a lot of changes that messed it up.

Bloomberg Quicktake (which is one of the few news outlet Youtube channels I recommend) did a good 20 minutes piece on Boeing's internal cultural demise [1].

[1] https://www.youtube.com/watch?v=EESYomdoeCs

I had an experience at a startup where the team was mostly experienced people who had been at larger companies and didn't tend to cut corners. It was a pretty good balance. We focused on doing the job well but didn't have big company meeting culture, etc.

I wouldn't want to work at a place like you described.

I think this is because Amazon loves to churn teams.

My team, not Amazon, had alot of churn when I joined. Basically a full turn over within a year. We lost alot of institutional knowledge and had to reverse engineer stuff all over the place.

I'm always surprised why industry leads (like Amazon) sometimes treat their products as an amateur treats his/her weekend projects. This is definitely not the worst as I know one of the leading options marketmaker has been using a giant shit mountain of MS Access/Excel VBA code to run their system since the 90s. Last time I heard about it (a few years ago) they are planning to replace that shit mountain with something new but I don't know if it's done now.

It's built into the culture to ship, ship, ship. Shipped code is better than good code, or clean code, or fast code. At my last job, the C-level was fascinated by Amazon success stories. They wanted to achieve the same success, so they urged us to ship, ship, ship. Unfortunately, we were all very seasoned engineers, and we knew the nightmare that would ensue if we purposely piled on the tech debt. The part of this article that refers to "every ticket taking 4 to 5 days regardless of complexity" should be a warning sign to ANYONE attempting to model their startup after Amazon.

The message should be: You are not Amazon, and you will NOT get to Amazon scale by modeling their worst practices.

I think this culture is OK or even good to a company in start-up because you have to be quick. Once it grow into maturity those rules should be abandonned.

There are a few problems with that approach. One is that there is rarely a good time to say, "ok, we've proven that this idea works, let's now go back and do it correctly". It's a constant stream of fixes on a system that "already works". Secondly, telling management that a team was able to go fast previously, but is now going to start being slow so that things can be done correctly, is a quick way to be shown the door. This is evidenced by Amazon and that thousand line Java method that's probably existed for 10 years. Finding that time where you're mature enough to switch gears never seems to happen in practice.

I now advocate a "cut corners, cut scope, but do NOT cut quality" approach. Unit tests do not take much longer to write - but they pay dividends when it's time to refactor. I'm now back in a startup where the code was written with that "just ship it" attitude. The code is so terrible that it can take days to fix a bug. I can rewrite entire features (correctly) in that amount of time.

Shipping tech debt should be compared to the realistic alternative which is never shipping at all. The solution is not to ship slower, but to attract a better team and then retain them. They say the CEO's #1 job is recruiting and this is why. Actually more important is growing your revenue faster, if revenue compounds faster than debt then you're good!

I mean you can also ship a lil slower.

Startup founders and management types are obsessed with optimizing dev time at a day/hour granularity though in my personal experience, so it's a lost cause.

Can you really ship slower?

In the end it’s a winner takes all market, and AWS needs to out-innovate azure to win.

And as customers of AWS we’re all looking forward to the next re:invent for new features, and we’re voting with our money buying huge amounts of AWS services.

Also, as a potential employee, I would always sign with the company that has positive cashflow through customers that pay for shipped value, rather than a company with a great code base, but a bad sales track record.

Apparently it works though. Since both examples are described as "industry leads".

> This is definitely not the worst as I know one of the leading options marketmaker has been using a giant shit mountain of MS Access/Excel VBA code to run their system since the 90s.

Only one? :)

Current AWS engineer here, can confirm. I'm absolutely broken. I'd second the point about the manager and pre-existing conditions making up the job. It's not clear to me if it's endemic, these are big orgs with teams run very differently.

That being said, the on-call sucks. It's really awful, and something I've never seen before. It's also typically the primary cause of team churn for people in my org. This varies, as I've seen other teams stacked with L6 engineers with very little churn (4-8 years of tenure each). This is very much a pit of despair of our own making, but I still haven't figured out how teams like mine get out of it. My own view is that normalization of deviance means that engineers who've only worked at AWS just accept that getting paged many times in a week at awful hours for false positives is OK.

There's certainly a view that the only way to get promoted (which is incredibly difficult) is to create new features or products. You read this of many orgs though, not just AWS. I'm not convinced it's fair to single out AWS. It can be endemic in some teams, and I've certainly worked with engineers who are clearly only focused on shining for promotion.

The worst I've seen was a team go from 8 engineers with > 2+ yrs tenure down to 4 people with < 6 months, over the course of a couple of months. This was for an enormous product. That team had a very tough 4th quarter.

AWS does handle operations failure incredibly well. If you've hopped on a LSE call before, the execution to identify, mitigate and review correction of error (COE) is world class. Doc / design review is also very thorough.

There's ample opportunity to learn a lot during your time at AWS, and many engineers have carved out incredible careers in this place. Just go in with your eyes wide open.

What is LSE?

Quote-Googling produced "Large Scale Event"

"large scale event" or something to that affect - these issues are visible to everyone company-wide

Large scale event, something that has big impact on services.

London School of Economics ;)

I’m sorry you’re having a rough time. Hang in there, and if you move on, good luck!

Thank you, I appreciate you saying this. I don't want to give the impression it's all awful though. We are reasonably well compensated (you do a lot better if you're based in certain countries than others).

There are part of this job that were a lot harder than I could have imagined. The aim of my earlier comment was to make this clear to people considering AWS. When the hiring manager interviews you and mentions there's "an on-call component to this job", realise that it _can_ be severe. The on-call time is also unpaid; it's also very difficult to spend significant time on improving this situation, if that's possible at all. Other comments have done a better job of describing this.

There are parts of the job that are fantastic. You have access to some outstanding engineers (this is also the case at many other companies though). Almost all of the principal engineers I've interacted with have been very generous with their time and knowledge. I thoroughly enjoy the Principals of Amazons talks, and subsequent discussions. I've also had the opportunity to be able to look very deeply at technical problems (this is a direct result of my manager). Having worked at a number of SMEs before, this wouldn't have been the case. You also work on systems being used by so many people (this is mostly wonderful in hindsight), which having worked on products that have evaporated into the ether in the past, is rewarding.

It's not for everyone, and it's certainly not forever at AWS. My guess is I'll walk away with some scars and a much better idea of what I want I don't want to spend my time doing.

You'll learn a lot and cry a lot.

> The on-call time is also unpaid

This is why on-call should never be unpaid. There is no incentive to reduce cost and improve that way.

Sorry to hear about your awful job, is this only in engineering or does it affect professional services like solutions architects etc too?

ProServe and SAs have little-to-no on-call responsibilities, but the general workload issues and mindsets affect those teams as well. Just as an example, resource management (aka staffing) and project scoping are things that AWS Sales is absolutely fucking awful at, and are things in particular that other consulting companies have figured out decades ago, but AWS does nothing to improve the shitty staffing processes because they have essentially just thrown up their hands and think the inefficient, inaccurate, and incredibly stress-inducing process is normal.

Of course, I guess there's no perfect isolation. A friend at Amazon set me up a video call last year with an SA at AWS (Seattle/NW) so I could inquire about working there. It sounded pretty cool but I wasn't sure. I didn't think that AWS consulting was much of a thing, since my experience client-side has been that AWS will advise but not build. Anyway, irrelevant to this thread. Not sure consulting is much better away from AWS as in my world, it's 90% dependent on the client you're with and boy have I had some awful projects.

Based on my experience working on different teams and interfacing with the SA teams (but not actually working as an SA, mind you), being an SA at AWS seems to be one of the better positions in terms of workload, limited on-call responsibilities, etc, but you do need to be very comfortable with constant customer-facing responsibilities (which can sometimes be better than on-call, and sometimes can be worse than on-call) and the team is still dragged down by the general Amazon/AWS culture.

> pit of despair of our own making, but I still haven't figured out how teams like mine get out of it.

I’ve only seen two ways out: 1. Team implodes when everybody leaves, reorg follows making it some other team’s problem 2. Management recognizes it’s a problem, takes it seriously by staffing the team with enough people to sustainably address the tech debt/operational load AND build new features

> We had a lot of little scripts here and there which would solve extremely specific situations but no focus was ever put on in building a general framework or trying to reduce the ticket count.

wow, honestly this is surprising. For me as an end customer I was always impressed with the way the services are engineered. Kudos to people like you for making this happen. But then I was also under the impression that AWS has very good best practices to take care of repeating issues.

Wouldn't interim patch-ups cause stability issues in the long term?

> Wouldn't interim patch-ups cause stability issues in the long term?

No matter what people tell you or put on their marketing blog, it feels like this is the state of play for 99% of software teams. The only time it doesn't end up like this is when you leave time up front to pay down technical debt (like Intel's tick-tock model), and almost no one does that.

my senior once said to me that the world is held together by two things.

Ducttape and shellscripts. This was a while ago, but the older i get the more i tend to agree.

>but I know of other teams who would have no issues in taking in a fresh college grad, making them do work for 6-12 months and then just randomly putting them on PIP.

stack ranking came up in another thread a few days ago and this practice of forced attrition seems like just another way to do the same thing.

I thought it was understood that this kind of structure just incentivizes internecine fighting and politics over and above any imagined positive effects.

Mind-boggling that there still exists upper management that thinks otherwise.

I mean, early Amazon must have hired a lot of ex-Microsofties with stack-ranking PTSD, being based in the same area. You would think they would know better.

It seems like the more employees you have under you, the less you see them as human beings instead of inputs and outputs into a Rube Goldbergian machine you are trying to keep running.

> It seems like the more employees you have under you, the less you see them as human beings instead of inputs and outputs

I mean, there's a department in most companies called "human *resources*" to give away that employees are still resources, just human ones.

well said

> Mind-boggling that there still exists upper management that thinks otherwise.

Far too many Harvard MBAs calling the shots, I guess?

I know one Harvard MBA (also an engineer) and she was a brilliant leader of tech and teams.

>Our on-call was pretty bad (40-60 tickets a week) and there was very little investment being put in to improve it. We had a lot of little scripts here and there which would solve extremely specific situations but no focus was ever put on in building a general framework or trying to reduce the ticket count.

AWS engineer here and I confirm everything you say, but this quote really struck home with me.

The thing I've noticed at Amazon is that not only are the pre-existing conditions awful, but nobody has any interest or willpower to fix it. Everyone will happily vent to you and tell you how awful things are, but any suggestions to fix it or make things more efficient (even if the fix is very simple and requires low effort) will be met with hostility. And I'm not just talking tech issues, but also process/workload issues.

I've worked across multiple teams and there is an "institutional ego" at Amazon where everyone, especially L7s+, think that Amazon is the best/smartest company in the world and have an attitude of "if Amazon, the best company ever, hasn't already figured out a way to solve this problem, then it must be an unsolvable problem and we won't even try". The thing is, a lot of these problems are in no way unique to Amazon and many other companies across the world have already found fantastic solutions to reduce things like on-call load. But adopting those solutions would require admitting that other companies were able to solve something that Amazon hasn't, which would hurt the ego.

This all applies to the very issue being talked about in the OP, too. Even managers will vent to you about how their team goes through 50% attrition every year, and how everyone is overloaded and finding new engineers is hard. They just accept 50% attrition as "something that just happens every year" as if having such a shitty team is normal, and there is no movement at all to fix it.

> The thing I've noticed at Amazon is that not only are the pre-existing conditions awful, but nobody has any interest or willpower to fix it. Everyone will happily vent to you and tell you how awful things are, but any suggestions to fix it or make things more efficient (even if the fix is very simple and requires low effort) will be met with hostility. And I'm not just talking tech issues, but also process/workload issues.

In my case, direct management seems interested in these issues and understand there are problems we need to fix, but ultimately the feature/product launches always make it into the sprint and the larger bug fixes never do. It's very much "actions speak louder than words".

Related to part of what you said. I've worked in a half dozen different industries; I've worked in little companies with a half-dozen employees and globe spanning companies with tens of thousands and employees. They all think that they have special unique problems that nobody else has - 90% of it is the same problem I've seen in other companies in different industries, with different industry specific acronyms and words.

Every damn job, the same basic problems over and over, with the insistence that these problems are specific to the industry and usually to that specific company and that specific product.

Same is true in academic research (though old-timers catch it often). The common pattern is:

* approach “A” was invented in 1970 or so and didn’t work

* “B” extends “A” in multiple ways and now works

* noobs assume “B” invented “A” and treat “B” as the root of modern knowledge. “B” often has more market presence so noobs (without deep understanding) don’t see the relationship to prior attempts.

Examples:

* AlexNet/deep learning/ML in general

* MapReduce/databases/functional programming primitives

* Docker/chroot “Jail” Containers

* Bitcoin/90s coins/blockchains

Ego and ignorance are rarely a great combination :)

The Emperor's new clothes (TENC). Most people don't do history, especially the Dunning-Kruger afflicted.

Docker (Linux containers) is, like jails: awful, leaky isolation pretending to be virtualization. If you want real resource and security containment, use virtualization. Docker is insecure in so many ways; it's like using PHP to write a TLS library.

"NIH" duplication of effort with "IH" bugs.

SMH. Will they ever learn?

Can you share your knowledge or reading materials for how to reduce on-call load?

I’ve worked at a number of big companies but all the problems driving the oncall load seemed, at best, domain specific if not application specific with highly variable fix times and unpredictable occurrence (eg started becoming more of a problem due to unrelated change X). As a result each team has to decide the cost of fixing the pain vs focusing on other things.

If there’s actually best-practices here that help that we’re not already doing, I’d be extremely eager to learn about them. I’m not an Amazon engineer but I’ve been bitten by oncall stuff.

I don't have any particular reading material, and my example of the on-call load at AWS that I'm referring to is probably very basic to most people.

On my team at AWS, leadership has given specific instruction that we do not believe in on-call runbooks or automation to triage issues, for example. Leadership's reasoning for this is that they think runbooks prevent engineers from applying personal judgement, and every single issue should be handled manually by an engineer on an ad-hoc basis.

This leads to a significant amount of on-call time and cognitive load spent doing stuff like verifying the most basic of issues. Even if you have seen the same issue come up for the 1000th time, and even though the previous 999 times it came up the answer was always the same, leadership still insists that the on-call engineer go through a full ad-hoc process of investigating the issue "just to be sure" that this time isn't different.

It's a similar situation with documenting our integration guidance for other teams. Our leadership insists that any documented guidance be vague, and that whenever another team wishes to integrate with our software they must schedule meetings with us to discuss even the most basic of design questions. I'm talking very simple stuff like "should you use the HTTPS endpoint to communicate with our service?" where the answer is "yes" 99.99% of the time, and could easily be included in some documentation. But leadership insists that we spend multiple hours per week in meetings to discuss this just in case that 0.01% design comes up.

There is something to be said for this approach. If the root cause is to be fixed someone needs to look at it in depth rather than running some play book procedure to recover. If you have too many problems though you're beyond the point where that helps. Let's say your software has worked flawlessly for a year, no issues, now an issue pops up, the engineers should definitely spend a lot of time understanding it, understanding why it popped up, fixing it properly and fixing the underlying process/org causes that made it pop up. It should not be "follow some playbook to recover". If issues pop up every week this is unsustainable, you're well beyond the point where stuff can actually be fixed. Automation has its own dangers, it is additional software to maintain, it has its own bugs etc. The right amount of automation makes life better for sure.

>If the root cause is to be fixed someone needs to look at it in depth

The root cause has already been looked at in depth 999 times when the same issue has come up. It's already been RCAed and the fix has been put in the backlog to be implemented sometime next year. In the meantime while we wait for the fix, we will continue to do a full, ad-hoc RCA every time the exact same issue appears, with the exact same results every time, because managers genuinely think it is a valuable way to spend our time.

I understand your point, but the relative utopia of a team you're describing is not really the situation I'm talking about. We have on-call periods where the exact same issue will appear 10-20 times per week, and each and every time it is treated as a completely novel issue with an ad-hoc response, even though we already know beforehand what the root cause is and what the fix is. It's an incredible waste of time and contributes significantly to on-call engineers being overloaded, and yet we continue to do it and then are baffled when all of our engineers leave the team due to being overworked.

There's also nothing excluding runbooks and root cause analyses from existing together, either. In fact, most good runbooks specifically include steps to determine when an RCA is necessary and how to conduct one. There really is no excuse to not use runbooks as much as possible. If over-reliance on runbooks is having a negative impact due to engineers not applying personal judgement, then that is certainly an issue to be addressed, but the answer is almost never to completely abolish runbooks and documentation.

> the exact same issue will appear 10-20 times per week, and each and every time it is treated as a completely novel issue with an ad-hoc response

Yeah, this sounds like a very bad situation where management won't let you do something that reduces ops pain because it isn't the most desirable solution, but they won't let you prioritize the right solution either. The next thing that happens is that on-call folks develop ad-hoc quasi-runbooks and share them amongst a subset of people (or just keep them to themselves to make their own life easier) and those quasi-runbooks become critical to ops, but not documented or shared by everyone. It's pure dysfunction.

This does sound pretty dysfunctional. You'd think that for something that's causing 999 on-calls getting the root cause fixed would be a priority. What I described obviously falls apart when the team has no ability to actually fix issues. Perhaps the original intent was to get those issues fixed but that somehow got lost as the org grows larger.

It is absolutely fascinating how wrong this approach is.

Every single issue that comes up in on-call should be evaluated under the lens of “does fixing this absolutely require human judgement, or can it be automated, ideally by fixing the code in the main system. If it does require human judgement, are there ways to redesign so that is no longer true?”

I'm not the parent but lemme chime in on this topic. It's pretty simple, if you don't build crappy software you won't get a heavy on-call load. You're really asking how to build great software. Build strong teams, with experienced people, follow good practices, reward quality and stability and not features or lines of code, reduce complexity, etc. etc. I've worked on software used by millions of people with a very low problem rate and then I worked on software used by hundreds of people where nothing ever works. Often in the latter the team, through lack of experience or ability, assumes that this is just the way all software is. There's plenty of examples of widely used software systems that are generally quite reliable and well built, and there's plenty of examples of stuff that's garbage, held together by duct tape, works by chance.

This is oversimplification.

One reality is more like "You are handed a system with a heavy on-call load. Make it better."

The other is "The system you built was great with 1e9 load, but now we're heading for 1e12 load. Make it better."

Sometimes throw it away and start again is the right answer

In my experience:

  1) Make a goal:  we should get paged for at most N incidents per week.  That goal should be closer to 0 than 10 IMO.
  2) Track stats on this goal, both in aggregate and broken down by cases you think you can address separately.  Example: tickets from alarms vs tickets filed by people.  Alarms having to do with external dependencies vs alarms caused by your own bugs.  Don't just make this a "it seems like we've had fewer pages lately" thing.  Real numbers.
  3) Review these stats on a graph every week. Someone should have an explanation of why they have spiked, why they haven't dropped, the breakdown of problem type, etc.  There should be congratulations when they drop and a request for plans when they don't.
  4) have management that can communicate upwards to leadership that ops improvement is a priority for your team and that you ultimately won't be able to continue other feature development if you are always mired in ops pain and people are either busy with mundanity or driven to leave the team.
  5) dedicate time in each sprint to working on the most recent identified target from plans made in #3.

This isn't particularly complicated, and typing it out almost sounds like I'm giving you worthless common sense advice, but I think the key here is that multiple levels of your organization need to commit to making this important enough to spend time reviewing it, agree it is a priority, and put actual dedicated dev work into it.

edit: formatted so indented list is readable on mobile. I have no idea how to do this without a code block. HN, please make this easier :)

There is a lot of SRE content about fixing on-call.

My short summary is, fixing on-call is a HUMAN problem, not a technical engineering problem.

Here’s an excellent place to start:

https://monitoring.love/articles/how-to-improve-on-call/

There is a wonderful list of talks and resources there, and those will lead you to yet more concepts and ideas to research.

> If there’s actually best-practices here that help that we’re not already doing...

Unless you have a leadership that recognizes the time sink on-call generates, all the best practices in the world are not going to help. This goes back to leadership culture. If they aren't in the trenches taking the same rotations with the on-call staff and at least simply staying up with them when the calls come in, then they will need metrics they believe to understand the deleterious effects of excessive on-call. This doesn't even begin to touch upon excessive on-call environments strongly signalling ignored technical and/or product debt that drags down new feature implementation and innovation. Leadership that truly comprehends this will allocate significant (20-25%) of time to addressing it, and push back on feature demands.

If you get to that point, then 80% of your struggle is over. Once you have leadership backing bringing the necessary time and resources to bear, reducing on-call is mostly lots of bookkeeping and documentation to identify trends and commonalities to prioritize and fix. A lot of the specific implementation details depend upon what you have available to leverage; ideally put someone in the PM role of tracking details for weekly stand-ups who is a relentless detailed note-taker and follow-upper.

The lack of investment into decreasing on-call pain is a real factor. I work on Oracle cloud (OCI) and at least some of the orgs (VP-down) have figured out that this is something worth focusing on, and the on-call gets better and better as a result. My original team had an average of something like 50 pager-worthy (sev2) events per week until we got moved into a new org that had the right philosophy and we relentlessly drove that down because management realized that engineers made miserable by mundane ops fake-emergencies would eventually get fed up and leave, and that's not what they wanted (afaik, OCI has no such forced attrition). So we got put on a program of relentlessly tracking and categorizing the sev2 counts and committing to improving those numbers over a period of time. 25% of dev sprints were dedicated to improving ops (tools, better alarms, fixing long-backlogged bugs that led to pages), and now that team's ops are pretty easy and they are free to work on new features, which everyone prefers. I've since moved to another team whose ops had already had this optimization done, and I've never experienced a bad week of on call there.

I won't pretend OCI is a panacea (lol google oracle cloud toxic work environment for latest stories) but at least they don't lack this particular piece of wisdom. The sheer number of regions they plan to operate doesn't really allow them to ignore dumb ops problems.

SDE1 here. IMMV of course but on my corner of the org I've seen a bunch of team members raising concerns regarding tech debt for the L7 to shut it down as it got in the way of delivering the features he wanted to deliver.

Also the elephant in the room is how the company relies on trial by fire as a form of performance evaluation, which involves inexperienced SDEs being pushed to deliver alone chunks of major projects in spite of lack of experience or insight.

> The thing I've noticed at Amazon is that not only are the pre-existing conditions awful, but nobody has any interest or willpower to fix it.

Jeff Bezos can afford a private space program because - in part - his Amazon retail model is based on working people until they are mentally and physically broken, while paying them a pittance, discarding them, and moving onto the next group of people. He would rather spend money on crying booths and astroturf campaigns and buying newspapers than change that.

Why would he think about the expendable meat-units in AWS any differently?

Because We are knowledge workers, Not manual labourers!

worked in AWS for a while, but it was 5 years ago.

you’re right that your manager can make or break your experience.

For the first couple of years there it kinda sucked, mostly due to oncall (our ticket queue was at 3000 tickets at some point) and constantly being yelled at when things broke. We would basically only work on sev2s. Having the pager was super-stressful. We “owned” so much cruft/dead projects/experiments that a couple of time we were paged for something that we didn’t know existed.

After I build a little bit of leverage / gathered some political capital I somehow ended up in the position of “team lead” with I guess management’s intent to move me to be a full time manager (after the team’s manager was PIPed).

I made a good case that half the team is going to leave, me included, if we don’t do anything (6 people team).

So what did we do? I introduced a “secondary” oncall. After you were oncall for a week, you were secondary for a week. While secondary you got the chance to try fixing some shit without worrying about being paged every hour. You were also motivated because you just got offcall. People went for annoyances that would generate a lot of busywork or for… fixing the alerting and monitoring (a lot of autocuts due to wrongly setup alerting thresholds or even alerts that should not have been alerts in the first place). After we exhausted the low hanging fruit, we put some effort into automating some tedious task that would take a lot of time but understood and never meant to be done manually at the scale we did them.

Towards the end of the journey we aggressively deprecated/migrated the shit that was not used. By the end of this (took more than one year) we had an empty-ish oncall queue and for the first time in ages people coupd breathe (we now got a sev2 every other week - which in Amazon terms is freaking awesome).

I wish this story had a happy ending. There was close to zero recognition for what happened there and most of the team migrated together, internally, to another opportunity after. I left Amazon 6 months after this migration. From what I hear from the people that stayed there, entropy took over and in another 2 years they were roughly in the same shitty place as far as oncall goes.

I use reducing the rate of incoming tickets as the primary OKR for on-call engineers. Has never failed to reduce that burden over time. Just like you say - I'm convinced it's the only correct strategy to make the problem go away.

if you set a goal but don't empower people I think overall you end up killing morale. IMHO, reducing the rate of the incoming tickets can only be done with investments that are not fire-fighting. If you're doing that I think it's awesome.

Yeah, absolutely - that's the whole point. If the on call expectations aren't serving that goal specifically, we need to fix/change the on call expectations, so it's a moving target over time.

If there's too much firefighting, that means we owe additional support to the team or need to rebalance other work appropriately in the interim.

The opposite is also true. If we don't have too much firefighting or valuable de-risking to do, we can take on more product work.

This is a familiar story. Reducing on call load is actually generating a lot of value by freeing up developers and keeping them stress free. This fact is almost never recognized by product driven managements. Incredibly fucking annoying, considering that fixing such a system is not a trivial task.

I was disappointed to hear that it ultimately didn’t work out. A quiet oncall rotation requires good processes and a somewhat competent dev team.

What is really problematic is that from a management perspective they're doing really great: company is growing and extremely valueable, stock price is doing great, revenue and profits are doing great, and bezos is the richest person in the world.

Management has zero incentive to change any of problems you signal, and probably don't see them as an issue. Probably the opposite, they see this as a winning system, and who can blaim them. AWS practices are often used as examples of best practices: you build it, you run it, 2 pizza teams, api first, etc. Survivorship bias and all, the probably regard the other characterestics of the current system as best practices as well.

> Management has zero incentive to change any of problems you signal, and probably don't see them as an issue.

It sounds like management intentionally created these practices. This is from a recent NY Times article about Amazon warehouse workers, but it wouldn't surprise me if this attitude was applied to all workers in the company to some degree:

https://www.nytimes.com/2021/06/15/us/politics/amazon-wareho...:

> 5. Many of Amazon’s most contentious policies go back to Jeff Bezos’ original vision. Some of the practices that most frustrate employees — the short-term-employment model, with little opportunity for advancement, and the use of technology to hire, monitor and manage workers — come from Jeff Bezos, Amazon’s founder and chief executive.

> He believed that an entrenched work force created a “march to mediocrity,” said David Niekerk, a former long-serving vice president who built the company’s original human resources operations in the warehouses.

> Company data showed that most employees became less eager over time, he said, and Mr. Bezos believed that people were inherently lazy. “What he would say is that our nature as humans is to expend as little energy as possible to get what we want or need,” Mr. Niekerk said. That conviction was embedded throughout the business, from the ease of instant ordering to the pervasive use of data to get the most out of employees.

In many cases software engineers overestimate their own value to the company, often to the point were they'll ape the attitudes of owners and management (e.g. rejecting the the idea of a union out of hand). But the fact of the matter is they're cogs just like warehouse workers, and in many situations competent management will exploit the hell out of them to extract maximum value for the shareholders.

When did humane treatment, job security, and the worker's collective voice leave the picture? Are we to simply praise everything laid out here?

Some time in the 1980s, when people were convinced that if they stabbed themselves in the back, they had a shot at being millionaires.

Honestly ? When communist stopped being a real threat sometimes in 80s.

As horrific as communism was, the average worker in the west probably indirectly benefited from capitalist being scared of something similar happening closer to home, and gave concessions.

After the fall of communism that trend reversed and workers are steadily loosing ground.

As someone who was born in one of those communist countries I am glad that its dead, and I don't want it to ever come back, but It did serve as an example of what can happen if you try to squeeze too much out of people.

It's a pretty interesting take given that Bezos himself is the most entrenched person that Amazon has.

If this was truly the case, humans would have evolved very differently (and it's needs would be substantially smaller.

Also, what about all those people working for little gain?

We'll it might be true from Bezos's perspective, even if it's not true generally. Basically Bezo's goal is to get people working as hard as possible to enrich him and his shareholders. I think in many cases people expend more energy than needed to get what they want or need, but it's almost never in the service of getting some asshole billionaire they've never met a little richer. The energy people get from a novel job or from trying to prove themselves to a new community is short lived.

Also, as another data-point: slaveholders of all eras could not stop complaining about how lazy their slaves were. They expected the slaves to give their all, even though practically nothing was in it for them.

They are doing a great job from a management perspective but it is not good place on average for a developer. Both can be true.

Management only needs to change if all developers go else where.

> Management has zero incentive to change any of problems you signal, and probably don't see them as an issue.

SDE1 here, I might be completely wrong but I'm inclined to believe that upper management is kept practically blind about the real struggles and issues experienced on lower-levels, and this scenario is abused by managers to scapegoat their way out of problems they have to deal with, specially the ones they created. I'm not going to provide examples as they could be easily traced back, but I can say that I've witnessed a L7 overpromissing on a project in spite of callouts from lower levels, and once postponing the delivery of some milestones started to become inevitable then I've started hearing said manager frequently mention "firing offenses" on dailies.

You're completely wrong. Upper management is where the policies come from. They don't push back on URA targets because they either don't care or don't have the backbone to.

If your L7 is doing that, look to change teams. Incompetent people can't be fixed.

Exactly, big company works for Wall street not for their founder, employees and small investors.

I'm in non-AWS and things aren't any better here. It is very much dependent on the manager and team, and I've been optimistic (~3 years now), but it gets harder as time goes on.

> I'm just not 100% sure about the whole PIP scene. Our service was extremely critical and we were extremely understaffed. So I don't think it applied to anyone in our org but I know of other teams who would have no issues in taking in a fresh college grad, making them do work for 6-12 months and then just randomly putting them on PIP.

Our team is over-worked and has a large ticket queue, constant sev-2 pages, understaffed, etc. - and yet they still PIP'd (and then fired) someone last year who didn't deserve it IMO.

Amazon has a lot of money to waste. It is cheaper to hire someone so they can fire them in a few months to keep the rest of the staff scared enough to overwork themselves so they can understaff.

They play a good game.

> Amazon has a lot of money to waste. It is cheaper to hire someone so they can fire them [...]

So... Amazon has a lot of money therefore they have to resort to money-saving practices? Or is the causality the other way around? How do these two sentences fit together?

I don’t think they do, but it’s a way to smear and make all Amazon engineers unemployable by proxy. I already have difficulties finding interviews at good companies because the Amazon name implies IBM level talent instead of Google level talent.

Amazon has a lot money. These practices make them additional money. If they didn't have a lot of money they couldn't afford to hire to fire.

How can they do both? - They waste money when they hire to fire and constantly onboard. - They make money by not staffing enough resources but by using fear of being fired to force overtime.

Maybe the hire to fire costs are justified because the savings they get by understaffing outweights the cost.

It’s a “virtuous cycle.”

Wow that’s a huge wtf. You don’t have enough manpower, and then you take somebody who’s contributing and fire them? Seems totally insane.

Same. My team was down to 4 people and they put the L6 external hire in Focus (the informal coaching plan precursor to PIP) and he left. And we own critical services.

Amazon begged to interview, so i did. My friend worked there, and thought I go call her and find out how things worked there. That is when I discovered this forced termination of 30%. I thought it was 15% , but who's counting. So I gave amazon a choice. 2 year contract forced pay contract, if they fired me for any reason, other than fraud, I would collect the full amount. I done this twice.

It was a defunct mobile company. I had the contracting company put it in the paperwork, to my surprise they signed. cause they have multiple times let go of contractors to make C level "bonus". It was known for it. I got some push back, but they signed it, they needed an engineer who understood mobile routing records.

3 months into the contract, they apparently forgot, and let me and everyone go. Reason 'services no longer needed' aka CEO wanted to make bonus. I said ok, then called the contractor's legal dept. After 3 days, i got a call back. Told him look at page 4 line 30.. he read it. Long silence. 5 minutes of silence. "I never seen a clause like this. I will call you back with an answer." Next day, Both lawyers call, one asked would i be interested in returning. I said "fufill the contract terms". "Your check will be delivered in two weeks." Contracting firm, was getting paid too. they were happy and mad. Happy for $$$, mad that they probably lost a semi big client.

I got my check, had a new contract before the 2 weeks.

Fast forword to Amazon. They read it and said no. I declined. They offered little more, but I said I will not work for Amazon until they fix their corporate envirnoment. My friend worked there, she lasted 4 months more, after moving to Seattle, and finding out the hard way that amazon is pure corporate monster.

> So I gave amazon a choice. 2 year contract forced pay contract, if they fired me for any reason, other than fraud, I would collect the full amount. I done this twice.

As someone who has hired contractors and full time employees many times, this is just insane. I don't care if you are Linus Torvalds himself, as an employer I would never sign a contract that is that lopsided, and to be honest I can't believe that any competent company would either. And given the description you have given of the company that was dumb enough to sign this, they definitely don't sound very competent.

So, I mean I guess congratulations that you got 2 companies to sign this, but I can't think of any decently run company that would ever sign something like that.

I think in the end it's all supply v/s demand. If the company needs you really desperately, they'd bend ANY of their policies to get you.

This was pretty common before the financial crisis, though the typical contract length was a year back then. The idea is that it was a 2 way street- the consultant is committing to your company as much as you are committing to them. And if you are a mega corp, honestly one person's salary for a year isn't really that big of a deal. The cost of hiring and firing a FTE is not much less.

Though the length of it is pretty head scratching. The whole point of using consultants was that you needed a short term amount of specific work done, unless you are talking about engaging large consulting companies to deliver a project.

how many months of work does it take to get the "stamp"? I'm assuming employers would be wary if it's only 6-12 months

If you're brand new to the industry I'd say stay for at least 2 years. I worked on a service which operates an enormous scale and while I'm thankful for the opportunity it takes several months to ramp up (I'd say about 6-8 months for my team). But obviously YMMV.

What I don’t get about Amazon is that AWS customers have way less oncall load, particularly Netflix, whose engineers could afford oncall 24x7 for weeks or months with perfectly normal work-life balance. Why can’t Amazon, the pioneer of cloud computing, achieve the same level of effectiveness?

Maybe I'm misunderstanding you but AWS customer's eningeers have less oncall load because the AWS engineers are doing so much of the difficult and failure-prone work.

That's largely what you're buying when paying for cloud services.

What I meant was that Netflix built their platforms and services on top of AWS services, just like AWS teams. Yet AWS teams have brutal oncalls, while Netflix teams enjoy great work-life balance.

Company culture and values flow from the top

But that’s because the AWS teams are presumably shouldering the burden, then.

I’m sure you’re right for some services, especially infra ones, such as EC2. Some other services, though, should be built on top of EC2, EBS, Lambda, S3, and etc, in which case Netflix and AWS teams use the same infra, yet Netflix internal services require much less oncall

No doubt Netflix is a much better run engineering org.

Its interesting that Netflix built their systems in a way that assumes the underlying platform is unstable. With Chaos Monkey and other systems they made sure things are resilient to flakey behaviour.

I don’t get this. In this market, if you’re good, why take a job with any on-call work? It’s thankless, shitty work. If you can deliver features, that’s good enough to get paid extremely well at other Amazon-scale companies with dedicated SRE teams. What’s making you stay?

Because changing jobs is really fucking difficult.

I'm probably one of the biggest proponents of "quit your job, you deserve better" that you'll ever find, but even I have to admit that finding a new job is ridiculously hard. Even in "this market", even if you're a top engineer, it's still ridiculously hard to even get an interview, let alone get hired.

There is only a limited amount of companies that will pay at the same level as Amazon, and those companies often have months-long interview processes with ridiculous requirements that, even if you are a top engineer, still require a lot of time and effort be set aside to prepare for the specific interview processes that the new company is looking for. And that's to say nothing of the nebulous "culture fit" that is just as likely to prevent you from getting an offer and is completely unaffected by how "good" of an engineer you are.

Almost everyone I know at AWS is interested in switching jobs/companies, but it really is not just something you wake up one morning and decide to do. It's a long, perpetual process that can take up a huge amount of time and effort (stuff you don't have a lot of when you're working on-call at AWS anyway), not to mention has huge implications if you are relying on your job for something like a work visa.

It sounds like the real reason is that the money is worth it.

The money is probably the thing that keeps most people in the job, but it's not like willingness to take a pay cut magically makes jobs fall into your lap, either. It expands the pool of companies you could possibly work for, sure, but everything else above that I said still applies.

There's also rarely a guarantee that the new company you're trying to join is significantly better, and companies intentionally make it difficult to get the inside scoop on work-life balance/on-call responsibilities until after you're hired. Personally, I would love to find a new company to join... but my experience interviewing is that it takes months of effort for a potential pay cut and no guarantee that I won't just end up in another shitty situation and paid less, so I struggle deciding if it's actually worth it.

> not to mention has huge implications if you are relying on your job for something like a work visa.

Isn't it just apply for a new job and as part of the hiring process they handle the paperwork? I know people on work visas that have jumped jobs every two years.

There are many, many companies that will pay at the same level as Amazon and hire you after less than ten hours total of interviews, as long as you think of compensation the correct way: in hourly terms

The answer is 1) many people at Amazon like me have inferior intellects and everyone knows it, so it’s harder to get a better job and 2) there are many services that don’t have dozens of sev2s per day because ultimately you do own what you build. It’s a big reason why Lambda is used to heavily internally now - we cut out a class of operational scaling issues out by design.

Also about a third of Amazon's engineers are on a visa. You can transfer jobs but it is never smooth sailing and full of gotchas.

Current Amazon engineer: it's far and away the most incompetently run bureaucracy with self-defeating dysfunctions forced on the huge number of layers. The recent "leaks" to Business Insider, pointed out to me by coworkers, are exactly what we see.

Everything the parent comment mentioned is exactly what we see. Laughably, our middle manager berates us as incapable noobs, entirely unaware that some of us at least actually have been very valued and trusted and dependable hard core results driven engineers for decades. It's like a failed brainwashing attempt, and it's embarrassing to be associated with such idiocy.

I contrast it with excellent experiences at Sun, Motorola, Apple and others over the past 20+ years, where in some cases I had very high engineering ranks with fabulous results, in very well run and just healthy orgs.

I do believe there's intentional treatment of engineers as fungible assets, because the engineering quality is so shoddy and the business plan prioritizes maintaining system uptime, with no true priority given to tech debt removal, which actually would in short order surface measurable benefits.

> I contrast it with excellent experiences at Sun, Motorola, Apple and others over the past 20+ years, where in some cases I had very high engineering ranks with fabulous results, in very well run and just healthy orgs.

I can understand how junior devs end up in these positions for a while and stay on the ride, but what is making someone with your level of experience you stick to Amazon? Is the compensation that good?

Compensation seems good, but I'm there because I got fooled and plan to quit very soon, when graciously (not screw team-mates) feasible, because it's just a waste of time that could be spent helping positive efforts in healthy orgs.

I have felt that way before, but it’s better just to quit ASAP. It’s the company that’s making your teammates suffer and their also better off just quitting.

I live by the motto "I work for money and appreciation, in that order. If you want loyalty, buy a dog!" It has served me well and removed a LOT of stress. I walked out of one crap-hole with a yelling management style on on week's notice. They told me it was unprofessional, and I laughed in the VP's face and told him he was lucky he got a week.

That is also my motto at work. You know what's unprofessional? Managers screaming at people. The fact that they thought dysfunction like that is OK but one week's notice is "unprofessional" tells you everything you need to know.

> ...I'm there because I got fooled and plan to quit very soon, when graciously (not screw team-mates) feasible, because it's just a waste of time that could be spent helping positive efforts in healthy orgs.

Quit now, and encourage your teammates to do likewise. I understand where you're coming from, but the attitude you have can be a source of exploitation (e.g. using loyalty to peers to keep you while the org screws all of you more).

There is usually on hire cash, that you must return if you don't work for a full year to prevent smart moves like this.

"Unique technical challenges"

Some of the problems you can work on in such an environment are interesting.

Combine that with the fact that you don't see this before joining and that some.manager s might be able to shield some of it you are where you are.

And then throwing hands in the air and leaving is a step, as most other companies have "smaller" problems to solve and "also bad management" ...

Only Apple is on Amazon's tier of technical success, and half of Apple's tech success is from non software.

So on what basis ia Amazon incompetent at software? It sounds like Amazon is results driven, and pretty well-unit-tested code isn't critical for that.

Sure, why test code when you can throw expendable people at it instead?

OP made no mention of software and mentioned Sun and Motorola, so I don't think they made any point specifically about software.

It may also be relevant to point out that Amazon is first and foremost a retailer, unlike the other companies mentioned.

> Amazon is first and foremost a retailer

I feel these days that it's more correct to say Amazon was first and foremost a retailer, but these days they are just UberMegaCorp across so many businesses (Amazon site proper, AWS, WholeFoods, Kindle, Echo/Alexa, movie studios, etc. etc.) that it's hard to say they are foremost a retailer.

I have a similar question, though, in that I've heard lots of nightmare anecdotes about Amazon code quality, but obviously whatever they are doing is working on some level.

There a plenty of things that decouple incompetency in their software org from business success:

1) Amazon has been incredibly successful frontrunning two MASSIVE verticals: online retail, and cloud computing. They were early winners and invested an INSANE amount of money to be #1. That gives them a lot of leeway to fuck up and still be on top. Even if they stopped doing anything at all but maintaining whatever shit show is going on behind the scenes, it'll be a while before they aren't #1 in at least one of those two huge markets.

2) Tech debt doesn't hurt you today, it hurts you tomorrow. You can cut corners and get products out the door fast, but eventually features and products that should take X amount of time, instead take 5X. Given their lead in their units heavily reliant on software, I doubt it would hurt them anytime soon.

3) You can shit on a lot of people for a long time before it hurts hiring when your stock has shot up the way Amazon's has. Money will make people do dumb things.

4) It takes a long time for poor hiring and management practices to incapacitate an organization the size of Amazon. Turn over at the bottom may be high, but I doubt it's the same at the top, at least up until the past few years. When leadership starts turning over more you'll see a lot more business mistakes.

well the good news (for amazon) is that I used to work at a place where I was trying to build a cloud. And management suffered from EXACTLY THE SAME PROBLEMS (though for different structural reasons, there was no "build a feature" incentive). And then we hired someone who used to work on amazon cloud who came with rave reviews, even though he completely failed my interview, which basically tested "do you read the spec?". Then I told him to take notes while I was onboarding him, which he didn't do, and then I quit, because of the management problems.

> the engineering quality is so shoddy

What stuns me is just how well AWS works. I mean, did you ever try to use Azure in 2015? Hell, even GCP was worse in that era. Now, it's on-par, but IMO "more confusing" which beggars belief considering how confusing AWS is.

Yeah it’s surprising that despite the negative comments in this thread, AWS works almost flawlessly for stuff that matters (uptime, reliability etc.). Of course it’s painful to work with sometimes, but so are other cloud vendors. There’s literally zero incentive to change things internally until external KPIs start turning red.

IMHO that's the down side of Amazon's customer obsession. As long as it works for the customer, nothing will change. Regardless of internal benefits. And that system quite obviously works.

Disclaimer: Former logistics Amazonian.

I feel for all of you in this thread that work at Amazon and complain of the poor conditions. There are no shortage of stories that point to a very large problem in that company, for certain. However, I believe the underlying theme, if there is one, is incompetent Management and Leadership.

This is systemic in Technology companies because, to me, the “bean counters” try to run the companies like a basic Manufacturing organization of commodities where line items and people are substitutable things and can be dialed and tweaked at the planning stage.

You nailed it. At the foundation of the dysfunction at most technology companies is the inability of the leaders to understand that software heavy products just can’t be estimated accurately by traditional “bean counting” methods.

Not just 2015. A lot of people, including me, would love for azure to be on par. Unfortunately it isn’t. Not in reliability, features, or even documentation. Hopefully it’ll get there.

GCP is an odd duck. It really wants you to do things its way. But if you are doing a multi-cloud strategy the last thing you want is to go all in on the GCP way. Also they just don’t seem to treat customers as seriously as Amazon and Azure. It isn’t life or death for Google and it shows.

Oh yeah, gcp customer service was atrocious. Took several months to find out that we couldn't do something undocumented but not obviously impossible.

> Everything the parent comment mentioned is exactly what we see.

Current sde1 bluebadge. I also see exactly what the parent comment mentioned. I also saw a few nasty episodes such as L6 managers quitting in less than a year in because they hated the pressure they were subjected to, and a yellowbadge say during a all-hands to a bunch of newly arrivals that he was at the company when they joined and he will still be at the company when they all left. I know of a team where SDE2s and SDE3s felt the need to pull in all-nighters and even work the entire weekend to meet the deadlines that their L7 leadership pulled out of their ass. I myself felt compelled to work close to 12 hours a day and weekends during a period just because my senior manager wanted to shine. And yes, I've already see a fair share of colleagues disappear.

Of course YMMV but I'm baffled why I'm seeing accurate descriptions of what I'm personally experiencing on a daily basis being faced with such a sense of incredulity.

> because the engineering quality is so shoddy and the business plan prioritizes maintaining system uptime,

I've created this throwaway account just to comment on the issue of engineering quality being shoddy.

It's true that engineering quality is shoddy but the problem is caused by Amazon's structure and not by the engineers themselves. For example, each and every single engineer is expected to deliver major tasks alone as part of their evaluation process, and this means that by design we are placed on tasks where we are way over our head and we are still expected to deliver things out of thin air. This sort of trial by fire is sold under the premise of allowing us to deliver consistently at the next level before being considered for a promotion, but in practice if you screw up that's expected to also be held against you.

Former AWS/Alexa engineer. Totally forgot about the whole badge color thing. Can't stand that crap.

Things are very dependent on team and manager and org but I will say that one of the things I'll never miss is our Re:Invent launch. That was horrible. Our team (startup acquisition) basically had a giant retro after the launch to figure out how we could avoid doing that again. Most of the startup members left the team of the company 2 years in. I learned a lot.

What do the badge colors mean?

Length of tenure

Former contractor here...

blue = full-time/permanent employee

white = seasonal/temp employee

yellow = vendors (and DSP employees, if I remember right).

It's more or less their way of tagging you with a yellow star so they know who they can treat like shit. Hah, just kidding, they treat everyone like shit.

This is incorrect: https://blog.aboutamazon.eu/discover-whats-behind-the-amazon....

I'm genuinely not sure what those are. They appear to have been proposed at some Seattle facility: https://www.seattletimes.com/business/amazon/amazons-sleek-n...

I've worked at two different facilities in CA and never once seen those. We still used the white/yellow/blue badge scheme.

Are there any ambassadors in this thread who know what those are?

If you saw any blue badge employees with a yellow/red/purple border around their badge, then that signifies how long they've been at the company.

Saying someone is a yellow badge badge is saying that they are a full time employee with 5-9 years of tenure, most SDEs don't work with contractors so we don't talk about yellow badge contractors all that often.

If you're interested in this sort of thing, the color is actually supposed to be orange, but the color came out yellow.

Honestly, I never ran into many blue badges down in logistics. Mostly all white and yellow badges, with a few exceptions... and most of the time I was in and out so fast I wouldn't have noticed any colored border on their badge. The only time I would've interacted with a blue badge is if there was a problem on-route.

TL;DR DSP drivers don't encounter many blue badges.

The article you linked is from five years ago when the new badges were being initially tested. They have been rolled out for years with color photos. Both styles still work and if you don't go into one of a few big offices you might not have easy access to the newer style.

The yellow stars shouldn't be taken lightly. Don't compare badge colors with what Nazis used for holocaust.

Not speculating as to the reason why mtmail is being downvoted, but for those wondering, the reason the nazi Yellow Badge comparison is almost always off-limits is because there's really only one hard and fast rule as to whether the comparison is valid:

Is an entity imposing an identification system for quickly singling out populations with invisible but immutable characteristics? If so, the Yellow Badge comparison is apt because that's exactly what the nazis did with their badging system.

Amazon identifying people within certain buckets of employment doesn't fit this comparison because opportunities exist (in theory) for people to take other roles, to quit, or to be terminated, making Amazon badges identifiers of a mutable characteristic.

Now, is badging people by employment class a crappy thing to do? Probably. Easy way to enable eyeballing a worker and denying certain perks, for instance. But it's also useful in determining whether a person is qualified to hear certain company-confidential information.

---

tl;dr: There's never a good reason to compel visible identification for an immutable trait, but that's also not what Amazon's badging scheme is, so using the comparison risks cheapening the original sin.

Hope this helps anyone reading. And to any historians who probably know better than I do, kindly fact-check me. I only know enough to be dangerous, not useful.

> I do believe there's intentional treatment of engineers as fungible assets, because the engineering quality is so shoddy and the business plan prioritizes maintaining system uptime, with no true priority given to tech debt removal, which actually would in short order surface measurable benefits.

I mean this how Amazon treats all of its employees. It can't quite lock devs and ops people to a desk and have them piss in bottles, but they would if they could.

(Disclaimer: Was with Amazon for ~7 years a long time ago).

I've been a customer of AWS across multiple startups and I've seen the overall quality of their products continuously degrade which complements your experience.

While they continue to launch new products at a rapid clip you can see small cracks beginning to appear as the products age. A permission issue that's not documented, a cryptic error message etc., They aren't show stoppers on their own but if you use AWS long enough you will be worn down by the cumulative pain.

> I can safely say that it's totally up to your manager and pre-existing conditions which make up the job

Isn't it like that at any job?

This jives with what I observed as an intern some on an AWS team some years ago. The oncall rotations seemed absolutely brutal and the engineers were so busy and stressed out fighting fires that they barely noticed my existence (which I was okay with) and the tech debt kept accumulating between because between new feature launches and firefighting there wasn't much scope for anything else.

My intern project was a fairly no brainer tech debt item that automated a lot of the deployment process and saved our lead engineer several hours a week in babysitting deploys. I resolved to never work on a cloud infra team after that -- while the internship was fine, being a full time engineer seemed absolutely miserable.

Had a relatively similar experience (finally understood what drinking out of a firehose meant), the silver lining of the experience though is it finally dispelled any notion of imposter syndrome when I realized everyone was running around as much of a headless chicken as myself ahah

Very weird if you do get called out do you not take TOIL? and get paid on call allowances? Having this cost exposed will lead to things getting fixed.

This solution works based on knowing developers on CSS the huge mainframe billing system and my own experience as the lead for a smaller BT billing system I worked on.

And TBH a 1-2 kloc method isn't that big I have seen individual SPROCS over 2.5k

Piggy-backing on the top-most-voted comment for those that will only read this one and not go to page 2 or onward.

Drama gets heavily upvoted.

If you keep scrolling you will find many individuals whose experience did NOT match this person (or OP's), who have a good work life balance, solve challenging tasks, and think this is the best job they've ever had.

I'm one of those people.

I've been at Amazon for almost 10 years, worked on a half dozen teams. Sure, there were outliers and a couple bad managers I've worked with, and some folks who left the company on bad terms.

But by and large, the experiences I had in 2 different countries on 2 different continents were mostly positive.

Yes, my job is tough. Yes, the oncall can be challenging sometimes too.

But I've also never stopped growing in the last decade, and have achieved bigger things than I ever thought possible.

Every single product and feature I've shipped have been immensely successful because of Amazon's deeply ingrained "Working backwards from the customer culture". And since that's what drives me - shipping cool stuff and having users on Twitter go crazy for it, I have never been more fulfilled.

Amazon is huge. There are now >60,000 Software Development Engineers worldwide. I'm not in Seattle but I'm in one of the largest offices outside of it.

So I'm sure every single horror story is true. But even so, they are a minute exception, and not most people's rule.

Why is it that amongst all the tech Co's, Amazon seems to have the highest proportion of horror stories? It's almost a meme at this point.

I mean the forced attrition and PIP culture is a known thing (x% of employees must be let go each year). Google/Apple/etc. do not have forced attrition or a known PIP culture.

Bezos is on the record for having said "humans are lazy and we need to give them incentives to quit".

No other Tech CEO has said that.

So perhaps your story is the anomaly?

I find it easy to believe that Amazon is a shit place to work (as a SWE) and that Amazon is a massive company with different bits being different. But I also find it easy to believe that a meme could just be wrong. It’s also a meme that Google has the best engineering and that their products get shut down after a year and that it sucks to be their customer. If everyone at Google is convinced at orientation that Google is some great company with great engineering then that can create a brand outside the company by the force of the size of their workforce. Maybe Amazon just has a different employer brand.

Don’t Netflix also hire relatively many employees also? Perhaps I’m just misremembering. And wasn’t stack ranking and the like quite popular say 20 years ago? It isn’t obvious that the Google or Facebook of Apple or Amazon way is the best way to manage a business. Lots of people online have complaints about each.

And I’m not even particularly put off by your Jeff Bezos quote: it doesn’t need to be a bad thing for someone to have an incentive to quit something they are not suited to as they could then go on to something they are better suited to, and it’s true that a lot of people (even well-paid with alternatives) will spend a long time going to a job they dislike because it is easier to stay with the status quo than to leap.

I still feel like the hire-to-fire stories are directionally right (and so are eg stories about shipping new products being the only way to get promotions at Google leading to warped incentives)

It’s also a meme that Google has the best engineering and that their products get shut down after a year and that it sucks to be their customer

To be fair there is a website dedicated to all the things killed by Google - https://killedbygoogle.com

No other tech co has a website like that, partly because it's incredibly difficult to get promoted at Google so you get a bunch of competing products that get spun up.

https://killedbymicrosoft.info/

You have the cause and effect backwards. Memes motivate people to make websites. Also, other large tech companies have shutdown websites as the meme has spread from Google.

Even in my charitable view of my company, Amazon is INTENSE.

We set extremely ambitious goals, and orgs are only expected to meet 70-80% of them.

It's an environment that indexes extremely high on autonomy and flexibility for each person and each team.

It is without a doubt a sink-or-swim environment. But, by and large, the ability to swim is based on individual merit.

Totally agree. My experience is my own and definitely not reflective of everyone else who works at Amazon. They employ tens of thousands of engineers and many of them are happy. I just wanted to share my story - not trying to generalize this for all of Amazon.

But I wouldn't say it's a minute exception with so much of it being revealed. For every story that's revealed you can assume that there are others in a similar situation who do not publicly share their story.

> If you keep scrolling you will find many individuals whose experience did NOT match this person

So I read this comment some time ago then scrolled quite far down the comments.

I have to report that there were very few people not expressing similar experiences than OP. It seems it is predominately resentment from people claiming to have worked for Amazon all the way down.

It's called Survivorship bias. Before coming here, I too thought the same. That the stories were exaggerated, that those experiences were outliers and a minority. But then I saw my team's L6 SDE get put on Focus within a year of joining and leave. If they had caused Sev1s or broken something critical, then perhaps justified. But no such thing.

This place puts you in the middle of the ocean when you don't know how to swim. You either sink or swim.

> If they had caused Sev1s or broken something critical, then perhaps justified.

That's not the metric for success or failure. It's delivering results.

Except there are 14 LPs and nobody says any one LP is superior to the rest. If you need to consider "Deliver Results" supreme irrespective of anything else, then it needs to be made clear. Just do away with the rest of the LPs.

So many of my FANG friends talk about the horrid code quality and sometimes terrible processes that are in place that I’m always surprised. Considering the amount of gatekeeping

I interviewed with Amazon and Facebook in the Spring/Summer of 2020. The difference was night and day. Amazon felt like I was part of a huge cattle herd. The recruiter contacted me and told me they needed to hire a bunch of people in my area. I was immediately given a course of study, link to common LeetCode questions and told I had a couple of weeks. That fine. I enjoy solving puzzles. Facebook felt VERY different. The recruiter had a conversation about career goals, told me how wonderful it is to work at FB, and then quizzed me on some basic CS concepts. The ensuing rounds for each got even more divergent. The Facebook interviewers constantly made me feel like my success was a "win" for them. It felt like a team! Amazon felt like they were waiting for me to screw up so they could disqualify me. In the end, I dropped out of Amazon because it felt disgusting. And I could tell, if this is how they bring me in, I can't see any reason why it'll magically get any better once I was "in".

My feeling on those who use LeetCode as some sort of indicator. Great. You've found the people that do the brain-teaser puzzles at Cracker-Barrel. I'm one of those people. Some of the best people I've worked with, would fail those tests immediately... yet, they've built scalable, performant enterprise software. I now see those tests as nothing more than a way to reduce the number of applicants.

I interviewed with google, fb, and amazon and fb really felt like the more human interview of the three. Now, do I like the leetcode/system designs interviews of fb? Nope. But definitely the least bad.

The biggest problem I have with those "canned" systems, is that there's no real conversation. I have a REALLY hard time even understanding what's being asked. In a real environment, I can ask probing questions. I have had to do a TON of HackerRank, LeetCode, and Code Katas to get used to how the questions are framed. But, I definitely enjoy solving those types of puzzles.

I had a similar experience at Amazon. About midway through the onsite at Amazon I kind of decided "I don't think I'd like working here and I already have other offers, so my brain will be going into neutral now." and kind of checked out. Felt very much like the interviewers didn't want to be in the room with me and just wanted me to fuck up so they could click "No hire" in the HR app and get out.

Walked out figuring I probably wouldn't get an offer, but given they apparently needed a ton of people maybe they'd down level me and offer (I figured I'd checked out to the point there was no way I'd get that SDE III level offer).

Recruiter actually sent me a very nasty email accusing me of wasting her time, I must be a bad engineer who over sold my experience, blah blah blah. Made it sound like I was blacklist for life. Three weeks later a hiring manager emails me and is like "So I found your interview notes in our internal database, you look like an amazing candidate and I want to fast track your hire onto my team."

I have no idea what is going on Amazon, I mean they built AWS so this must work on some level, but I just have such a bad taste in my mouth from my experience I don't want in on it. Maybe it's working as intended in that they want a process that skews toward hiring certain personality types so having other folks self select out is a win for them. Although I get pinged by them 2x a month and regularly offered fast track hirings based on those notes from that interview I did 14 months ago which is also weird. Which makes me think it's more likely their hiring a mess and they survive via throwing money at the problem and big old RSU offers at candidates.

e: Actually not 14 months, it was pre COVID lockdown, so even further back.

If you bombed your interview amazing candidate could be code for they need someone to hire to fire.

It's that subtle view into what your life might become that makes it so much easier to decline to move on. I'll admit, it's probably easier when I already had a job that paid well and I didn't hate...

A recruiter that couldn't keep it professional... that is EXACTLY how I felt! I get that when you work for FAANG, you're used to the stigma your company carries. And you're used to people salivating over the idea of working for you. But there are some of us out here who already make a decent wage, and have been in the business long enough that we don't see having FAANG on our resume as the most important thing. I'm at the phase of life now where I really am interviewing YOU as much as you're interviewing ME. If I don't like what I see, I can walk away.

Amazon has a hiring status that is basically “probably good to hire, but not a fit for my team”. Other team managers can go check that status and look for candidates that have been through the process already and basically do 1 short interview then hire the person. I think it’s called recycling?

HR and anyone in engineering have no idea who each other are, and never talk. It's not some top to bottom well oiled machine. It's all random and hit or miss.

I had a very similar experience with Amazon and Facebook interviews.

How long ago was this? Its been a while since I talked to either amzn or fb, but their processes were basically the same to me. Your description of amzn was almost exactly what I experienced with fb. It would be nice to hear hiring practices changing at a big tech co though.

Spring/Summer 2020

Amazon was rebranded LeetCode for their "entrance level". FB was writing code in what appeared to be more Microsoft Word-like. (There was no console to check my work). But the conversations with actual humans was where things diverged. The systems design stuff for Amazon was very "we're looking to zing you". Facebook was definitely "hmmm, that's an interesting approach, have you considered....?" or "That sounds like it would work but, what if we...." That type of interviewing really makes one feel like they're part of a team instead of being completely alone. Amazon seemed to thrive on the "you're scared, alone, and you need to build a bloom filter and figure out an efficient hashing algorithm NOW." I realize this is probably not everyone's experience. I am not saying "Amazon sucks, all their interviews are bad" I was simply sharing my experience as it's what led me to remove my candidacy.

For sure. The last time I talked to both FB and AMZN, I came in as a higher level referral for a specific job into both. But like a year or two earlier than yours.

AMZN's was very pleasant, though I think because I was a referral, I was side channeled and would have gone through their normal unpleasant process eventually if it was a good fit (it wasn't).

My FB interview was very similar to your AMZN description. They gave me a CS interview guide, told me to spend a few weeks studying it, and did the algorithm lottery on me. The main reason I declined to move forward was because they couldn't guarantee I'd actually get the role I was referred into and basically none of the process was role-relavant. It felt indistinguishable from the AMZN tech interview.

I interviewed for a Sr. Technical Product Manager role at AWS. This was shortly before Covid happened.

Every interview seemed to go well until the on-site. It literally felt like I was wasting their time. They gave me very bad holier than though vibes. It seemed like every answer I gave was just given a head nod then on to the next question. They asked very basic level PdM interview questions. Things I would expect a first time PdM to know, which surprised me because I was interviewing for a Sr. role.

I assume I didn’t integrate their “principles” enough into my answers because I was rejected and put on a 6 month waiting list to reapply again.

That ended up being the last straw for me. I was already on the fence about working at a BigTechCo based on previous bad experience. I ended up starting my own company. Best decision I could have made.

Wow, https://www.haekka.com/ looks awesome. Security awareness based on culture-building, delivered as a Slack app. This is such a tricky market to frame viable, accessible solutions for. Very cool approach.

The "free training" links at the bottom of the site also caught my eye, and I found they link to long-form one-pagers. So now I'm learning about https://docs.haekka.com/hipaa-training, with https://docs.haekka.com/Free-SOC-2-Training-ee02fd6d1b3044c7..., https://docs.haekka.com/Free-GDPR-Training-6c099e88af2642a59... and https://docs.haekka.com/Free-Security-Training-55722324e2424... queued. A nice touch.

(I really like the site design :) pixel-art FTW)

I got a similar feeling when Amazon once reached out to me.

I got a recruiter email and I wasn't super interested in working there, but was a little curious, so I responded. They wouldn't tell me what project it was or what I would be doing exactly, but said that once I was brought in for a full interview loop I'd sign an NDA and they'd tell me all about it. I said I wouldn't mind having a conversation on the phone just to learn what I could from the hiring manager since I wasn't comfortable spending time on a phone screen if I wasn't even interested in the job or the project. The recruiter set me up with a call with somebody on the team.

So I had a phone conversation with somebody on the team, expecting it would be an informal chat about what they do and what they're looking for, but after about 20 minutes they started asking me design and behavioral questions (i.e. given situation X, tell us what you would do). I was really annoyed because I just wanted to get a feel for what the role entailed and what they were looking for, but they turned into a sort of interrogation to see if I had what it takes to join at all.

I was already starting to look for a new job at the time and was familiar with Amazon's principles, so was sort of prepared for it and I answered all their questions quite well, and they seemed happy. I kept trying to steer it back to things I wanted to know, however, but it was hard.

Overall, I felt like the call was a waste of my time since I didn't learn anything about the job. I guess you could say it saved me time later since I decided not to respond to any future emails that other recruiters sent me.

FWIW, the FB recruiter I spoke with also sent me Leetcode links. But the rest of the process was one of the best interview experiences I had, for a company of this scale. Not sure how well FB takes care of its employees, but they sure take good care of candidates.

> I now see those tests as nothing more than a way to reduce the number of applicants.

More like a way to discriminate neurodiverse people. This is probably illegal, but I have not heard of anyone challenging it. If you create a test that certain people fail, that would otherwise do their job fine, then this is likely testing for protected characteristic and disguised as competence test.

Would you have to prove only that the test discriminates, or also that it was intended to discriminate? The latter sounds nearly impossible.

In the USA, intention is not necessary.

> My feeling on those who use LeetCode as some sort of indicator.

I'm fine if you need me to at least demonstrate _basic_ competency with a few simple Leetcode questions. There are a LOT of candidates who don't actually know what they're doing (either they're new or incompetent). It's okay for you to make sure that I have half a clue of what I'm doing. The problem becomes when Leetcode is used as an actual measurement of competency ceiling.

The best engineers I work with would completely fail at Leetcode challenges because they've found easier, simpler ways to implement something.

For example, anything that lodash provides is Leetcode/HackerRank under the hood (not a dig a lodash, just the type of problems Leetcode tends to ask about). I would expect a senior engineer to be able to replicate any of that code, but I'd first expect them to know that it's not worth their time to replicate that code. Instead, they should find a well tested tool instead.

I worked at a company that hired several people out of Amazon. Amazon is a big company, so there is significant variation from team to team.

However, several of the ex-Amazon employees were clearly scarred from their time working at Amazon. Whenever something went wrong (delays, outages, missed deadlines) one of them went so far as to spend hours or days preparing documents showing how it was actually someone else's fault, not his, because he thought it was necessary to avoid being fired. He said his experience at Amazon was basically one large game of "not it" whenever it came time to assign blame for the latest issue, and everyone had to become very good at blaming someone else. The worst was when he was actual at fault for something, because he would panic and scramble to distract from the issue he caused by raising concerns about everyone else around him. Easily the most toxic person I've worked with in the past few years.

On the other hand, I've known other ex-Amazon people who said their work was nothing like this, so YMMV

That "CYA" response really doesn't feel like "us" to me, at least not the AWS side. I can't speak to the experiences on the store side, nor on the devices side. Over in AWS, we practice the "blameless postmortem" model you'll read about from time to time, and it is very rare to see any professional or personal consequences beyond some light joking come about from "breaking prod"

To that point, I know personally the engineers who triggered several of the AWS outages you will have read about in the news over the last decade and more. Some of them are still with us, some have been promoted since then.

Your experience also matches mine from my time in the advertising org. There's no benefit to placing blame, just figure out what went wrong, how to fix it, and how to prevent something like it from happening again. In fact it was one of the few parts of the culture that I tried to take with me after leaving.

Reminds me of that classic parable about the CEO and the worker who just made a $100,000 error. The worker says, "I assume you'll be firing me now" and the CEO says, "Fire you?! I just spent $100,000 training you!"

Depends on the team. In my team in AWS, if something was your fault, the manager would call you out in front of the entire team. Then they wondered why attrition was so high.

I’m agreeing with the “it’s not us” vibe. I work on a team that has some planet wide scale software pipelines.

I broke it, hard. I’m not a software engineer so I had no idea how to undo what I had done.

People from 3 different teams, two of them experts from other teams who heard my call for help hopped on a call and spent all day helping me fix the issue I made and then some. This was a hard stop on a cortical package pipeline for 24 hours and at no point did I feel like my job was at stake despite it clearly being my fault, and due to me running a command I know I shouldn’t have and cutting corners.

As an example; the individual who caused this outage was not fired for it, even though the final cause was user error.

https://aws.amazon.com/message/41926/

Sometimes COEs are used to punish engineers. Eg. take a look at so called "moustache" COE (which I was part of).

Ex-Amazon here: indeed the level of skills, management, code quality and so on changes a lot across teams.

Unfortunately people outside of Amazon make sweeping generalizations about the company.

Funny to see people ask questions like LSE, lol you can probably figure out if someone on an alt account is a real amazonian or ex amozonian by asking them a few abbreviations.

My experience is that everything this guy mentioned in this post is true and more worse things are true that are not mentioned.

Working at amazon broke me. I had to go on Xanax after working at amazon and after quiting Xanax lost three jobs consequetively.

I had to work so hard to regain my self respect and confidence.

The situation described in this post is no different in Amazon Cape Town or any other place.

Btw the internal dev tooling is absolutely terrible and a pain to work with. The dev tooling is a result of "not invented here" syndrome. Forget about using a good opensource library to do a given task if that library is a google open source library.

It is hard to join amazon, but also to leave it. It operates a bit like a prison gang. Your phone tool icons are your prison gang tattoo's. Like a prison gang you will move up the ranks by doing things, sometimes even taking out members of your own gang. The longer you are part of the gang and the deeper you get...the harder it becomes to leave until it becomes impossible to leave.

My two cents is that until amazon becomes customer obsessed with its "internal customers" aka the employees it is bound to fail.

Similarly, oracle hates anything not built by oracle (especially google). They built their own version of google sheets, built their own outlook plugin to avoid ms entirely, etc. I’m not a dev but would have to call clients and say “please send me a blank google sheet, then I can use it to set up a doc we can use together”.

Hmmm I was just an intern but during my code reviews I was told to use a lot of stuff from Google Guava or something like that?

But I did hate and not really know how to use the internal tooling.

I work at a company that hires developers out of Amazon. We joke about having to deprogram them, but it's not really a joke. They bring a lot of emotional issues and toxic workplace habits with them. Don't get me wrong, the Amazon engineers are fantastic, but there is clearly something unhealthy going on at Amazon.

I left a place that was full of death-marches. Those old habits really can stick with you. It took me months at my new job where I kept asking myself, "why do I still feel stressed? No one is behind me telling me I have to deliver" Because I was so used to that constant stress, I had begun to place those expectations on myself! What's worse, I was frustrated that no one else was on Slack on Saturday at 11am to look at my PRs... I've come down off of that lately. And it's such a relief.

"Deprogram" is the same word I came here to use... I haven't seen it myself, but supposedly it's inevitable that you'll eventually have to have "the talk" with a former Amazon employee who tries to schedule a meeting at 7am

Anecdote. I used to work for Microsoft from the Bay Area and I was up in Seattle once for a business trip when I took an Uber somewhere. I struck up a conversation, "I bet you drive a lot of Microsoft employees around." The driver said, "Yeah. Also Amazon." I asked him if he observed any differences between these two groups. "Yeah," he said. "The Amazon employees are often crying."

> "Yeah," he said. "The Amazon employees are often crying."

Ex-AWS. Can confirm. More often, those were tears of joy from rising stock prices. /s

Current (long tenured, moderately senior) AWS engineer here. I've been at the company long enough it's pretty clear that I'm a "good culture fit", so take what I'm saying with that in mind.

While I absolutely believe that there are pockets of the company that work this way, more because of sheer scale than anything systemic, I have sat in the annual ratings meeting for engineers enough times, in enough organizations within the company, that I am pretty confident that this experience isn't universal.

It sucks that this author had this experience, and I wish they had said which team that was, so that I could use what social cachet I have to steer people clear of it from inside. Nobody should have that experience.

> steer people clear of it from inside

What's really needed here is a way to maintain super-scale in a way that is, shall we say, "eventually morally consistent". Applying selection theory, if you create a pocket of badness wrapped in a function that reliably extracts all the good out of it... well all you'll be left with is a local maximum of even more suffering. Yeow.

Of course, designing and maintaining such social structures seems close to P=NP in complexity...

tell me about a time you delivered awesome results while operating out of a pocket of badness

It’s a thing. The author was not unlucky. It’s actually a thing (I worked in AWS, I have heard this from several managers, some which actually had trouble struggling with how stupid the system is).

You have a team of X engineers and you want to grow. You hire a couple more. The current engineers have 0 incentives to help the new ones. Most people don’t have the chops (technical or emotional) to go up against a whole team.

Come review time, what do you think is going to happen? As a boss will you let go someone who’s been there for 5 years and knows the service inside out or the new guy who seems to be struggling.

Not all people are jerks, and there are good pockets (but mostly the deck is stacked against you when you join).

Also, IMHO Amazon is going to have a really hard time hiring people with the reputation they created for themselves.

> Also, IMHO Amazon is going to have a really hard time hiring people with the reputation they created for themselves.

The majority cares about money and convenience. If people really cared about reputation then Riot Games, Microsoft, Tesla, Facebook, that big ride sharing company whose name slipped my mind, Shell, and lots of other companies, would be struggling to hire.

hmm. Do Microsoft or Tesla pay well? Last time I interviewed at M$ I had to turn down their offer because of how bad the comp was.

I don't know about Microsoft, but Tesla is well known for having terrible compensation.

Does Tesla give out heavily subsidized (or even free) cars to employees?

Had a friend who worked at Mercedes Benz corporate and he could pretty much pick any non-AMG car in rotating 6-month leases. The monthly lease payment for him to lease a SL550 (a six figure convertible) was about the same as leasing a Honda Civic.

Seems other car companies do similar programs for their employees.

Microsoft really only pays comparatively at each end of the spectrum, so for new grads out of college and people who have Wikipedia articles.

For the majority of people in their career (the Seniors and Principals/Staffs) MS pays significantly worse than other FAANG-type companies, often being well under half in terms of stock compensation for example.

The company is well aware of this as it's constantly brought up in all-hands (both within individual departments as well as at the all-company level), and they always respond with "we've done research and we believe we actually do pay comparatively in this market segment".

> "we've done research and we believe we actually do pay comparatively in this market segment".

Sounds like market fixing and likely illegal, however I can imagine poorly paid employee wouldn't be able to afford a lawsuit.

the 'poorly paid' employees can definitely afford a lawsuit. here's the thing though: if you know what the situation is and continue to work there who is to blame?

also, most stories I've heard is that the sub-optimal comp comes bundled with pretty good WLB. So maybe if you were to double the number of hours worked and double the comp M$ would blow other BigTech out of the water. Maybe this is the play (half the money for half the effort) and everyone knows it.

Do you base affordability on what? The type of lawyer average employee can afford will be easily crushed by a top law firm with virtually unlimited funds.

The fact that one can change job shouldn't enable companies to continue like this.

my point was that the people know the deal they are getting. if they still work there they are okay with it.

to play devil’s advocate (from a top lawyer firm): how can one force a private entity to pay a certain wage that is way over what the minimum wage is?

People have no choice if all companies are doing same thing.

The minimum wage should be tied to company revenue to reflect the value workers generate. I mean revenue, not profit, because those companies are masters at hiding profits to avoid tax.

if the compensation is artificially kept low yes, this should be (and is?) illegal. the fact that there are other places nearby that pay really well tells me that, at a minimum, people have options.

lol. “i actually did my reservations and your compensation is not competitive”

>The current engineers have 0 incentives to help the new ones. Most people don’t have the chops (technical or emotional) to go up against a whole team.

This is why companies want to hire rockstars who can be productive in a couple of months without being helped by colleagues. One can't get such rockstars just by leetcode. Maybe, these companies should pay $1M per annum for such rockstars.

you know. being a rockstar is not a solo thing. It’s a team thing. Usually rockstars teams are born out of high functioning teams where everyone is making everyone better.

I'll tell you mine, L7. Seen and experienced personally things the author describes.

Like I said, I don't doubt that they happen, but given that you've also left the company and seem to have hit one of these areas of toxicity yourself, I'm not surprised you'd think so. Who was your last manager? I'm curious if it's anybody I know?

This went from "weird, defensive throwaway" to "possible Amazon HR agent" in 5 seconds.

I think asking for the name of someone's former manager on a public forum without giving any details in return is pretty suspect.

For those who aren't already aware, Amazon has a history of covertly paying its employees to "represent" the company in social media: https://news.ycombinator.com/item?id=26636021

> It sucks that this author had this experience, and I wish they had said which team that was, so that I could use what social cachet I have to steer people clear of it from inside. Nobody should have that experience.

"I have soft power". Suspicious indeed.

No, far from it; I'm not HR, nowhere close, but I am senior enough that people sometimes listen to me. I'd like to make things better for people if I can, and part of that is "knowing where the problems are". Believe me or don't, it's no skin off my ass, but that's the opposite of my goal.

> seem to have hit one of these areas of toxicity yourself

Have you considered that the original poster's experience is actually the norm and that your experience is the one that is the anomaly? I was 1 for 2 in organizations with shitty leadership, and the organization that was run properly had zero open headcount. Everywhere people are hiring into is not one of the "good ones".

Check out the old-fart tool, 85% of the company has been at Amazon for 3 years or less. Do you think that if the normal/average organization/team was a great place to be, that there would be so much attrition?

I have considered it. I don't see the pattern widely, and I'm watching for it. I've seen teams implode because of it, and other toxic patterns, so it's not that they're not there... they just seem to be in the minority. There are teams I won't send friends to work for, for sure.

I've been doing software development for nearly a decade now and I've seen 0 teams implode. Nothing I would call a "toxic pattern" springs to mind either. If you've seen multiple occurrences of both at Amazon and think it's an ok place to work then I think you've just normalized the dysfunction.

Have you visited teamblind.com?

But hasn't Amazon expanded massively in the last 3 years? I'm not sure how meaningful those numbers are unless the employee count is relatively static.

Not gonna dox other people, you can easy figure out my name and reach out with a "real name".

> Who was your last manager? I'm curious if it's anybody I know?

Would you truly ask that? And have someone's personal name be searchable forever? What is the impetus?

This isn't the first time I've read about Amazon's 30% mandatory turnover rate, how it effects new hires mentally when they can't help but think they might be fired at any moment, and how it effects the ones who actually did get fired. This sounds like the definition of a systemic problem.

>steer people clear of that org

Um, shouldn't we fix the org instead!!!

Very glad that I've been denied from Amazon final round twice now. It was great interview practice and nothing more

I didn't reveal my team to maintain some form of anonymity. If you're an L8 or above then you may have some power to do what you said. Otherwise I don't think it'll make a difference.

Think before you join any large company. Nothing here is unique to AWS.

We outside have the illusion that everything is perfect on the inside of these profitable and successful companies, but in most cases it’s generally a lot of broken code and a boiler room. Staff is usually burned out after a few years - why do you think turnover in the tech industry is so high??

The amount of bugs and issues in Facebook advertising is atrocious.

Google is just getting some of their admin stuff right.

The lesson to take away is that even if you have a small side business - perfection never usually makes money.

It’s more important to get an imperfect product to market that’s held together on the back end by spit and strings.

Or as my dad said, (he was an artist) - “I could continue working on this painting for months more, but I have you kids mouths to feed”.

We have more than enough data - be it Glassdoor, stories here, etc. - to say with certainty that this kind of thing is far more common at Amazon. Portraying it as "a thing everywhere" is deeply misleading.

Many companies have teams that can push you hard (I burned out at Microsoft, who in general is fairly good for work/life balance), but none of them make it a core part of their identity. Amazon prides themselves on bare metal optimizations, tracking and micromanaging the smallest time quantums they can - it's why they have a fleet of warehouses that grind picking employees into a broken down mess, it's why their offices have almost no perks, and it's why there are so many stories of their codebase and process being a mess.

Living in Seattle, I'm fortunate enough to have friends who work or have worked in all the big cloud orgs - AWS, Azure, GCP, and Oracle. The ranting, burnout, and raw shitshow quotient of Amazon is off the charts. It's the only company that any personal friends have ever warned me against joining, and ALL my friends that have ever tried to work there eventually gave me that warning.

Not sure I agree. The tone from the top varies quite a lot across big tech companies, and sets the tone for the culture of the teams. Then within that context, a lot depends on your team, your manager, your ability to connect with stakeholders in the company, and the problem you’re working on.

I work at Shopify, and while we have had intense projects, I haven’t experienced anything like what’s described here about Amazon. Shopify, culturally, is quite different than other large tech companies I’ve worked at. It’s opinionated about tooling instead of dozens of things that do the same thing. We emphasize sustainably doing work and not burning people out. After all, why would we want to burn people out when we want to retain them? We emphasize “building for the long term” and to do that we do seem to put people first. One example, a colleague of mine has been out nearly 10 months on parental leave. And nobody has batted an eye.

> One example, a colleague of mine has been out nearly 10 months on parental leave. And nobody has batted an eye.

laughs in Swedish 480 parental days

Seriously though, good to hear sane parental policies are spreading to more and more companies. Are those 10 months paid for by the employer?

> why do you think turnover in the tech industry is so high??

Pretty sure it's mostly driven by raises not being able to catch up with job hopping offer.

It's driven by a lot of things. No only raises.

Just like anything else in the world, its not one-dimensional.

For example, someone might switch jobs because you think its the higher offer elsewhere. But its also the burnout at the old job. Or they did not like the manager, and they also want to work with another tech stack.

The higher salary is the push, but there are other lesser reasons that add into the decision as well.

Amazon has had this reputation for as long as I can remember. Sounds like they treat their employees like shit at every level and in every country. When I was looking for a new job it was not even a consideration to apply.

The only defense of the company seems to be "well maybe you'll get lucky and work for one of the good managers who will treat you with a modicum of respect and they'll stay good as long as you're there" - not exactly the dice roll I'd aim for.

When I was at Amazon the org structure at my org changed about six times in half as many years. I’d say your chances of having the same manager for a while are not great.

I've changed the title to try to make it less linkbaity and more neutral, in accordance with the site guidelines (https://news.ycombinator.com/newsguidelines.html). If anyone can suggest a better title—i.e. more accurate and neutral, using representative language from the article itself—we can change it again.

Usually this sort of ambiguously-sourced riler-upper doesn't make for good HN discussions, but this thread is extraordinarily good, with many informed comments from all sides of the question, so I don't want to downweight it.

I joined Amazon and was soon put on the devlist (i.e. preparatory chopping board) by a new manager.

I only knew that I was at risk of losing my job after 4-ish months, at which point the secret deadline was very close.

That period was extremely stressful and honestly left some scars (that have mostly healed).

I saw friends with families and fresh mortgages get fired instead of me. I was essentially fighting against them (who were unbeknownst to them also on devlist) to showcase my deliverables to leadership and demonstrate I pass the bar.

After that I moved to a team in AWS that I love. Management puts focus on our health, invests in operational improvements, and people work normal hours. Senior SDEs are vocal about taking it easy and taking care of ourselves when we need to.

I've had two job offers from AWS for SA and TAM and I turned both down. The actual position was always lower than what I was originally interviewing for (Senior instead of Staff)

Literally the only company I've seen that does that. It's like a bait and switch. If I'm not actually qualified just say no lol

This might be a dumb question but if it's standard to drop everyone a level (I've heard this too) is it possible to interview for Principal or something where they would drop you back to Staff?

I’d interview for CEO instead and see where that lands me.

Funny but also not a bad approach to life

I think down leveling is quite common, especially at higher levels as the expectations grow quite a bit between senior and staff. Depending on the position, actually filling it with +/-1 of the target level is a win. Interviewing is expensive and if a candidate is good for the role, but perhaps just a bit below the bar, then a recommendation to hire at L-1 tries to keep the candidate and give them a career path.

This is from my perspective at Google, I've never worked at Amazon.

My friend works there and said that virtually everyone gets dropped a rank on the way in. It’s so common that people commonly introduce themselves as “I’m an X but pre-Amazon I was an X+1”.

Sounds like an HR policy to reduce labor costs. As long as Amazon keeps a steady supply of labor behind you and at least some of them don't have other comparable options, they're going to agree to these terms and their hiring teams are going to continue this nefarious practice. Some may agree to it alone in an unconscious sunk cost fallacy.

There really need to be laws with teeth about false advertisement from both employees and employers so we can skip this sort of non-sense false advertising.

If someone was L+1 at a similar-tier company, why would they accept the downlevelling instead of a finding a suitable gig elsewhere? Is Amazon that appealing? Are openings that scarce?

Compared to smaller/less prestigious/non-tech companies, a certain level at Amazon/Google/FB will generally come with a higher expectation for technical competence, as well as more comp -- even an L4, mid level engineer at Google makes like 250k. So someone who's a "Senior Software Engineer" from some rando firm in the Midwest may still be doubling their salary or more.

The extreme version of this is being CTO as a startup of 10 people vs CTO at a company of 10,000.

I'm at the point of my career where it's all about the manager, the hours, the comp, and the kind of work I'll be doing. I no longer care as much about the title or level. In fact if the level is lower, great -- I'll have an even easier time getting high perf scores.

These days, when I negotiate for a new job in Big Tech, I first and foremost maximize base pay. That's [1] immediate (no waiting for vest or anything), [2] durable (carries forward into future jobs), [3] non-variable (doesn't arbitrarily change much year to year while at the same company), and [4] the basis for bonus.

Then, I pay very close attention to who my manager will be. In fact I've once switched companies just to follow a great manager.

After that, I make sure I won't be carrying a pager that can ever "go off" in the middle of the night. That's a non-negotiable for me at this point in my career.

Then I negotiate up-front for a couple of pre-planned multi-week trips in my first two years to get around the bullsh*t of "you don't get real vacation time until you've earned it by staying at the company long enough."

Finally I look at the small print in the non-compete and the benefits. Oh wait, you mean my long-term disability coverage won't actually become effective until I've been with the company for 12 months? Nope; it's effective immediately. Oh, you want 18 months of non-compete? Nope, you're doing what every other company does. 12 months.

I'm finding that I'm in way too high demand among tech employers to put up with anything less. I am well aware that my demands are too much for places like Amazon, which is no skin off my back.

> After that, I make sure I won't be carrying a pager that can ever "go off" in the middle of the night. That's a non-negotiable for me at this point in my career.

I'm currently in undergrad so starting my actual "career" still seems far enough off for me, but serious question how do people actually accept that? Will I have to accept that when I apply for junior/entry level positions? I don't think I'm asking too much if I want to have a full night sleep and a strong work/life divide. I might be a bit young and naive, but I hope I won't get comfortable with living at the beck and call of my employer. People are more than just their individual contribution to lining the pockets of their bosses, no?

It's entirely dependent. This is also partially why "do you have a good manager" is included. I've had a few that absolutely will be in the line of fire first, and not make their team do anything they wouldn't do themselves.

From experience: a good team/manager, when forced to do this, will often freely be first on call, give you a day off after if you had an incident at night or let you sleep in til noon or later without complaint, etc.

If you are European or similar country with better working conditions/employee laws, it will be less painful. IIRC, in US salaried tech employees can effectively have unlimited unpaid overtime as a specific exemption.

Good advice about managers, it sounds like it may take a while to settle down and find a team that fits. It's good that you have been able to find a manager and team that actually works as a, well, team.

> IIRC, in US salaried tech employees can effectively have unlimited unpaid overtime as a specific exemption.

Yeah I see that daily with my dad working from home. He works at a local newspaper (wait those still exist?) doing stuff with maps, datavis and page layout/design. He works probably twice the amount of hours he gets paid for early morning until late at night, and can never take time off even with his measly "paid vacation time" allotment. He somehow manages to trudge along, which is unimaginable to me. Generational difference? Or maybe I just don't have the full picture yet

Unfortunately, you don't always know if on-call will be involved until you start at the job. Often many companies don't even hire you with a particular role in mind, instead you are matched to a team after you're signed on. (Especially for new grads). And sometimes on-call is introduced in an existing role, it can be difficult to refuse, especially for people in more junior roles.

But definitely ask about on-call when interviewing, in case they say they have it you can bail out before you sign.

I ask about it repeatedly to all interviewers to the point that some companies disqualify me. Call it a survivorship bias in reverse. It doesn't always work, amazingly, but it always sends a strong signal.

It depends a lot on the company and specific role. As a junior person there is a higher chance that the position you apply for will require this kind of commitment but not all. Just make sure you ask during the interview process.

> switched companies just to follow a great manager.

This is gold. I'm starting to realize (20 years in) that who you work with, and for, makes the difference. I really need to make this a priority for my next gig.

>Oh, you want 18 months of non-compete?

Is this even enforceable? Maybe if your C level, and they pay you while you wait.

Depending on what state you're in, yes, it may very well be enforceable. If you'd like to test those waters, more power to you. I wish you luck.

What did you do in your career to get to a point where you could ask for those things?

Nothing spectacular, really. After 20 years of working at several FAANGs, you just tend to move up the ranks to senior/staff-level positions. You also build a reputation that can follow you around the industry as people you've worked with also move around.

Once you're pegged for a Google L6/Facebook E6/Microsoft 67/etc. role, there's a lot that those companies are willing to agree to in order to get you on-board vis-a-vis Google L3/Facebook E3/Microsoft 59/etc.

AFAIK, there is no Staff designation. On the IC track, L6 is Senior, L7 is Principal, and L8 is Senior Principal.

Yes whatever is above Senior, I don't have every corporations IC structure memorized

http://levels.fyi is a great reference for converting between levels at different companies. Cuts through a lot of the bullshit recruiters and HR will try to sell you.

Realistically, the expectations for technical competence for a senior engineer are going to be higher at a huge successful tech behemoth like Amazon or Google than most smaller companies (though maybe not tech start ups), especially non-tech ones.

An important lesson I've learned is to ask in the interviews "how is employee performance evaluated?"

If the response is blather about 360 peer review, be wary.

Also scan all available information about if they perform any stack ranking.

I worked at a place where managers would bring in other people just so they could meet their "must fire" quotas but keep their existing teams intact. I left shortly thereafter.

An organization must be able to identify and cope with true "non-performers" but when it turns into a global quota for all teams you're in the hunger-games.

The culture of a place radiates from upper management / C Suite; if they're sociopaths you end up with this sort of organization where certain types of people filter up.

360 peer reviews are such bullshit. I've seen it happen where strong developers on a team of strong developers stagnate at the same level for years while mediocre developers surrounded by terrible developers get promoted. It all comes down to who's evaluating you and who you're in good with. It has nothing at all to do with how good you actually are.

What is 360 peer review?

I believe it is when you review your colleagues, they review you and your manager collates all the reviews to access your performance. Some places allow you to nominate who you want to provide your review, perhaps people you worked closely with the last couple of months etc.

The problem is that this is a "factory worker" mindset applied to creative and IP based work.

When a factory worker makes a chair, company does not profit from it indefinitely.

When a developer creates code he or she gets paid once, but company profits forever.

So even if they fire a developer, they still are profiting the from work they did. Question is, why developers sign IP transfer and royalty waiver in their contracts?

> When a developer creates code he or she gets paid once, but company profits forever.

Forever usually being 1-5 years?

It's a figure of speech, but I've seen 10 year old commits still doing well in production and some written before even git was a thing.

Given how much value this work generates, it's time developers got together and put a pressure on companies to pay fairer and proportionally to profits they are generating.

I'd even opt for a legislation that would make contract clauses about giving up royalties illegal.

Current AWS employee. Technical sales side.

I don't disagree with what OP was saying. I have friends who experienced this. But my experience is no way closer to this. Think of this. The company have million+ people working. It got many companies in it (Acquisitions and independent business units). At least 30% are technical side. Out of it around 50% face this issue who are on development side. Which is still significant. Especially on call. other stuff like URA still applies for everyone. Work life balance is upto manager, team, business unit etc. Can't really generalize. You get to work with great companies and build great experience for your career if you got lucky. If someone doing their job,it's really hard to go things wrong way. Managers are always under hiring pressure and they tend to hire wrong folks most times. The interview process have lot of bias baked in. We miss lot of good folks and hire wrong folks too.

But I agree this toxic culture need to change.

Ground reality in all big tech seems to be pretty much the same? Like 1% of the people hoard 99% of the good work?

[Technically I imply a heavily skewed exponential distribution, maybe not exact numbers].

"million+ people working". Yeah, most of them are filling and taping boxes in warehouses. I think people are discussing software engineers which might only be 20K or so.

I cringe when I see similar comments about Apple. The vast majority of their staff are not highly paid engineers that roam HN. Instead, they are Apple Store staff or phone technicians (is that still a thing in 2021?).

Our technical staff is probably closer to 100k than it is to 20k. It was 20k when I started over 10 years ago.

I did digging on this a few weeks ago, this is correct if you add up SDE, SDM, TPM, Scientist etc roles.

Thank you for the correction!

There are only two instances when you're happy at Amazon - when you start and when you quit. Doing the latter was the best decision I made, perhaps, in my entire career. I can honestly tell that those RSUs and bonuses weren't worth it.

I can assure you that in the other areas of the company (non-SDE roles and not exclusively in India) things are WAY worse.

Avoid it at all costs. You've been warned.

I worked at Amazon years ago (2010-2012) on the retail website search navigation. I didn't like it, but it was nowhere near as bad as this hyperbolic post.

Joining Amazon was the best thing I could have done with my career at that stage in my life. I learned a ton! But the culture does grind you down, especially when your alternatives are much more cushy.

I didn’t get any cushier offers out of undergrad and have worked at Amazon since - do you think that reflects poorly on me? I self harm sometimes because many people think it does…

If you don't desire to seek medical help, the best thing you can do is stop self harming and stop caring about all the possible negatives you are imagining.

I've gotten fat from all the stress of being a working man. I need to stop caring about all of the pitfalls and just live my life.

You're obviously right. I started going to therapy and it helps (and I've stopped) but it still hurts.

Good luck to you my friend. May you find some mental ease soon.

AWS engineer of three years: there are a massive range of experiences at Amazon and AWS. The consistent themes derive from the company values and the rigor with which they're implemented. Customer obsession is followed to a fault where operational sustainability and technical debt will permanently be deprioritized except in exceptional circumstances. Frugality necessitates the least headcount required to keep the lights on. Factor in the MBA-driven climate and you've created the worst situation possible in which to be an engineer in general. There is significant variation based on the quality of management in an organization. That said, good management can only do so much, and fixing a broken culture requires more faith and persistence than any reasonable person could put in. My coworkers and I have a saying: anyone smart would leave our team.

I currently work at Amazon as an engineer(NA), joined around 1 year 5 months ago. My experience has been quite different than the one linked here and commonly talked about on the Blind community so I'm posting here to add a data point.

Maybe I'm just lucky but having first hand experience around my team and the 5 teams surrounding it, things are very manageable. Our oncall is a bit hectic but you're not oncall that frequently for it to bother me personally. I work on projects that have a lot of impact and challenge me technically. The key here is to manage expectations very clearly before the beginning of a new project, this sets you up for success for the next -2-3 months in implementation.

Overall, I feel like I don't overwork myself, my manager cares about me and my mental health and supports me in growing as a professional. The only time I've seen people work a ton of hours is if they slacked off , which then becomes a problem they brought onto themselves. If there is a crunch time that was a result of something out of our hands, my manager is empathetic to that and encourages us to take time off after the project is done.

I'm a happy person to be working at Amazon, under my current manager and I see no reason to change that in the near future for me. Amazon is not for everyone. While my manager cares about me, he also has certain standards in the quality and timely delivery of value from me. Working 9-5 is entirely possible at least in my org, and most people that I know do work 9-5, but those 9-5 hours are going to be busy and you have to prioritize well.

If you are a completely new in the field, want to coast or chill I think there are better places than Amazon to work.

Edit 1: grammar/spelling

Amazon is a loose federation of different teams. Some are fine and some are bad.

My counter-anecdote: I worked at Amazon for two years and it was mostly fine. Normal boring job.

If the majority of stock options vest in years 3 and 4, may I ask why you left after 2 years?

Random guess: new job with higher total compensation, especially big signing bonus and earlier vesting.

You have the strongest negotiation power with other companies at year 2 assuming the stock has done well. Companies need to match/exceed your projected comp 2 years out, and for some reason are more willing to do so when someone else is paying you at a certain level.

At Amazon you may face a cliff in year 5 if you haven't been performing at the 80%+ mark for your level and haven't been promoted in the last 4 years. At the higher levels where advancement is more difficult this increases the incentive to lock in a 4 year compensation package of your 3 & 4 year comp.

I didn’t like working there for normal, pedestrian reasons: dead-end project, mediocre code, difficult-to-use tools. The next job I got was substantially better paid because I got an implicit promotion.

Honestly I've been to offices at Amazon in India, and this was not my impression. I would slack Indian coworkers and not hear back for ages even during normal working hours.

At the end of the day the reality at Amazon is that your manager dictates your experience. If you join a bad team you will have a miserable time. The best work experiences of my life were also at AWS.

I’m going to offer a different perspective here as someone who was lucky enough to get placed on a pretty good team within Prime Video. I just hit the 1 yr mark, fwiw. The engineering bar is top notch, lots of attention to code quality, and management seems to care and listen to us engineers, though we occasionally clash of course. With Amazon, YMMV, so try your best to choose a good team though it’s a bit of a lottery ultimately.

I’ve heard AWS can be super brutal, the consensus seems to avoid it.

I was contacted by an Amazon recruiter some month back. The experience was so unpersonal, I felt like a soulless robot: I just got a link to a HackerRank test and solved some automated puzzles with no one but me and the clock ticking. How I understand coding tests is that there should be a human being that evaluates how you approach a problem, not a fully automated assignment.

That's just the initial filter. You'll eventually get a in person interview. Tbh its pretty standard.

Current AWS engineer here. I've been working here for ~2 years in a pretty critical service, in the Dublin office

Back when I joined the company, you could already find dozens of online reviews talking about a toxic culture and awful management practices. I was about to turn down the offer because of that. 2 years later, I have to admit there have been good and bad moments, but if I had to make the decision again today, knowing everything that I know after all this time, I would 100% still accept the offer.

Amazon improved my life and my career in ways I would have never expected. When I look back, it really seems unbelievable to think that I've improved so much as a software engineer in that period of time. And not only on the technical side, but also on aspects like writing, caring about customers or thinking about operations: you realize that work is much more than coding or shipping new features. Besides, from its culture I've learned processes or ways of thinking that have incredibly helped me even on my personal life, like "one/two way doors", "working backwards" or "mechanisms over good intentions"

I'm not going to pretend that all the other comments are lies and that all of that did not happen. Obviously there's a lot of people with really bad experiences at the company. But what I'm trying to say is that Amazon is a huge company, and you can find both great and awful experiences. If you're considering applying here, or even accepting an offer, don't get discouraged just because here you find mostly bad opinions.

In case it helps, something that convinced me to join when I already had the offer and had to make the decision was talking with my future manager. I directly mentioned that I found reviews about a toxic culture and wanted to know his feeling about it. His answer was simply "Look, I cannot talk about other orgs, or even other teams. What I can say is that in my team we really try to create an inclusive and healthy environment, and that we really care for each other". He could have just lied and said that all the reviews were fake, that the culture was great and that all those problems did not exist, but he was honest and admitted that he could only talk about the areas that he knew about.

To me that was a good reminder that even companies like Amazon are formed by normal people, and that while there will be people that only care about themselves and getting promotions, there are also some that really care about making the company a great place to work

Current Amazon engineer (not AWS, not in NA or India) here. I feel very surprised to see folks in India having such a bad experience working in Amazon, because we're almost completely the opposite here. I feel bad for those who suffered.

I've been working for 2 years here now, never heard about the "intent to fire" thing. Our oncall is not perfect but we're definitely working to improve service stability. Working hours have been worse than before since the start of COVID (working at home makes it easier to overwork), but it's still manageable, and when we were in the office it was mostly a 9-5 (or 10-6) job. Unless you're oncall and get paged, nobody would expect you to think about work from the moment you walk out of the office in the evening.

Like people have said, managers really decide your experience. I've had some bad managers and did internal transfer to improve my experience. There were some projects where I worked with teams located in India, and sometimes I do feel that they're quite different. Some PMs and even leadership there would push really hard to get things done, to the extent that we could feel their pressure. That never happened in other projects we did before, and this post seems to give me some hints on why that happened.

Update: minor edits on grammar

I'm an ex-Amazon software engineer who worked in AWS for a couple years and pretty much everything in this article and comments rings true to me and lines up with my personal experience. I won't say much about the PIPs and the brutal oncall rotations because that has been written about enough already, but I do want to express that working at Amazon was devastating for my mental health.

I was more stressed than I have ever been in my life. I had to start seeing a therapist specifically to help me with the constant anxiety I had about my job. I had to see a doctor to get anti-anxiety medication. I tried all the things people suggest: diet, exercise, therapy, medication, meditation, nothing was helping with the escalating stress and anxiety I had about the job. After a lot of internal debate I quit with nothing else lined up. I just needed to get out.

I took a 30% pay cut to join a new company and I couldn't be happier. Sure every job has its occasional stresses and pain points, and this one is no different, but I'm learning that constant stress and anxiety about your job is not normal and at least for me not worth the money.

I still have a small heart attack every time I hear someone's ringtone go off which is the same ringtone I had set on pong paging.

This article was on the front page for all of 30 minutes before being buried to oblivion.

If the article has more comments than upvotes the algorithm uses that as a signal that a flame war is happening and buries it. If it gets upvotes it'll move back up.

Isn't that almost like modern slavery? Company makes billions, barely pay any tax and employees are exploited to the last drop of sweat. Disgusting company and I have no respect to developers who work there. Why won't developers unionise?

Former Amazon (retail division) mobile engineer here.

A lot of the comments in the OP are true, but possibly a bit overstated.

They definitely expect a lot of engineers to leave after one year with only 5% options. In fact, they'll refuse to give raises if your stock price hikes put your pay "above band," but then they won't make up the missing raise if the stock price tanks.

They do PIP a fixed percentage of the workforce every year, even if they're talented at their jobs-- but it was about 15% when I was there-- and half got to keep their jobs if they ran the gauntlet of hard tasks without a complete mental breakdown.

There's no question Amazon is a difficult employer. But the exact details vary from division to division. It can also be a place where you meet a few very talented engineers and learn a lot. But the good ones rarely stick around. I found that the best managers got quickly hired away for better positions at other companies. I personally had four managers within the span of one year.

They also seemed to have a ridiculous legal policy that drastically limited speaking at conferences, blogging, or personal open source projects. It was far easier to moonlight and get the okay from legal than it was to give a talk at your local Meetup or contribute to Open Source outside of work. I stopped asking for permission or telling anyone.

I also noticed that the corporation seemed to encourage taking on unpaid extra work as a bar raiser or security reviewer, but your division might take no interest in anything you did to help the wider company when performance review time happens. There's a lot of politics and jealousy at play. If your team is working 60, 70+ hours, they won't care that you're producing just as much quality work in 40-50. Managers routinely sacrifice their employees as pawns to gain favor for themselves.

One of the most troubling things wasn't the long hours, busy on-call periods, stack ranking, or PIP policy, but their complete disinterest in firing execs who used abusive behavior. There was one in particular who would scream at our managers and cause some to cry in meetings, but rather than disciplining him and making him change his ways, he was able to transfer to another division where he continued his abuse.

Is it ironic that Amazon uses leetcode for their programming screeners?

That may be the reason the author posted to leetcode.

tin foil hat on: that post is going to get removed.

Maybe in India but I don't recall leetcode being involved at all in my process.

I'm in the US and have been approached by multiple recruiters who explained the the first part of hiring was a leetcode-like online assessment, and a later part of the process was a full day onsite (which I understood to be leetcode-like, or systems design questions, just like other big tech companies). I did not actually go through the process.

They might not be using leetcode.com itself, but they are doing similar things.

Well then the point about leetcode doesn't mean much here.

Yes, most companies do coding assessments, but that doesn't mean it's ironic that the comment was posted on leetcode itself.

My experience with Amazon had no self guided coding assessment, it involved a human guiding you through a problem or multiple problems.

Hit and miss for me with Amazon recruiters. Some have a Hacker Rank and some say I can skip it.

I've interviewed there twice and both times the first step was an automated coding assessment. Not leetcode but very similar. You're given a prompt and the expected function signature, and your code must pass a dozen or so test cases.

But again, the parent comment was it was ironic because it was posted on leetcode. If they're using some other platform there's no irony there, notwithstanding any alanis morissette-directed conversation.

Agreed! Just giving my experience with them for the HN record.

SDE3 here with 5+ years on the retail side. Listen up new SDE1. Here is the score when you get hired at Amazon. You have 2 to 3 years to get promoted to SDE2 or you are gone. SDE2's you are in the same boat but with less risk of a PIP. You'll need to be making a case for SDE3 in 3 to 5 years or you are going to see you comp decrease over time. Once your at SDE3 you now are really Amazon employee instead of a trainee in the eyes of management. You can more easily push back on stupid management decisions at the L7+ level, except hiring, and use your years of history at Amazon as leverage, most managers are new anyway. Most SDE3s however have figured out that the path to Principal is easier by boomeranging. There are too few L7 level projects to go around. If you want to survive this climb then here are some tips.

1. Really own your product. Know it inside and out. Too many devs rely on tribal knowledge and out dated docs to tell them what they own. The code is the truth, read it, poke at it, review previous CR's and SIMs to piece together its history. You will become the master of the truth.

2. Drive your career. Understand what the moving to criteria for your level are and keep notes for how you are achieving them. Focus on taking on work that gives you more evidence that you are on a promo path. Write the documents that make the argument and iterate on them often. You need to actively manage this or it will manage you.

3. Pushback. Many managers at Amazon are just spreadsheet jockeys that are making things up as they go along. If something is stupid or determental to the product or team say so and have evidence for why. Make the argument and build consensus on the team for that argument. You won't win them all but you will erase from your manager's mind that your are a passive code monkey. The more you do this and can demonstrate results the more you'll have the ability to drive your work.

If you want to be spoonfed a career don't work for Amazon.

Being a current engineer at Amazon, reading these stories and comments feels so foreign to me. I've literally never seen any of the toxic behavior & mechanisms that I've read about time and time again. I know that usually either AWS or just Amazon India are the culprits, but still, I'd imagine that I'd have witnessed _something_ down here in Consumer.

The literal worst I've seen so far was an "idea guy" L8 that pushed a number of L6's (including my manager who I followed) to a neighboring org. Other than that, I've really only seen underperforming/overplaced wallflowers "deciding to pursue a career outside of Amazon" outside of personal ambition. The skeptic/naive optimist in my tends to place a lot of these anecdotes into the points of view of these unfortunate bunch, but I realize that's probably just wishful thinking.

Regardless, for now I'm just keeping my head up and trying to grow professionally as much as I can while I'm still lucky enough to have such a positive experience. And of course, I'll follow my manager to the ends of the Earth to keep it that way :^). Hopefully by then I'll have built enough street cred to continue the ritualistic FAGMAN promotional job hop.

Question: is it possible to achieve the heights of human achievement and capability / productivity without an abusive culture?

It’s a legitimate non-ironic question. I ask because I’ve vowed to try to build something great without that and I often wonder if it can be done.

I don’t see many examples. Most high achieving top of their field groups seem abusive and dominated by abusive personalities.

Sorry that the OP has experienced this. I work for AWS (SA), have heard about this but does not seems as prevalent at AWS SDE teams as in Amazon teams. Agreed that there is a huge risk to loose skills and then it is hard to find an exciting job. Another point to have into consideration is how massively underpaid most folks are. It is common to hear stories of folks that move to the competition after carefully preparing for interviews and get good offer. Often AWS managers match or go above. Recently one of my colleagues left to the competition to rejoin AWS 3 months later with a 40k pay raise (same job, same team, same work output from the person). Knowing what i know after working at AWS for several years, be mindful that the company is so big that you might be the one doing crappy or low value work. Moreover, if you decide to take the risk of loosing your skillset, make sure you negotiate well and keep coding on the side.

Current AWS employee. I haven't seen anything remotely like what is described in the OP. It strikes me as hyperbolic and a regurgitation of common posts on Blind.

My work life balance is good. My manager is supportive of me and my org takes mental health seriously. We don't hire to fire; in fact we can't hire enough people to keep up with customer demand.

Amazon is a huuuuge place with many different orgs. Maybe you can get stuck on a bad team in a unsupportive org. But to say every org operates as the OP alleges is simply not true.

I'm sorry they had a bad experience at Amazon. But their experience is in no way indicative of every org as they allege.

Also, I find it somewhat interesting that everyone who leaves Amazon after a negative experience blames URA and interoffice politics. It's almost as if it's never the person's fault they got fired.

How do you explain the high turn over rate that can be easily seen using the old fart tool?

I'm not sure that it's fair to ask a random employee (myself) to explain the employee churn for an employer that has over 1 million employees. That being said, it is largely based on the org you're in.

My org (AWS support) has a high churn rate because it's largely seen as a stepping-stone to other places in AWS. People get hired into my org, learn AWS really well, and then go one of a few different places:

- AWS operations (e.g. SOC, NOC) - Service team SDEs - Technical account managers - Solutions architects - Training and certification

Or they simply get enticed to work for someone who uses AWS. I've worked on literally thousands of support cases. I've gotten really good at diagnosing and fixing things on AWS. And as such, I have companies reaching out to me nearly daily with some job or another.

Similarly, for SDEs, working for a FAANG can open lots of doors for you.

The question can be similarly asked: If Amazon is 'lethal' to your career and health as the OP said, why are there so many boomerangs?

Please define 'many boomerangs' in the context of 1 million employees (FC workers shouldn't count).

And yet, as a random Amazon employee, you feel empowered to dismiss other's cautionary tales as hyperbolic and regurgitation of blind posts?

I worked at Amazon India (Retail, not AWS) for about two years straight out of college and experienced pretty much none of the issues described in OP (maybe with the exception of the delayed vesting*) I had the opportunity to work with some pretty smart people (whom I'm still in touch with), on great projects, with manageable oncall, and a good manager who didn't micromanage. In my time there, not a single person in my 40-50+ org was fired. If anything, the problem at Amazon India is that there are too few senior engineers, as most leave for better opportunities in Seattle or at other companies (since better/more important teams and projects tend to be located in Seattle.)

* Something the original post fails to mention is that this difference is made up for by a fixed joining bonus that is roughly equal in value (or at least it was, for me.)

From one of the posts:

>Bezos is on the record for having said "humans are lazy and we need to give them incentives to quit".

Keeping aside all other considerations for a moment. Isn't his fundamental observation correct?

(If you respond, I'd appreciate if you mention a little background of yourself especially if you have experience running or observing closely atleast a small business, the reason I ask this that I'm generally surrounded by people who are the 'middle' class and software developers who generally overwork and are rarely lazy. I have far less of an idea of how the vast majority of the lower socio-economic strata of the society operate. )

When a company grow bigger, it was have this issue? This is a management problem. The bigger they grown, the more cost for management. Really prefer the Google structure that you have alphabet to manage each product individually. However, Google is too big to be split now. The brand is more valuable compared to the parent company.

Looks like Amazon try to do the same thing as Huawei does. 10% refreshment of Human resources particular on 40+ employees. 30% looks is really crazy, it is hard on development team, which will dramatically affect the code quality.

“Live in New York City once, but leave before it makes you hard. Live in Northern California once, but leave before it makes you soft” [1]

Amazon is the New York equivalent in the analogy. The NYT article in 2015 [2] changed a few things, but not all.

Worked 4+ years in Amazon, or as the OldFart tool would say, was among the 4% oldest employees.

Left to preserve my sanity and to not become toxic, I could feel the negative change permeating my personality. I avoid most ex-colleagues from there, they "are now become the KoolAid".

The toxicity and gaming of the system by old timers is bad. Promotions and upwards-management are well managed by a select few who start charming the next level of approvers from day one. The ones who put their heads down and work will get spat out by the system. There is pressure on managers to "top-grade" the system, i.e. to remove bottom performers, never mind the fact that in a hyper-competitive environment that screens for the brightest, the so called bottom performers are merely relative to the top ones.

It's a system designed to burn and churn - burnout and churn the folks before they complete the 2 year cliff (5% and 15% equity at years one and two).

The system gnaws at your self-worth. It's good if these points come out in the open.

[1]: Mary Schmich, Wear Sunscreen: A Primer for Real Life. http://www.davidpbrown.co.uk/poetry/mary-schmich.html

[2]: Inside Amazon, wrestling big ideas in a bruising place ... https://www.nytimes.com/2015/08/16/technology/inside-amazon-...

I’m guessing another lousy side of this is the toxic HR as mentioned in the post. Let’s say you lasted a year and left. Now you can put Amazon on your resume. However, HR is so toxic that they’ll never give you a good reference if someone calls in. Now all that work and effort was really for nothing.

Edit: I meant if someone calls into HR to verify employment. You run the risk of HR secretly bad mouthing you. Yes, it’s probably illegal, but if they are doing all this other stuff, then would you be confident what they say or don’t say?

HR at a large company basically never gives a good reference if someone calls. They'll acknowledge the dates you worked there.

That makes no sense. HR gives employment confirmation. Individual people you know choose whether to give references.

Amazon HR won't give a reference, period. They'll verify employment dates, and I bet that's handled by a third party. I haven't had a reference check in the past 3 jobs over 5 years.

Do references really work like this? I've always given them the name of someone I worked closely with.

> I meant if someone calls into HR to verify employment. You run the risk of HR secretly bad mouthing you. Yes, it’s probably illegal, but if they are doing all this other stuff, then would you be confident what they say or don’t say?

Employment/salary verification is outsourced to The Work Number (an Equifax product). Nobody at Amazon is involved.

The Chemical Worker's Song (Process Man) https://www.youtube.com/watch?v=GzcGOgxDoEk

Somebody should forward this to Bezos with a question mark.

This is a great example of culture clash. In the US, being cut means nothing, you've always got to be ready for another job search. Only a few workplaces (Govt, places with strong unions, full professors) have any protection against retrenchment or arbitrary firing. But in India, jobs are careers for life, and your employer is part of your personal brand. The cachet of AMZN is great, until you realize they are going to fire 20% of you every year. People will say "they must have fired him for a reason" because they don't understand it's just about metrics.

It's also common in India to solve problems by throwing people at it (and indeed the US if it involves minimum wage workers) rather than difficult tasks like fixing the root cause or improving the process. It's also super hierarchical (I recall watching the manager watching the engineer, who watched the junior engineer, who watched the tech type at the console, telling him what to do), encouraging lots of butt kissing. So overstaffed teams, 1/3 of whom do not much, 1/3 (with replacement) who suck up ferociously to their manager, plenty of drudge work, and fear of the stigma of being fired. Great combination.

> in India, jobs are careers for life, and your employer is part of your personal brand.

This might have been true at some point in the past, but it is increasingly changing. In Bangalore, especially in tech companies, it is not a big deal to get fired. You can easily get offers within a week if you are any good.

Yes it is changing fast, but that doesn't mean your parents understand, or mainstream companies.

It's probably better to understand that these jobs are temporary and always be hunting for your next gig, but it is against the cultural norm, where your uncle works at the same PSU he started at after graduation. It probably doesn't help that every IIT graduate is expected to earn 1 Cr.

> But in India, jobs are careers for life, and your employer is part of your personal brand.

In the last decade, I might have very rarely seen a resume with more than say 10 years of experience at a single place. The average would be somewhere in the range of 1-4 years.

I agree 100%. When we hire in or from India, the turnover on their CVs is incredible. Some people only stay one year, and then leave. How much impact can you really have after one year? It's bizarre, but I try very hard not to discriminate and just chalk it up to "local working culture".

An average software engineer can traverse the entire SDLC of a major revision of a product in less than a year. Perhaps in big tech that would be ~2 years if not less.

Makes sense, people with 10+ years at one place are looking for a new job less often ;-)

You theory does't makes much sense.

Amzon USA - 71% software engineers recommends Amzon to a friend in Glasdoor. 87% Approves of CEO.

Amazon India - 87% software engineers recommends Amazon to a friend in Glasdoor. 94% approves of CEO.

Glassdoor is not reliable

> But in India, jobs are careers for life

But remarks like this with no data whatsoever on the average turn over time of Indian vs US engineers are more reliable than Glassdoor rating of thousands of people?

What are your go-to alternatives? Blind?

> In India, jobs are careers for life, and your employer is part of your personal brand.

This has not been the case for at least the last 10+ years. I cant speak of the 'product' companies such as Microsoft, Google etc. But for anyone working in the WITCH like companies, only way to get your salary changed is to switch companies. People change their jobs every 2 years or so, especially in their first 10 years. After that, it becomes less, probably because WITCHes feast on the young ones more, making more money from them. Openings for 10+ years are very less in these companies. WITCHes hire fresh graduates and pay them very less, hardly enough to survive as a bachelor in a big city like Bengaluru. So the only way to get your salary changed is by finding another job. When a person with 2 years experience change job, they often get double of what they were getting earlier. And every job change from then on comes with a 20-40% increase. Loyalty penalty is a very real thing in these companies.

I don't think anyone in the Indian IT industry thinks jobs are careers for life. I feel like our shelf life is around 40 years of age. Only a few survive the industry after that. Sad thing is there is no social security like in US, so we are completely on our own. I was pleasantly surprised to find out that there were so many senior engineers working in US companies. In WITCH like companies, every single team I have seen are structured mostly with 0-3 years, a few 4-8 years. People with more experience expect more salary. Clients want cheap 'resources'. So, these companies hire mostly the cheaper ones.

> WITCH

I would presume: Wipro, Infosys, TCS, Cognizant, HCL

What are WITCH companies?

W- Wipro I- Infosys T- TCS C- Cognizant H- HCL A- Accenture India.

These companies take up sweatshop jobs from companies in US/europe ( manual testing jobs mostly ).

Below are some pros and cons of joining. ( Speaking only for new joiners at level 12 Associate Software Engineer )

Pros -

1). You won't need any skills to get in, if you can speak english you are already in.

2). Provides employment to a lot of undergraduate students ( talking in hundreds of thousands ) who will be probably jobless due to lack of skills in their field.

3). Work is easy, you will get paid around 400 USD a month ( new comers, which is livable in metro cities )

4). Health insurance is given.

5). Good company to work for if you have absolutely nothing in hand, come from a poor background and have to put food on the table. Have to give props to these companies for that. They have created a lot of job opportunities for the lower middle class people who have undergraduate degrees and cannot find job anywhere else.

6). These companies have been doing great, and will not probably fail in the future.

Cons- 1). Shit work, absolute garbage work. ( Predominantly manual testing QA).

2). NO CAREER GROWTH. If you stay long enough you will have to keep working in these type of companies because no one will hire a guy who has been manual testing for years into dev roles.

3). High attrition rates ( why do you think they hire in hundreds of thousands ? )

4). They call you Associate Software Engineer but there is 0 absolutely 0 engineering job. The title is a joke just to make people at the bottom of the barrel feel good.

5). People who stay in these companies for more than 4-5 years stay here for life because they know they cannot probably get a job anywhere else with the limited skillset.

6). The pay for new joiners have NOT INCREASED SINCE THE BEGINNING OF TIME. Its the same freaking money since they started operating in india !! 400 USD a month since probably 2005-2006. Inflation is a hoax to these companies.

End note.

If you are even a bit ambitious and find yourself working here, please work a bit harder and go to a decent enough company which ships actual products and make you write code.

At some point you need to weigh work life balance, career fulfillment, responsibility(more at a startup) vs a FANG salary. Some people don't mind either working their ass off for a large salary or being a cog in a machine. But for most, the idealization of FANG is not what it's cracked up to be

“ If you're someone who likes your 7 hours of sleep a day, stay away.” how do people function long-term with so little sleep? I need at least 9 hours to not be a useless husk the next day.

People have very different metabolisms and lifestyles. I know people who need less than 5h per night and are completely healthy. I myself feel like shit if I wake up before 9am.

Do they "only" need 5 hours of sleep or do they say they need it? Are they as productive as they could be when fully rested, or will they "sleep when they are dead".

Arianna Huffington also thought she was being greatly productive at no sleep, until her body told her to fuck off and she collapsed in her office from exhaustion.

They only need 5 hours of sleep. Why is it so difficult to understand that not everyone the same ? There are a lot of variability in human.

Why is it so difficult to understand that not everyone the same ?

Touchy. Perhaps you need some more sleep.

Because these people are exceedingly rare. I get 5 hours of sleep a night because I have some pretty bad lifestyle choices. The few weeks where I have a normal sleep schedule I am absolutely a different person.

I mean I can function on 5-hours, I'm able to hold a job and live a life; but my well being could be so much better if I got a normal nights sleep everyday.

How rare?

So you are not the one how can function with 5 hour sleep but doesn't mean other people can't.

Because this kind of reply gets posted every single time sleep deprivation is brought up.

Needing this little sleep is usually a lie or a self-delusion. Perpetuating it has negative consequences for the rest of us. Do some of these unicorns exist? Sure. But they are nowhere near as common as this type of comment suggests. Most of the people in question would be better off with more sleep.

Can they survive on 5 hours? Sure. Most of us can also survive on nothing but pizza. Doing this doesn't make you different, just unhealthy.

How are you sure its a lie ?

Not common? How do you know?

Eating nothing but pizza is not healthy? I disagree, Pizza has carbohydrates, protein, vegetables, vitamin, etc. Not always but it can be healthy.

How can you be sure it's not a lie? What I meant by "lie" is this: let's say you ask someone how much they sleep. Currently society tells us that working as much as possible is a virtue. So they tell you they sleep 4 hours every night. You have no way of verifying this information. For all you know they could be sleeping 8 or 9 hours. But it is very easy and tempting for them to lie for a small boost in social standing.

Just because its easy/tempting to lie doesn't mean they are lying.

Do you have data showing that it's "usually a lie or self-delusion"? Because there's a genetic explanation[1] for why folks need less sleep. The only way I can sleep 9 hours in a night is if I work up a sleep debt.

[1] https://www.nih.gov/news-events/nih-research-matters/gene-id...

It’s incredibly rare though.

That one mutation is rare, but it's an advantageous one in a society of workaholics, so there's probably a sampling bias: we're more likely to encounter folks with that mutation when selecting for high performers. It's also a single mutation -- there may be others that provide a similar, if more moderate, effect.

I personally don't need much more than five hours. As in, even when I'm completely on my own free time and I don't set an alarm I'll go to bed at 24:00 - 1:00 and wake up at about 5:30 - 6:30. My father is the same, he even often sleeps only four hours, I guess it's genetic.

I used to get by on about 5-6 hours. If I slept early, I'd wake up long before my alarm feeling well rested, alert, and ready to go. Something changed when I was around 30, and it took a few years to figure out that I wasn't getting enough sleep. But even so, it's rare that I sleep more than 7 hours.

I can get by on less sleep but I still won’t be much use to anyone working more than a usual hour load. There’s more to life than work and sleep.

I'm "one of those" that actually has the low sleep gene. 5hrs is typical, 6 makes me feel groggy all day. However, this only works without alcohol; add booze and I need 8 hrs. So, I rarely if ever drink any alcohol.

Do they take naps during day time? A half-hour nap can "reboot" one's body. But still, even if they do nap, I'm very jealous.

I started today with 4.5 hours of sleep and I’ll do just great with that. Once a week my body grabs 10-14 hours in a single night to “catch up”.

The vast majority of people would be under performing without even noticing at 4.5 hrs. Yet somehow, everyone thinks they're part of the 1% that has some magic genetic fix for this. Sort of like how 80% of people think they're above average at driving.

The last sentence is brilliant! Sometimes, I feel I am the only person that I know who admits to being a terrible driver. I am so easily distracted by beautiful nature, construction sites, other car crashes, whatever... Thank goodness I never had an accident, but many close calls. As a result, I try to drive as little as possible.

Average is not the same as mode. 99.9% of people (at least) have more arms than the average for instance.

So yes, theoretically 80% may be better at driving than the average, it is just so that the other 20% are even worse at it.

Given the average driver, imagine how bad that other 20% must be.

Unless you're a genetic outlier it is likely that you're doing long-term damage to your health.

like?

High correlation between sleep deprivation and altzimers

Why are you downvoting this? I also know people like this, however I personally need at least 7-8.

The last weeks I have practised to not focus too much on if Ive had a sleep deprived night and that helps and works for a few days.

"Catching up" on sleep is a myth. If you're getting wildly different amounts of sleep night to night (even on a "schedule" like yours) it's a biological/physiological certainly that you're operating at a deficit. Maybe not cognitive, but there is a deficit there whether you realize it or not.

I am not sure about this, i had periods of 5h a sleep days for a week or two and then slept 12 hours for a day and felt very fresh and carried on with 5h a day. Sometimes it's 7-8 hours a day for couple weeks, sometimes it's 5. Feels about equal to me, maybe it has to do with volatile working hours for me, not sure. All I know the worst is hangover days and on holidays I stay in bed for 10hours only to feel unusually tired. Maybe environment is a bigger factor than genetics.

I was under the impression that the idea that one can just oversleep to "catch up" was debunked already, e.g. https://www.scientificamerican.com/article/fact-or-fiction-c...

I do 7 but 7.5 is my sweet spot. I pose the opposite: How do so many on this site need so much sleep?

Some of us have always been like this. Others have been doing it for so long that we don’t even know what it feels to be fully rested anymore either. The mental fog of being continuously sleep deprived is all they remember.

There's a lot of natural variance in this. There's no a priori reason that your 7.5 makes more sense than somebody else's 9. Or a house cat's 12-16. It's like asking whether 5' 8" or 6' 1" is the correct height. You roll the genetic dice and they land where they land.

Used to pull a lot of all-nighters coding in my teens and twenties. Being twice as old now I'll regularly clock 8 hours sleep and doing an all-nighter never crosses my mind.

I knew someone who went years with 5 hours a night. He was notoriously quick to anger and tended to over react to everything. Toward the end of my knowing him he started to take better care of himself and suddenly he was the nicest man I'd ever met. Sleep matters.

This is me, in waves. When I become conscious of my crappy behavior, I take care to course correct. It's doesn't last though and I start to get burned out and stubborn and unpleasant. It's not a thing of pride. It's a known issue that I've only ever avoided while unemployed.

I knew a guy who claimed he only ever slept 3 hours a night. :) There was nothing obviously wrong with him at all. Older guy, too. His job was his passion, though, which I imagine helps quite a bit. I doubt he had to drag himself out of bed ever.

Some people have a weird tendency to talk up how little they sleep like it's some point of valor.

Some people can do 6 hours a sleep a night for a long time and be fine but I don't think there's much support for maintaining health on 5 or fewer for long periods of time.

Sleep is important and people need to find ways to get what they need. Even people with infants need outside support so they can get what they need. It's not something to be embarrassed about.

Companies should be very cognizant of the types of constraints they may be putting on a work-life balance. Those that don't obviously treat employees as inherently disposable.

Another unappreciated aspect of sleep is how much of it one needs depending on their age. As I've gotten older, I've noticed that I need far less sleep. I still try to get 8 hours if I can, but most of the time I get away with 6 or 7 just fine and I get up much earlier in the morning. I think adolescents need more like 9 hours of sleep, but for some reason we (in America anyway) make kids get up earlier than most adults and give them work to do at home so they have less time to get to bed early, giving them all the incentive to stay up later. Adults trivialize sleep, which is funny because in high school and college it seemed agreed upon by everyone that we actually needed that extra hour or two. Teachers and parents would tell us that we were merely "slacking off" if we slept for more than 8 hours and didn't get up at the buttcrack of dawn.

Millennial parents undoubtedly have their own set of problems distinct from past generations, but I hope they've learned by now that their parent's views on sleep don't need to be repeated just as they're not repeating the 9-to-5 butts-in-seats mentality that is no longer a universal.

Some people just don’t need much sleep.

Sure, and the short sleep gene people are brought up here in this thread a bunch.

But that's not < 4 hours a night without naps for long periods of time.

I've been functioning on ~3.5 hours sleep per night for about 5 years now. You get used to it.

You might consider the concept of "Normalization of Deviance": https://en.wikipedia.org/wiki/Normalization_of_deviance

I agree people can get used to almost anything. We've all been living through a pandemic. Society made it through the black plague. But part of "functioning" through bad conditions is losing touch with the the possibility of something better.

Don't you have better days and feel smarter, when you have slept more? It's a night and day difference to me. With little sleep, work is a chore and spare time is unorganized, with enough sleep I handle both better.

Apparently [0] some people are genetically disposed to require much less sleep than most, but no, you can't train yourself to perform normally when sleep deprived.

[0] https://news.ycombinator.com/item?id=27471247

You may get used to it, but that doesn't mean the negative effects disappear. 3.5 hours of sleep is well into the territory of chronic sleep deprivation, regardless of your genetic makeup.

There is no known combination of genes that makes 3.5 hours of sleep acceptable. The "short sleep" genes don't shrink the sleep need window that much.

We obviously can't know your personal circumstances, but I would caution that according to everything we know you are likely to pay a price for this chronic sleep deprivation, especially if it continues.

Me too. I need some 6-7 hours in week days and 8 hours in weekends. I'd trade a few years at the end of my life (hope I don't die soon) for 2-2.5 hour reduction of sleep plus still keeping my sanity and productivity. It would be a good trade.

I usually average 4.5 hours a night. I am still living, but it is not a happy life. I slept 8 hours one night, felt like a new person the next morning. But I fall back into the habit again.

Just doing mundane things like decluttering and dishes at night helps relax the brain and wire it into the sleeping habit loop.

Don't start anything major after 6, and definitely not "squeeze in" a movie or a show before midnight. That's how it happens.

It's surprising how fast the time lapses if you do nothing but muck around the house getting ready for bed.

Obviously everyones situation is different, but if you can manage more sleep you'll likely find the rest of your day is overall more productive despite being shorter.

I'm a parent to two small children. Sleep is incredibly important but I think I probably have another year at least of seven hours interrupted at least once as my normal.

It sucks.

It was an amazing feeling when the interrupted nights stopped, after roughly 9 years over three kids.

Youngest is 2.5. Oldest quit interrupting on a regular basis when she was 3.5. Hoping youngest will follow trend but I have no expectations at this point.

I was the same then I had kids and now I probably get 6-7 hours per night, and they're fragmented. Your body adjusts. You use caffeine more effectively. One big learning for me is that i blamed a lot of being tired on lack of sleep, when in reality there are other causes (ie a big or greasy mid-day meal, sugar/caffeine crashes, etc).

That said, i'm sure it has long term health consequences though.

Having worked in an environment where there was a constant onslaught of customer issues. I can tell you that even though I was offered more than 7 hours of sleep, my days were absolutely terrible. Fire-fighting all day, and never getting to work on the features that we promised to deliver led to me lying in bed awake. My mental health was extremely fragile. My physical health was worse. The promise of "Work/Life Balance" needs to be clarified. Does the "work" part bleed into your "life" part? Does the job make it possible to truly disconnect? Some of that is based on personality (I aim to provide value, and am truly jazzed when I know people are pleased with my performance), but some of that is based on what your daily grind looks like. If you are constantly dealing with a nightmare for even 8 hours of your day, it doesn't matter what the rest of your day is like.

Not in India, but a good opportunity to share my experience interviewing for a Software Engineer role with a team in AWS that was part of a recent acquisition (CloudEndure):

After the interview was scheduled, I had some questions but up to this point, I didn't have any human interaction with the interviewer. It took almost two weeks to schedule a call with my recruiter (mostly because she wasn't great at responding to emails). During that, she explained that this specific team (CloudEndure) had a different hiring process: I was to have two 90 (!) minutes interviews on separate days, after which it'll be decided whether they'd like to continue my interview process within that team, recycle me with another team or reject me. The content of each interview, she told me, was to include Amazon Leadership Principles, algorithms/data structures and possibly some system design (the main SDI interview is only in a later stage).

While I was preparing for the LP principles and the more standard parts of the interview, another Amazon recruiter contacted me and scheduled an interview to another group. Few days later he told me that since I already had an interview with Cloudendure schedule, and sinc "all interviews in amazon are uniform and have the same 45 minutes format", he'll cancel the interview that he scheduled and we can talk after my interview with Cloudendure. I reached back and asked whether anything has changed regarding Cloudendure's interview format and he apologized, explaining that he didn't knew they had a different format.

The interview itself: as the interview started I learned that the guy who was interviewing me wasn't the guy I thought was going to interview me but someone from his team who filled in for him (later I learned that day US-East had a major outage which I guess was the reason for this). The 2nd thing he mentioned was "we won't do any leadership principles on this interview things". At that point I realized I just spend few days working on my stories, linking them back with each LP, but I let it pass.

He dedicated 30-40 min to explain what the team is doing, then 5-10 more minutes where I went over my experience with him.

Then came the technical part:

The algorithmic problem was rather easy and I was familiar with it so I let him know of the latter, allowing him to choose whether he want to hear the gist and switch to another question or let me solve it as if I never saw it before. He chose neither and instead questioned me as to who told me about this question, which was an awkward way to ask where I met this problem, regardless of the problem itself which is rather common (easy LC question). As per his instruction, I continued to solve that question, where the tricky part are the possible inputs for parsing a string, so I went over all the edge cases prior to writing any code, proceeded by explaining my idea of how I'd solve it, then coding it while explaining what I was doing, and finally traced an example input by hand.

The interviewer then proceeded to the 2nd question which was more vague and consisted of a system given as a synchronous single machine, single threaded black box, on top of which I had to implement undo/redo. I won't go into all the details here but I went from a naive space inefficient solution to an optimal solution, making sure I don't cause any infinite feedback loops.

The interviewer and I were discussing the possible solutions through out the 2nd question. He did provide me with hints seemed happy with my solution and by the end asked me if I could allocate few more minutes where we continued discussing different designs.

The interview in total took 2 hours and 5 minutes (!!), 35 minutes over time and I thought I did great, the interviewer seemed to like my approach to problem solving.

Two days later I got an email that they "have decided to continue with other candidates". I was confused since the recruiter clearly mentioned that such decision was to be taken only after the 2nd interview, and so I emailed her, asking to schedule a call sometime that week. She didn't answer my (two) emails nor my two phone calls.

I contacted the other group recruiter too. He answered promptly and told me that he'll try to check if he can see whether he can schedule another interview for me, but few days later he apologized, saying that he can't nor does he have access to anything within the team that I was interviewing with.

I was very frustrated with that experience, most of all, by not knowing why I failed the interview. I have interviewed successfully and unsuccessfully with FAANG in the past, but never got zero feedback and complete ghosting from the recruiter.

"The work here isn't technically challenging at all"

Why isnt the work technically changing? What is this work likely to be?

What sorts of CS work is the OP hoping for?

AWS employee not in India, less than a year in. I haven’t seen anything remotely like this. So far it’s the dream job. I’ve been the one learning to mature, adjust, and up my game, no the job failing me.

But everywhere is different. If any job is I packing your health and wellness, mental or physical. Get out while you can. Life is short! No matter what you do, there is work that will treat you well and value you.

I read a telling point about Jeff Bezos yesterday, he believes people, all people, are inherently lazy and will work to avoid work. The description seemed to be trying to say, without saying, he's an Ayn Rand disciple, meaning he drunk the Kool-Aid to believe he's a John Gault or an Roark and we're just the peons preventing him from greatness and his destiny.

Jeff Bezos wants to pay the least for the most amount of work, and the workers want the most money for the least amount of work. It sounds like both parties should meet themselves half-way as opposed to living in what is effectively modern-day slavery. Suggesting such a thing, however, is "radical" and "communist", and essentially one step away from a dystopian fascism hellscape something something ANTIFA.

This isn't specific to Bezos, and it's not specific to any particular class or type of "worker." If Bezos could double his money today with zero work he'd certainly do it. The family business down the road would cut every employees' salary in half tomorrow if it could.

It's just the free market, nothing particularly interesting or scary about it. Everyone wants to get the absolute best deal for themselves, and some are more successful than others.

There are a LOT of small business owners who choose to pay their employees more than they “have” to because they genuinely want to. There are a LOT of entrepreneurs who see “making my company a great place to work” as one of the core values of their job.

I don’t think this is so common when you were talking about mega companies, in part because the work of operating a mega company is a lot less fun than a smaller company, and so you have this selection pressure where (a) people who pursue that path are more likely to value wealth and growth over quality of life, and (b) ruthlessness seems to usually help companies compete and win in the market. Thus the biggest and most famous companies of the world are more likely to be focused on cutthroat efficiency and, as a result, miserable places to work.

But that’s no more a feature of capitalism than cancer is a feature of DNA. It’s a pervasive malfunction, but I believe it’s treatable, particularly through aggressive anti-trust and wealth taxes.

Remember that the vast majority of capitalism is little businesses like your local veterinarian or florist, not FAANG.

>Remember that the vast majority of capitalism is little businesses like your local veterinarian or florist, not FAANG.

That is the vast minority. Corporations dictate much of the business in the states and the world. It's really easy to see this without resorting to statistics.

What is the ratio of your friends who work for corporations vs. the amount that own/work for small businesses? The anecdotal percentage here is a good indicator of the real percentage of economic output produced by corporations vs. small businesses.

You will find that as how most of your friends direct their own economic output in service of corporations so does most of America.

I believe it, I remember reading an article saying that there were 3 times as many small businesses in the 70s than there are today.

It makes sense. There are many requirements and licenses for to start hair salons, barbers, wanting to garden, etc the system is set up to support the established elite today. At least IMO.

This is a very myopic way of looking at things. There are plenty of family businesses that aren't only interested in capturing an ever greater share of profits.

Yes absolutely -- in many places, like Taiwan, Korea, Japan, Germany, Austria, France, Netherlands, etc.

Nah. That's not "the free market". And that's not how most employers are. Wanting extra useless billions even if it immiserates others isn't commerce, it's sociopathy.

Even sticking with industrial titans, look at Henry Ford, who insisted on paying his workers a living wage. That was revolutionary at the time, and put America on the road to having a significant middle class.

> Even sticking with industrial titans, look at Henry Ford, who insisted on paying his workers a living wage. That was revolutionary at the time, and put America on the road to having a significant middle class.

Henry Ford didn't pay a "living wage" (I have no idea if the amount Ford paid was actually technically a living wage or not, but just using your terminology) out of the goodness of his heart - he did so in order to combat extremely high turnover. It actually cost less money to pay his workers more and spend less money on hiring and training new staff.

Sure, and that's a lesson that Bezos failed to learn. There are a lot of people who consider it ideal to treat workers terribly. It's a common ethos; "The cruelty is the point." Which is why it was revolutionary for Ford to say, "Wait, what if we treated people well?"

That's lesson that the car industry sadly lost track of over the years too. A good example is the NUMMI plant, a joint venture where Toyota tried to teach GM its superior ways. They took GM's worst plant and made it a top performer by treating workers with respect. This American Life did a moving story about it a few years back: https://www.thisamericanlife.org/561/nummi-2015

I understand Ford asked "what if we treated people well?", but he did it in the context of "useless extra billions" (to use your language) - he paid people a little bit better than his competitors in order to enable a competitive advantage which earned him more money. If treating them worse would have earned him more money, he would have done that instead. (n.b. that the increased wage also came with a bunch of moralistic lifestyle strings attached that were rigorously enforced.)

My point is that there exist businesses and business owners who aren't solely motivated by increasing their share of the profits (even though our system strongly encourages this) - I just don't think Henry Ford just is a good example of that.

Would you please cite your sources for Ford's motivations being solely pecuniary? I am not trying to set up him up as an example of an amazing do-gooder; he was in many ways awful. But I don't think his biography fits with the notion that he was only in it for the money.

[1] https://www.forbes.com/sites/timworstall/2012/03/04/the-stor...

I actually agree with you that as a person he wasn't "only in it for the money" - I am only arguing that his wage-setting policies were purely about the money.

That is nothing like a reliable source. It's an opinion piece from a far-right ideologue with an ax to grind. It contains not one word from Ford himself.

Ford's whole life shows that he was up to more than just maximizing profit. Look at his energetic pacifism or his vigorous paternalism toward workers. Or his antisemitism for that matter. Wealth to him was a means to a variety of ends.

If you want to argue that the the higher wage was purely profit-driven, you'll have to show it was not just profitable, but unrelated to those other motivations that drove him. I don't think that's doable. And if it were, I'm not seeing it as relevant to my point, which is that Bezos is being mean-spirited and short-sighted even in comparison to jerks like Henry Ford.

> That is nothing like a reliable source. It's an opinion piece from a far-right ideologue with an ax to grind. It contains not one word from Ford himself.

I simply picked it as one of the many long list of sites that you'll find if you simply search "Henry Ford $5 wage". They all tie the increased wages to turnover issues and financial motivations. If you disagree with what appears to be the prevailing wisdom, then please feel free to cite sources to refute it.

> Ford's whole life shows that he was up to more than just maximizing profit. Look at his energetic pacifism or his vigorous paternalism toward workers. Or his antisemitism for that matter. Wealth to him was a means to a variety of ends.

You'll see I said that 'I actually agree with you that as a person he wasn't "only in it for the money" - I am only arguing that his wage-setting policies were purely about the money.' You'll need to back up the last statement with sources.

> If you want to argue that the the higher wage was purely profit-driven, you'll have to show it was not just profitable, but unrelated to those other motivations that drove him. I don't think that's doable. And if it were, I'm not seeing it as relevant to my point, which is that Bezos is being mean-spirited and short-sighted even in comparison to jerks like Henry Ford.

The onus is on you to disprove what most people have to say about the issue. It's also not even clear that Bezos would earn more money even if he raised wages; Amazon already pays a $15 minimum wage which is much higher than the competition (and much higher than the $7ish federal minimum wage), so I think it's pretty clear that both Henry Ford and Jeff Bezos paid/are paying enough in order to avoid extremely high turnover while asking their workers to do laborious jobs.

If your approach to understanding this is to randomly Google things and pick the first link that looks like it agrees with you, I'm not seeing a lot of utility in further discussing this. You clearly have an opinion on Ford. From my perspective, you're welcome to keep being wrong about the topic.

> randomly Google things and pick the first link that looks like it agrees with you

The entire first page of Google agrees with me, which seems a reasonable proxy for the prevailing wisdom.

I don't have a particularly strongly-held opinion on the matter to be honest, but what you're saying isn't what the internet writ large appears to be saying, so I would love to see the sources which support your point of view.

> The entire first page of Google agrees with me, which seems a reasonable proxy for the prevailing wisdom.

I appreciate you clearing up that talking with you about this is definitely a waste of my time.

I read that in an interview with Henry Ford, the interviewer tried to get Ford to justify why his labor costs were higher than other automakers and Ford stopped him and said his labor costs were actually the lowest, per unit.

Also same article said the pace of work in Ford factories was brutally unsustainable. Which might also be why he went to an 8 hour work day.

This is not a fair fight, however. Do you know how often US lawmakers talk on the phone with a billionaire? About once a week. They get the preferential treatment, they get the laws passed, and if the "normals" want to do something crazy like organize to negotiate working conditions, then it's a code-red emergency in Washington the sky is falling somebody do something.

> Jeff Bezos wants to pay the least for the most amount of work, and the workers want the most money for the least amount of work. It sounds like both parties should meet themselves half-way as opposed to living in what is effectively modern-day slavery.

The way it currently works is what you describe here. Amazon pays the least they can for the most amount of work, and workers work the least they can for the most amount of pay. They meet at the equilibrium where Amazon receives adequate labor, and the workers receive adequate pay.

There's nothing radical or communistic about that idea.

Where "the equilibrium" ends up changes over time though. Over the past 40-50 years in most developed countries, capital's (i.e. businesses') share of gross national income has been rising and labor's has been falling. This means that on average "the middle" has been trending towards companies paying less for the same amount of labor.

This is true, and it wouldn't suggest anything is amiss.

In the US for example, there's a lot of factors in increasing labor supply: (1) From 1955 to today, women have gone from 36% labor force participation to 59%. (2) The percent of immigrants in the US in the 1950s to 1970s was less than 50% of what it is today.

When the labor supply grows, it'll push downward on wages as the equilibrium price shifts.

America's a country designed to be optimized for freedom, which includes women having the ability to participate in equal numbers to men in the workforce if desired, and immigrants having the ability to come and build their lives here if desired. That will lower wages, which is just one part of the picture in terms of economics and policy.

This isn't actually true because the growth of Gross National Income is proportional to the size of the working population. I'm also not talking about wages in absolute terms, I'm talking about the wage share (the percentage of GNI which goes to wages, as opposed to capital). Definitionally, the wage share has been declining since the 1970s because rates of growth have been slowing, which increases the capital/income ratio and (assuming return rates on capital remain relatively constant) will increase the capital share of income (and thus decrease the wage share). I'm not necessarily claiming to know what the correct value should be for the wage share, only that it's significantly lower than it used to be (and will probably continue to decline for the forseeable future). That being said, the wage share is obviously closely tied to inequality; if you reduced returns on capital and/or increased growth, you'd likely also increase the wage share.

Amazon is unapologetically Randian, and many senior people are proud of that. That doesn't have anything to do with the rest of your comment though.

Where did you read this? That's important context.

There were a number of recent articles. Here's one:

https://www.businessinsider.com/amazon-polices-based-jeff-be...

Business insider is a lying trash rag.

The primary source is the NYT, not Business Insider. They clearly name their source too, "David Niekerk, a former Amazon vice president who built the warehouse human resources operations"

1: https://www.nytimes.com/interactive/2021/06/15/us/amazon-wor...

NY times I can get behind. Business insider is like citing the national enquirer.

I said that there were a number of recent articles. When you decided you didn't like the one I gave you, did you look for any of the rest?

I had already read the No times article when you posted your comment.

My issue with not with your point, my issue is with business insider being cited as an authority for any news.

I don't think I cited it as an authority. If you believe they misrepresented the NYT article, feel free to say so. But otherwise, maybe there are better places to ride your hobbyhorse?

You have a profound misunderstanding of Ayn Rand and, I suspect, Jeff Bezos.

I doubt that. She wrote impossible fantasy that convinced wealthy fools they were destined.

Maybe Amazon just needs better SRE. They should read Google's SRE books.

The point of the system is that engineers operate what they own, that's fundamentally different from SRE. It allows for significantly greater organizational decoupling. At Google, they had to go through an entire ceremony just to get Rust into the monorepo (I actually don't know if they ever ended up doing that outside of the Fuschia repos) while at Amazon someone just created a build script that wraps cargo and now AWS heavily uses Rust for several control plane systems and Firecracker. This kind of agility is not easily possible with an SRE system.

This is really orthogonal to SRE. Google, in many ways, prefers centralized infrastructure. They invest in a few ways of doing things and integrate them all together. Devs have a few tools/libraries/services etc. to pick from and they mostly work together very well.

This makes SRE easier since devs can throw various applications over the wall and SRE only has to learn a few different pieces of infrastructure.

But in practice, any pariticular SRE/dev team pair would work together for a long time and use a small set of infrastructure anyway even if it wasn't the same as the rest of the company. SREs just become a little less interchangeable since other teams will use different sets of infra.

I mean by this logic all tier1 service engineers are just rotating SREs anyway (though I do know some services already have dedicated SREs for operations work)

Current Amazon engineer.

As others have said, management at Amazon will only direct focus on the development of new features or new services, at the complete detriment of improving existing services, or even the overall architectural design of a particular space of the business. The definition of completing a service or feature is only that a customer is using it without complaint. Management has no interest in technical reasoning, which causes the design decisions that rest at their level to be unreasonable. This leads to a few problems:

1. Services are only ever about 50% complete. Unit tests typically exist to some degree, but integration tests, documentation, complete monitoring and operational automation are rarely done. Services typically have numerous obvious bugs, grossly bad optimization, hideous over engineering, and sometimes design issues. Because the customer cannot detect these things when the first use the service (maybe it will reflect the second time as a bug, or as slow performance later down the line, or long times to develop new features), there is no interest in fixing them.

2. The graph of service dependencies is entirely unmanaged. Any service can depend on any other service, for any reason. This results in a massive, undesigned spaghetti of a system. Something like s3 or whatever will usually be supported in some way by a spaghetti built for s3, and if s3 fails, it is usually not immediately obvious which service in the spaghetti is responsible. It makes adding something new to the overall system take a very long time.

3. Even if a customer is encountering an acute problem, and management is asking for it to be fixed, if the problem is rooted at a system level outside the boundaries of a single service, thus at the level of management, management is unable to engage with any reasoning as to how it should be solved. Only management holds the keys to assigning work (senior or principal devs hold basically zero sway) and thus management must have the technical reasoning ability to make these decisions.

4. Management will sometimes intrude in service level problems and make unreasonable decisions. Examples:

  4.1 I was told python is not performant (despite my history at the company having me deploy python code to every single physical host in the fleet) and asked to research and explain why it isn't scalable and we should switch off it. I declined to work on the issue

  4.2 Management had an issue raised to them where a single user had sev2ed us because they couldn't paste into a field on our service. Investigation quickly revealed the user was trying to paste text with a space into a numerical field, and the browser was preventing it. The issue already had two solutions suggested: add highlighting to invalid inputs, and strip spaces from inputs. Despite this management decided a formal review was required, where somehow we would have to dig deeper than the existing explanation and explain how this happened and what should be done.

Article does not support headline. Even if you don't stay at Amazon for many years, that doesn't mean your career is over or worse than having not worked there.

Also, it's about Amazon India, so take care before generalizing globally.

You're ignoring the part about the impact the psychological toll will have on your ability to perform and the fact that you can't use your manager there as a reference since they've intentionally railroaded you and have fake peer reviews on file that do the same thing.

What does Amazon need thousands of engineers for, exactly?

Amazon.com is a fine website. It is finished. It is complete, perfect as it is. The website could remain the exact same for the next ten decades, with minor adjustments to the product menus to reflect new products, and it would serve its purpose perfectly without risking being dethroned by competitors.

Why do millions of lines of code need to be written each month? It certainly isn't reflected in my browsing experience.

The British cracked the German naval codes with no more mathematicians than can sit at a table. But Amazon needs thousands of engineers to run an e-commerce website (and its concurrent AWS, which could be run by less than 50 engineers)?

(Not a software engineer or someone who's ever worked for $AMZN)

Edit: Woosh, way too many people failed to understand that this post was mostly sarcasm. Cunningham's Law in action.

Most Amazon engineers do not work on the amazon.com front end.

They have other suites of products apart from the AMZN.com. - AWS and the shitload of solutions on AWS come to mind. - A lot of internal tools for managing supply chain, financials etc also come to my mind.

AWS is a money printing machine in a very dynamic market. They need to keep the pace or MS will eat their lunch.

Amazon also sells hardware (kindles, echo, sidewalk), has a media platform through Prime, has logistics software for vendor and order fulfillment, in addition you are underestimating the cost of running a global e-commerce site (legal complicance, security, privacy, accessibility, etc. are constant technical draws on even established products)

Amazon web services runs a fair share of the internet. Around 33% as of Sept 2020 according to Forbes [^1].

[^1]: https://www.forbes.com/sites/danrunkevicius/2020/09/03/how-a...

>AWS, which could be run by less than 50 engineers)?

What the...

> (Not a software engineer

Ah, yes.

It shows.

50 engineers is probably less than the size of some support teams within Amazon that handle a single large corporate AWS customer.

This is a peak HN comment (the one you are replying to)! The other day on the Wikipedia thread, there was a data scientist who said he could run a website like Wikipedia all by himself. "How hard it is?". And here we have the same thing for Amazon..

AWS have close to 200 services. If the origina cmmenter js saying each service has 50 folks, it is 10k people. Which sounds about right the company have more than a million people working across all sub organizations. So you can imagine all kinds of work environments

Admittedly I've even heard senior executives rhetorically ask "What do all those people at $LOCATION even do?" At scale, you need a lot to sell, support customers, put marketing programs in place, do developer outreach, test, be on call, etc. etc. But, yeah, you'd think it would obvious that even just talking about straight engineering, you need more than a fraction of a person to develop and enhance each AWS service.

Listen if you can't run 33% of the internet with 50 engineers what are you even doing?

How do you know it's "finished"? You could have said the same thing in 2006 and been completely wrong. What makes you think the customer front end of Amazon is where the bulk of the engineering is?

Edit: engineering and maintaining the infrastructure of Amazon compared Vs cracking a code in world war two seems like a strange comparison.

AWS alone requires thousands of software engineers. And there is Amazon Echo, Twitch, Prime Video, a lot of software for warehouses and logistics, their autonomous stores, game studios, Kindle, Amazon Fire devices, their app store, Audible and I barely scratched the surface.

> and its concurrent AWS, which could be run by less than 50 engineers

One engineer for every four offerings.

Heavily-utilized websites are like icebergs. You see the homepage, but what about the constant ongoing work to ensure sellers don't game your review system? What about emergent product categories? What about integration with your competitor's new smart home device? Amazon doesn't pay most of its engineers to sit around.

Too many assumptions here.

You need to consider the number of the tech products and the number of markets.

The resulting number would be vast.

Consider this metaphor:

You go on vacation. You stay at a hotel, not a “B&B.” Your vacation, your money, so you find a place close to the things you want to see, with plenty of free amenities like free continental breakfast and free WiFi.

It’s somewhat reasonably priced, too.

Now you go on a business trip financed by your employer, who books you into a “business class” hotel. It’s also close to things you want to see, and it has a few nicer finishes than your vacation hotel.

But the service is worse! You have to order WiFi, and they have some annoying provider that puts roadblocks between you and their slow WiFi. There’s free coffee in the room, but if you want breakfast, you pay nosebleed prices for room service or you have to line up for a table in their brunch restaurant.

And their “rack rate” for rooms is double what you paid on vacation! How could your employer be so stupid as to book you into this expensive hotel and force you to suffer worse service?

Makes no sense. Ok, what’s the connection?

———

Well, WHY does your employer prefer the expensive place with bad service? The answer is that it provides good service for your employer, but not for you. YOU AREN’T THEIR CUSTOMER.

The business-class hotel integrates with your employer’s billing and expense systems. Your company can easily book dozens, hundreds, or even thousands of rooms when they’re putting on a conference. They have incentive pricing that appeals to companies, not people.

They have GREAT customer service when you’re calling about a billing code, but not when you’re calling about an extra pot of hot water for your tea.

We don’t see that as guests of the hotel. So if we were to discover that the big hotel chain has hundreds or even thousands of programmers, we’d ask, “What do they need so many engineers for? All they have is a crappy web site that can take a reservation, in a crappy way.”

We don’t see the people putting in the work to integrate with systems we can’t see, serving the needs of people and companies we don’t even know exist.

I betcha it’s the same at Amazon. We can’t judge the complexity of their operations looking at the page they serve us when we buy a book online. We have no idea what’s involved integrating with the corporations that provide them with music, TV shows, and movies to stream.

We have no idea how complicated it is to integrate with all the logistics systems around shipping products around the world, in real time.

There’s a massive machine we can’t see. That’s what all those programmers are building and maintaining.

While I don't disagree with your general point, I'm not sure it broadly applies here.

Honestly, a nice B&B is probably priced pretty similarly to a business-class hotel.

Why do I usually stay in business class hotels in cities? Chain hotels like Marriott's brands are pretty consistent quantities. A random B&B really isn't. Business class hotels also tend to have 24-hour desks, I can leave my luggage after checkout, etc. If I don't really care much about the hotel, which tends to be the case when I'm traveling in a city on business, some mid-level chain hotel is just less mental overhead.

All true, but I’m not comparing a business-class hotel to a B&B, I’m comparing one kind of hotel to another. All the major chains have offerings in both markets I describe:

1. Hotels that appeal to guests who pay their own way, and; 2. Hotels that appeal to companies with business travel needs.

The second type of hotel has the massive machine hidden from guests.

Certainly all the business class hotels have the ability to reserve room blocks, cater events, pre-book rooms, etc. But those are essentially additional profit centers. I'm just saying that many/most of us essentially deal with hotels 1:1 even for business travel.

But, yes, there are some brands of e.g. Marriott that are in part oriented to corporate events and the like and other that are mostly for individual travelers whether on vacation or business.

Of course we do. I’m just explaining why such a place might have many, many more programmers and other employees than we would guess are needed judging by our experience out at the periphery of their business.

Absolutely, and I'm usually perfectly cool with some nice B&B having some janky third-party website or even, gasp, having to call them on the phone to make changes etc. I'm not fine with Marriott not having a streamlined mobile app, a loyalty program, 24-hour customer service, etc.There are certainly economies of scale with companies generally but there are also costs.

My wife and I were recently escorted from a Ritz (a sub-brand of Marriot);

We were paying in points (a lot of points), they checked us in but still couldn't figure out their back-office payment processing or something because they confronted us at 9:00pm and asked us to swipe a card because "they couldn't verify the certificate"

After much back and forth between us and the front office and a customer support rep on the phone (who repeatedly suggested that the "certificate" was there and valid), we decided we didn't really want to deal with the hassle of staying where we were being treated like criminals.

I went up to get our stuff from our room and my card had been deactivated; I had be escorted to the room by a member of the Ritz security staff to get our luggage.

So, you're not really assured a good experience no matter where you stay.

I have to meet the first employer that does the hotel bookings for me. I always need to sort it myself and expense back. The only thing the employer did is arranging a discounted room price.

Normally, meant that I spend at least one day a month doing my hotel/travel expenses. That's ~40-46 lost working days :)

Pay yourself and expense it back is usually an SMB strategy for keeping costs down by offloading work onto employees. It’s not just your time that’s wasted, but all the work to organize expense reports, make sure you used the right codes, &c.

It’s a headache for you and the folks in accounts, but at small scale, that works. But as the company grows, this becomes harder to justify. At some point they centralize a lot of this stuff, and when they do, many choose to start booking people into hotels that cater to businesses who book people into hotels.

Seems like you should get off your butt and create an AWS competitor with 50 engineers. You'll be able to underprice them by so much you'll completely eat their lunch.

You should ask yourself how you can be so self-assured yet wrong by several orders of magnitude. I'm not saying that as an insult -- it's probably worth re-evaluating your confidence : accuracy relationship.

Indeed, a truly remarkable disconnect with reality that honestly makes me question the point of having an account on a site called "Hacker News."

Well this controversial opinion sure got pounded down.

> The British cracked the German naval codes with no more mathematicians than can sit at a table.

The Government Code and Cipher School employed almost 10k people during ww2.

Yeah, but you couldn't fit all those 10k people in a Hollywood movie.

And the Polish mathematicians who reverse engineered the Enigma machines weren't invited to sit at the table once they shared their work with the British and French

>> Not a software engineer or someone who's ever worked for AMZN.

Exactly. There is SO much more going on than you realize.

The Amazon retail website gets hundreds of thousands of orders every second. New code gets deployed literally every night.

The retail website is like the tip of an iceberg floating above the water. There are hundreds of web services underneath that provide the backend and data stores. Each one of those services has a team of engineers to maintain the service with one engineer oncall 24/7.

They have a big product portfolio, so less than 50 engineers is a big stretch even in the best of circumstances.

But they are clearly in a very deep hole of technical debt that they will never be able to dig themselves out of, so all they can do is throw more people at the problem.

I don’t think this warrants being downvoted to the point of being barely readable. The estimate may be way off, but it’s a perfectly good basis for a discussion. Please don’t do this when you can use words instead.

HAHAHAHHHHa hah... hah. ha hah cough

Oh, wow, what an amazing take. You owe me a new keyboard.

Side note: I’d love to use a forum like this but filter out all the Indian specific things. Is that possible?

Who are the people who still dream of careers? I don't know any.

Young ambitious people.

Young ambitious people are starting companies, not going corporate.

How hard is it to become competent at full stack development without working for someone else first?

> Who are the people who still dream of careers?

Those who like the paycheck attached to a career. 1 million in stock over 5 years is a good deal.

For me, I'm not quite ready to give up on the dream having a job that I actually enjoy. I'm still young, so effort I put into my career now will pay dividends for the rest of my life. Giving up the most productive hours of my day to a company is a big deal, and I'd at least like the job to be intellectually stimulating, if not financially rewarding. At this stage in my life, showing up at work just for the paychecks seems like giving up on a brighter future.

Despite ballooning tuitions, software development is still a career that people of modest means can aspire to.

That we don’t talk about it more here I think says something about how you only talk about escaping in certain company. You can’t pass for upper class if people know where you come from.

Oh man, these “I don’t dream of labor” types are so fucking cringe, lol.

Also condescending!

A close friend of mine beat cancer as a child. Ever since then he’s dreamed of working in oncology, and is finishing school just this year. He’s very excited, and I’m happy to see his career goals coming to fruition.

Other examples abound.

I find, in my social circle anyway, the type who ask, “who dreams of careers?” are usually the type to have had cushy upbringings that afford them the ability to ignore dreaming of finding stable, well-paying careers. Let alone the mission-style careers I described earlier. Must be nice.

In any case, I’ve worked on teams in both Amazon and AWS. Have not experienced the described culture. However, I’ve only ever worked in science roles/orgs, and I hear the difference between science and software (particularly product) can be quite stark.

As others have mentioned: most variation is based on your manager and skip-level manager.

Finding stable, well paying careers is more of a means to an end. I think some in this group of people (myself included) are closer to the "I don't dream of labor" type than the "mission style careers" type. That said, I am open to changing my view and don't resent anyone who has found a mission worth working towards.

Hi, I'm Nate. Nice to meet you.

Hire to fire might just be the sign of a healthy organization that only wants performant employees. You can't really tell how well someone is going to do just form an interview. Given a reasonable sample of work you'll know if you should keep them or not. Of course, human nature will lead to some of the perversions of the core idea and lead to good people being let go and buddies being kept on.

I can’t possibly imagine why anyone would ever go work at Amazon. How low would your prospects have to sink in order to make someone do that to themselves? I suppose if you’re completely homeless and in danger of starvation. Personally, I’d rather eat the weeds that form on the sidewalk or probably go dumpster diving. That’s much more dignified than working at amazon.

Just imagine the damage you’re doing to your resume. How do you explain having worked at the worst company in the world?

well… you’re being downvoted.

first: if you were truly homeless and in danger if starvation you’d work there. it’s also not cool to diminish people because they’re homeless.

second: having Amazon on your resume is not bad. It shows that you can work hard and if you managed to survive in there you’re probably thrive in other environments.

third: overall I agree with the sentiment you are expressing

Reply on news.ycombinator.com