Hacker News Reader

Comment by discodave | Hacker News Reader

I saw pleeeeeenty of untested code at Amazon/AWS. Looking back it was almost like the most important services/code had the least amount of testing. While internal boondoggle projects (I worked on a couple) had complicated test plans and debates about coverage metrics.

View on HN

It's gotta be a whole thing to even think about how to accurately test this kind of software. Simulating all kinds of hardware failures, network partitions, power failures, or the thousand other failure modes.

Then again they get like $100B in revenue that should buy some decent unit tests.

This is almost always the case.

The most important services get the most attention from leaders who apply the most pressure, especially in the first ~2y of a fast-growing or high-potential product. So people skip tests.

reality most of the real world successful projects are mostly untested because that's not actually a high ROI endeavor. it kills me to realize that mediocre code you can hack all over to do unnatural things is generally higher value in phase I than the same code done well in twice the time.

This attitude is why modern software is a continuing controlled flight into terrain.

I think the pendulum swing back is going to be designing code that is harder to make bad.

Typescript is a good example of trying to fix this. Rust is even better.

Deno, I think, takes things in a better direction as well.

Ultimately we're going to need systems that just don't let you do "unnatural" things but still maintain a great deal of forward mobility. I don't think that's an unreasonable ask of the future.

Finger wagging has also failed to produce any solutions.

Interesting. I also work for a cloud provider. My team work on both internal infrastructure as well as product features. We take testing coverage very seriously and tie the metrics to the team's perf. Any product feature must have unit tests, integration tests at each layer of the stack, staging test, production test and continuous probers in production. But our reliability is still far from satisfactory. Now with your observation at AWS, I start wondering whether the coverage effort and different types of tests really help or not...

> Now with your observation at AWS, I start wondering whether the coverage effort and different types of tests really help or not...

Figuring out ROI for testing is a very tricky problem. I'm glad to hear your team invests in testing. I agree it's hard to know if you're wasting money or not doing enough!

my take is that the overwhelming majority of services insufficiently invest in making testing easy. the services that need to grow fast due to customer demand skip the tests while the services that aren't going much of anywhere spend way too much time on testing.

Reply on news.ycombinator.com