Product Integration Testing at the Speed of Netflix | Hacker News Reader

Product Integration Testing at the Speed of Netflix

techblog.netflix.com

49 pointshepha197912 comments

Usually Netflix engineering impresses me. This does not.

This sounds like all of the typical separate test org antipatterns that healthy companies eventually eliminate by eliminating the entire test team.

Integrated tests are a scam. The only way to succeed in a micro services architecture is with a strong focus on quality inside each service and consumer driven contract testing between.

I have to disagree with your last statement (integrated tests are a scam) very strongly. Think about it as follows:

We assert unit tests are useful, because although we may "trust" primitives (1+1=2) we can introduce semantic bugs in how we compose them.

Does it not also follow that if our primitives are now "microservice functions" and our composition is the "integrated service", we should test the composition for validity?

I truly believe in E2E testing, I could go on at length about esoteric bugs we found only in interactions between properly functioning microservices (Hell just today I watched a coworker work out a fun timezone doozie). One could make a feasible argument that with a more well defined set of constraints and capabilities, the microservice could have sufficiently covered unit test cases for all possible interactions from a stub perspective, but pragmatically, _this does not happen in the real world_, and E2E testing is a nice way of making sure your product is indeed continuing to not be on fire (mostly).

One could feasibly (from my view of the world) say "Integration tests are a cheap fix", to which I'd say "yes, yes they are" and then continue to perform them, even if they are only in the form of a click through smoke test by hand after a prod deployment :)

JB says it better than I can:

http://blog.thecodewhisperer.com/permalink/integrated-tests-...

https://vimeo.com/80533536

He had me agreeing at first, but the blog at least kinda devolved into a highly numbered breakdown of the impossibility of total test coverage. As I said in my post, and the blog seemed to echo, I accept that an integration test is a sign that unit tests are not sufficient, but I didn't see a compelling argument in his blog as to how perfect unit test coverage (sufficient to not integration test the _happy path_, not all things ever) is obtained.

I certainly can see the situation he's describing, but it feels like he goes on to expand it into a pathological strawman (ALL the unit tests EVERYWHERE testing EVERYTHING run at EVERY BUILD) more than a situation you manage a little more smartly. (As the oft-hammered quote, these things are all tools, the moment one stops not being useful/stops being worth the cost, we stop using it)

> The only way to succeed in a micro services architecture is with a strong focus on quality inside each service and consumer driven contract testing between.

Do you have any examples from other large companies that do micro-services at Netflixs' scale?

Really disappointed everytime one of these types of articles comes out without any working code to go with it. It's great that high level information is being shared, but even better if the code was shared as well. It's not like Netflix is in the business of selling testing services.

Exactly my thoughts. Its been same in many startups and companies. Whats something they are uniquely doing is not mentioned at all

That sounds a lot like entitlement to me.

> "Secure Connection Failed"

Link is broken.

Looks fine here. Are you running HTTPS everywhere?

No. Doesn't appear to be an add-on problem. Firefox reports "Secure Connection Failed" and chrome reports "This site can’t be reached".

Weird, just tested on Firefox for android (still connected to my local wifi) and it works fine. Debug time.

I get the same issue on firefox/chrome/edge here.

Reply on news.ycombinator.com