For developers under pressure, it’s better for bugs to be found in production
amazingcto.com
amazingcto.com
See the first point
If I have to ballpark an estimate on a tool/project, I say upfront something like: it'll take me 6 days olus a day for very limited tests, so 7. Twice as much with almost complete coverage. But each time we'll have to improve/refractor, a bit of that time will be recovered.
Basically I ask how important the new project will be,without asking that (because 99% of the time the response is 'very').
I'm realizing I just say I was managing my managers. Should I ask for senior devops position now? :)
I am always extremely clear with my estimates being just that, estimates, and that a complete testing double dev time. Most of the time I'm told 'we don't care right now', but on some projects, management accept longer dev time for more stability and less bugs (we build internal tools).
A - puts a brand new project on CV and moves on before go-live.
B - stays behind to handle going-live issues and maintain.
Guess which kind sees pay growth and promotions?
The investment calculation is quite complex and many of the variables require guesses. A lot of returns on automation work are not positive.
I find having really good metrics and a tight development cycle allows for quickly iterating on distributed systems problems. Obviously the best situation is to have all of the above: unit tests, integration tests, and a tight development cycle in prod.
If I had to pick one because I am time constrained, I would choose testing in prod with good metrics which is maybe what the article is getting at.
The investment in this stuff can be quite high...unless you've got premade tooling for all of this that you can just drop in.
> Tests can only tell developers they made a mistake. There is no gain at that moment.
The value is locating the problem code faster, and that generally can't be done efficiently in production. Outside a testing environment, you rely on logs alone to debug, or if you're more proactive you might grab session data but that means potentially sensitive user data is being cached and passed around to the support/triage team. At a security minded organization this should be impossible or at least difficult.
I agree it's not worth chasing Test Driven Development until you're actually testing during development.
This is a real problem, and it falls squarely on management. Testing needs time and resources, it needs priority and planning.
That's the reason I started speaking at conferences, started an online testing training course specifically for software managers, and am currently writing a book on the subject.
An excellent point for most cases that ultimately cannot be satisfied for others. For instance, this most recent HTTP/2 Rapid Reset vulnerability, could it be solved without shared PCAPs?
This is the dumbest thing I have seen on HN in a long, long time. Full test coverage supported by good mocks is one of the quickest and most efficient ways to find regressions that I know. Don't discount the value in helping developers see when they've made mistakes, especially when you're rotating junior and mid-level developers on and off your team.
When reviewing, the first thing I do is look at the tests. That is how I make sure developers understood the new requirements. Only after verifying that, so I then do a code review to critique implementation.
Tests that test services that do interesting thing in my code. I find that I can get 90% of my work done without ever firing up a web browser. It's cool to know that by the time you hit up the ui the code already works perfectly and I spent less time coding than otherwise.
So idk what this article is pointing out. Tests for test sake is probably that, but a big chunk of unit testing is definitely not bad for developers.
Tests are great for the codebase, for stability, for onboarding. But for devs who have management/exec pressure, and especially if the team is 'agile', they will slow your sprint, discover edge cases or bugs you never put in your cards and ultimately 'slow down' initial development, and if your company is especially bad, you'll get performance reviews. If you let the edge case/bug happen in prod, it'll take more time to correct it than if you caught it prior to that, you might have to modify an loop or a data structure and introduce more bugs, but at least the software was delivered in time, and you'll get paid for that time spent correcting those bug if you're a contractor.
One reason this doesn't make much sense is that this requires hoping that the bugs pass QA. If QA finds those bugs, sends them back to dev, then they still have to fix that + very significant time overhead of this round trip which doesn't make it "worth it".
Eventually it's easier to just agree with them and do your best than waste even more time arguing with them. Eventually you fail to ship or you ship a buggy mess. The software engineers know the code quality is shit but they don't have time to think or set anything up.
But it's always "we don't write tests because there's no time" and not "we don't write tests since they might reveal bugs, and we don't want that" as the article claims.
Without testing, you're letting all sorts of bugs into production - both low impact and catastrophic data destruction bugs.
Hahaha, I have been penalized for going steady enough times that I reached a point where I have no incentive to make services any better. I found it better to sweep issues under the rug and switch teams before it blows up.
It pains me but I am not going to care about it if I am going to be penalized for it.
Client/ Manager :- When is this going to be ready?
Me :- Estimated time for development is n1 days and around n2 days for testing, and depending on the bugs n3 days to fix them.
Client/manager :- No no, we don't have time, do it in n1-1 days.
And when you ship it with bugs, they have a shocked Pikachu face.
Don't give estimates separated by test and features.
New feature at hand? First thing you code is the scaffolding to call or trigger new feature (this may be trivial on some projects, or impossible on others).
So long as you start out that way when asked, how long? The tail end always contains the feature, and they can't say cut it.
I worked at a place once where I was setting up a minimal CI/CD pipeline for the project -- I took the first few weeks to do this. In a meeting I was called out by another engineer that had longer tenure at the company. "Are you working on 'devops' stuff because you don't know how to do the primary objective". I responded with a very long triad of why all this other stuff IS the primary objective, and was never questioned again (by that same engineer anyways). All this to say, part of being an engineer is also to educate fellow engineers on ways of doing things better than status quo. It can be hard, and you can put your self at risk. But it's a choice you have to make for your self, the engineer who cuts corners to make mostly arbitrary deadlines, or the one who build robust software. You don't have to go in 100%, but at least have a leaning to one side or the other and make the better choice when it really matters.
I've never talked about solving problems like that, always just talked about the time to fix the bug / implement the feature, and test were implicitly part of that. I've also never had a manager start micromanaging me over how many tests I was writing. That's always been my own judgment call to make.
This doesn't actually fix the incentive problem, though. If the reason developers don't like writing tests is because finding bugs puts them under additional pressure and isn't rewarded, requiring this so doesn't actually help them. Unless, of course, management comes to understand and accept that more time will be spent fixing bugs/improving code quality as a result.
That's not my goal. I like delivering high quality stuff to users. I like 'moving the needle'. I like coming up with a plan to produce business value, and doing it. I like getting stuff done.
This article perpetuates the 'developer vs manager' concept which is a ridiculous mindset rooted in primitive human flaws. Fix your relationships with your colleagues and work together to make quality stuff. No deception. Only cooperation.
Can you tell this to my manager? Especially the "no deception" part?
Sprint "commitments" in my world. A single word with major psychological impacts.
A low effort management practice is to make engineers move fast at the expense of quality.
An even lower effort practice is to then turn around and hold engineers accountable for the choices made by management in the first place.
In my experience, this just means asking them to move faster, faster, faster but hold them "accountable" for the inevitable outage.
Unless engineers have the overriding power to define and redefine timelines, without management retaliation, "holding" them accountable== make
This lets you try the feature out end-to-end. Click buttons that call APIs and see if the right actions occur. Fix all the stuff that breaks when you first try this out.
Then show it to a friendly. Product manager maybe, engineering manager, another developer... whatever. Someone who understands you're just looking for someone to try out this feature with you.
It works now? Great. Put unit and integration tests around it. Make sure that this happy thing you have running won't accidentally break. Now make it pretty. Give it the design product actually asked for. They'll have feedback, which you can now incorporate safely because you have tests.
When test-driven development is applicable, it’s so damn nice. If for no other reason, just because it requires so many fewer keystrokes and clicks, just have a process watching for changes and rerunning the test suite. Less chance of developing RSI, it’s good for developers.
I added the caveat “when applicable” because for straight-up UI behaviors it’s not, but the longer I write code, the more I see ways to separate out pure logic from UI and other external effects and I’ve never regretted doing it for reasons including: general reasoning, readability, TDD, future changes, and bug fixes.
Still, I hate having bugs in my code and strive to make sure they don't happen. Still I have had managers release my code when I tell them there are active bugs being worked on.
For those that don't know, this is a reference to:
I write unit tests only if the logic is completely encapsulated by the "unit". The less you need to mock, the higher value the tests bring. That's one of the reasons I prefer to work on monoliths, it's much easier to test almost-end-to-end.
Most of the integration tests are for happy paths, i.e. ones where you can catch the obvious regressions. Testing corner cases is much more troublesome, because you have to set up a whole lot of specifics in multiple services. These are much easier to simulate with mocks.
Mind you, I hate unit tests for specific functions: I much prefer them as component tests, where you start with the inputs to the module and check the outputs and calls outside of the module. These ease refactoring where you actually change the underlying code and without changing the unit tests, you know that you also handled all those pesky corner cases.
Integration tests are better at accurately reflecting the real software. There is less of a leap of faith between "this test passes" and "the software actually works".
Unit tests are better at telling you exactly what failed. There is less of a research project between "this test fails" and "this code right here needs to be fixed".
I don't really love anxiety-inducing leaps of faith or time-consuming research projects, but I don't know of one form of testing that avoids both.
But at the other end of the spectrum (say, php), things are very different. You want that code exercised, just to be confident that it will actually do something, anything. Even if the question wether those things it does are right or wrong is left to higher level tests.
How could it possibly be that unit tests are "good" or "bad" in general, for all software development?
You should prefer to instantiate your class with real objects, if you can’t do that then use fakes, as a very last resort you can use a mock but at that point your test is probably useless.
(The exception) If you have an class that is a wrapper around some HTTP requests, then it is fine to mock these HTTP calls. If you have a test that depends on this class, then you can use either mock these calls again, or upgrade to a fake if the mocking is too complex.
Unit tests are less valuable for testing applications because the unit tests end up just being mirrors of your application classes/functions and you have to make changes in 2 places now any time you need to tweak application logic.
For applications I think black box tests have the highest ROI. Your application should have a headless mode through which you can send all of your test cases (real life inputs) and verify the output of your app has expected shape/value. As you encounter issues from user feedback, simply add to your list of black box tests. The beauty is that down the road you can "rewrite your app in Rust" or whatever and you'll have a huge set of black box regression tests you can use to validate the rewrite.