How do you go about it? Full coverage? End to end only? Integration only? How much dev time does it end up taking you?
The answers here are all correct for the use case of the person answering it. We don't know your use case so we can't give you a good answer.
For example a seed funded health tech company needs 100% test coverage to be compliant with regulations.
A bootstrapped online collaboration tool with no users doesn't need any tests.
A seed funded online collaboration tool needs some tests so the developers can work together without breaking each others code.
Use TDD for bits that have very limited scope and function. For instance, an order book class or similar. Otherwise code first, test bugs if needed. For smaller projects (that aren't critical espc.) overtesting can kill your momentum and motivation, which is an often overlooked resource.
The goal is to have the major functionality of the platform tested automatically so we can be confident that new features don’t introduce new bugs.
Integration tests are typically best bang for buck: https://kentcdodds.com/blog/write-tests
Once manual testing starts causing noticeable pain, you can start automating some of it. This may very well be beyond $10-100M in valuation.
A lot of people say unit tests give them confidence, but that's a delusion. Losing confidence in your code means you have outgrown your architecture. However, you can temporarily address it with more tests, because addressing architecture issues takes some time. The secret is to write as little tests as possible to bring confidence back to acceptable level.
Don't spend more than 10% time on tests.
Don't let junior engineers write tests.
Being confident my stuff works without unit tests would require me to write unit tests and then delete them.
- Keep unit tests that test key algorithms for which there is a broad, formal, independent oracle of correctness, and for which there is ascribable business value.
Though I'd say that for low-level code (without dependencies) there's basically no difference between unit/integration/e2e tests. They are just tests.
The only difference I found is that with unit tests people will also test implementation details. So it's still better to have integration test mindset even when your only option is unit tests.
But for self contained code that is guaranteed to have edge cases (working with dates, date math, unstructured user input like SMS responses, almost always yes, but after some of the code is written and is getting cumbersome to test different cases manually.
Well, unit tests are fine and often we do it for things like testing that email parsers and unit conversion works. As long as it's easier than manual testing.
It's also easier to do tests than debug sometimes.
Back end is a different story - the ROI on tests increases drastically because tests are easier to write and failure is more expensive.
I generally write as little code as possible overall, and try to sell it as fast as possible.
I hate source code, it never ever is the result for me. So if i do not have to write code than I get to not write tests.
My goal is always to write as little code as absolutely possible. That's it
I mean how does it even works ? What if you want to build an app for a company which is wishing for digitization of their business - you need to have code - Don't you ?
Secondly how come founder are not liability - I have seen a stage in start ups when it comes to the situation where startup is either sold or purchased by other competitive business and after some months when the startup is absorbed the founders are gone or have a silent positions... Doesn't it make them liability afterwards?
Usually this is the practice I have seen successful in all the big projects
- unit test for each function
- Functional Integration test to test the building blocks and functions together (for example rest API with business layer and database
- BDD Behavior driven development where you write automated test to test your business scenarios by initializing the seed data in the test systems
- Performance testing
If you have freelancer who charge per hour or if you have consultants which are expensive - they will write too many test and there code will be so complex in the end that when you want to hire permanent employees or other freelancers they will say "I would like to rewrite it"
The developers now a days like to do clean coding and for simple things they discuss too much even for a program of 2 + 2. there will be endless discussions and in some companies (diverse or multi culture) developers are always under mining each other with what they like and don't like
I've seen so many absolutist opinions on testing that I'm distrustful of the wisdom of. When I see people say that there's only 1 way to test or not test things, and you always have to do it in a certain way, and fully test everything, I'm not sure that they're right. I think your need for code testing, and how much testing you do, matters more based on the context and risk you see through experience and good judgement.
> Making software for nuclear power plants? Sure, go slow and test the shit out of it.
> Making software for cars or space ships? Sure, go slow and test the shit out of it.
> Making large scale software for a big financial company where millions of dollars are at stake in the case of a fuckup? Sure, test the shit out of it. You can afford to throw 8 full teams of developers on every continent into nothing but testing, and then testing the tests, and then testing the tests for the tests.
> Building an API that multiple teams and customer are going to depend on? Yeah, testing that sounds like a good idea to ensure everything is working as expected.
> Running a profitable SaaS? Sure, at a certain point where you're not in wartime mode struggling for survival, fully testing and formalizing a lot of things makes some sense.
> But trying to get a tiny 1-human startup off of the ground that no lives depend on? I don't know what works for you best, but would you rather write code to make you money, or spend lots of time writing tests on stuff that's probably going to change a lot later anyway or that you don't even know if it really is a realistic business?
> Building a shitty app for some dumb marketing firm or whatever? I've literally never seen anybody in many types of businesses demand formal code testing. The only thing that matters to them is user-acceptance: the client and/or their QA team will manually test all of the functionality and tell you whether or not it's acceptable to them.
Tiny mistakes that I make are caught early on, and development gets a solid foundation in which to build the proper abstract syntax tree.
Oh, test code is about 1/4th the size of the original source code, line-wise, but it has 100% coverage on syntax linkage (and not necessarily 100% data value validation due to dependency on other field settings).
I decided tests are a good thing to have, and especially when already set up and you can easily add more by following the pattern.
I prefer unit tests that are narrow in scope and avoid IO interactions (for example databases)
Of course anyone using my framework can ignore the tests if they desire. But my gut feeling is tests save you time in the short run as well as long run.
For the frontend we do automated ui tests with a service called Autify.
This is not by any means extensive, BUT with very little time investment we get catch most of our bugs.
Ghost Inspector used to be my go to, but I’ve been looking for something in the $20-30 per month range. Makes it a much easier decision when we aren’t yet collecting much revenue.
I guess it really depends on what your company is doing, experience of the team and the language/tech you are using.
somewhat political perspective: if the org doesn't value testing or software quality, wait until there's obviously a business need to lift quality. E.g. the CTO is saying things such as "constant defects in component X are destroying most of the value of our product", users complain that the Y feature keeps getting broken in successive releases, customers refuse to renew subscriptions or upgrade to a new version until Y feature works reliably. Then propose the plan to improve QA by setting up an automated regression test suite, to allow refactor-to-test of the offending code to happen in a controlled, low risk way.
That said at my last company after 12 months we hired a few engineers so we had to transition to requiring tests for new functionality. This was just to stop regressions and make sure we could continue moving quickly.
1) mission critical parts of my system that will cause damage if they break
2) small functions that I can't wrap my head around (eg some complex regex extraction)
Everything else can be fixed fast with minimal damage as it comes up. My test coverage is less than 1%.
Firmware is mostly tested manually.
No, risk-based.