Ask HN: How much time do you invest in writing tests? | Hacker News Reader

Ask HN: How much time do you invest in writing tests?

25 pointsamirathi18 comments

I primarily work with Ruby on Rails, Django. I spend significant amount of time writing tests, sometimes more than what it took me to write the code it is testing. What is your experience with this?

Any tips/tools to get to similar coverage in shorter amount of time?

View on HN

A similar question, whose answer will likely be revealing, is: How much time do you spend manually testing? Or, worse, how much time do your users spend/waste on bugs?

Note the key difference here: invest vs. spend. Time spent on writing good automated tests is an investment. That investment pays dividends every time the test suite is ran. On the other hand, time spent manually testing is lost. If you ever want to run the same test, you have to spend the same amount of time.

I gave a presentation on continuous integration and testing at PyOhio that goes into this idea in more depth: https://m.youtube.com/watch?v=K-iii4kMLWE

To answer your question: as much as it takes. We work hard to never ship code that isn't well tested. I don't even think in terms of writing code without tests anymore, the testing is just a part of the overall process needed to finish the job.

>Note the key difference here: invest vs. spend. Time spent on writing good automated tests is an investment. That investment pays dividends every time the test suite is ran. On the other hand, time spent manually testing is lost. If you ever want to run the same test, you have to spend the same amount of time.

Well I totally understand the benefits of automated testing. That's not the point here. My motivation for asking this question was to see how much time does it take for others to write automated tests (measured as % of time spent in coding the feature itself). If it's significant chunk of time, can we build some tooling to speed up this process?

I suspect that building tooling, as in things to automate test-generation, likely only going to be successful in situations which cry out for some duplication to be eliminated. A thing that can be done is to improve the way that the frameworks we use support testability. For instance, the company I'm at felt like rails controllers had gotten hard to write tests for and so they replaced them with https://github.com/gocardless/coach.

Here's a blog post if you're interested in using it: https://gocardless.com/blog/getting-started-with-coach/

As a percentage it is high and I think that’s the main reason some teams try to get away without doing it.

I’d estimate 25-50% at least depending on the complexity of the feature.

The point above is valid though, how much time was spent dealing with the fall out of not writing tests? Not easy to measure unless people are meticulous about tracking hours over numerous releases which probably doesn’t happen. Leads and managers need to see the ROI in a visible and measurable way.

Tests help you unpack the problem you're coding for, so it's common to spend more time on them than the app code. I generally include tests for each acceptance criteria of the thing I'm working on, and also tests for "weird bits" -- things I'm afraid someone after me is going to break (myself included).

How do you get similar coverage in shorter time? Keep doing it. In a few years it will be second nature, and you'll stop seeing it as a separate activity, but a natural part of feature development.

>How do you get similar coverage in shorter time? Keep doing it. In a few years it will be second nature, and you'll stop seeing it as a separate activity, but a natural part of feature development.

Practice will definitely help. Do you think that there is lack of tooling in this space?

A lot. And for good reason. I generally guide people away from the negativity around the thought of "it takes longer to write tests than code". Coding is simply the logic put into the language you are writing something in. Programming is the act of thinking and solving the problem and then putting it into code.

If someone said they take longer to think about the code than to write the code, we'd instinctively guide them towards saying that's natural.

Writing tests is very much a way of thinking through a problem and bringing clarity to exactly which piece you want to solve at the time. That is always going to take longer because it involves thinking through the problem first. It's natural.

The answer to how much time is invested into writing tests?

For me, it's usually more than the time taken to write the code to pass the tests. By the time I'm done writing the tests I know what I need to do to make it pass.

A second answer to the question is, as much time as it takes. Because I definitely don't want to repay that time later in the form of regressions and stress when change comes knocking at the door.

i been coding for over 16 years now and i rarely write unit tests. only time i write them are mission critical functions where i need to easily check across many different situations. these accounts for less than 10% of my code.

however, i do write functional tests for all my api endpoints. this gives me pretty solid coverage and a good balance of not over testing.

How much time do those functional tests take to write (measured as % of time spent in coding the feature itself)? And what's the primary programming language here?

The more tests you have, the easier it is to write new tests. It works like this:

"Let's see, I need my setup to be X. Well, this other test has a setup X' that is almost the same; I'll just copy that and tweak it. Then I need to open a connection to server Y; this other test does that. Copy again (or make it a subroutine)..."

After all of that, I only have to write maybe 20% of the test. That makes it a lot faster...

I hate to say it, but practice has helped me in writing tests faster. I can't give you any advice on Rails and Django as I mostly do React tests.

1. I've built up "muscle memory" for common patterns.

2. I've learned where I'm over-testing and that I don't need to write as much.

3. I've altered how I write code in order to make it easier to test. A common sign it will be harder to test is if I get data from an outside source that I didn't provide to the function/class.

I spend about 30% of my time on a new feature writing tests. BTW that is in RoR which the primary language I code in. I would recommend watching this video by Sandi Matz on testing: https://www.youtube.com/watch?v=URSWYvyc42M. I have used this methodology a lot to determine what I should test.

None on smaller systems with simple tech that we know will be replaced within 5-10 years because tech moves on.

60% devtime on larger systems with millions of lines of code that will live on.

We used to test on smalller applications as well, but we rarely had a need for it because the systems rarely changed until technology and architecture forced rewrites. Like moving from windows based applications to asp.net web-apps to apis with JS fronts to cloud native.

Our productivity is up by 80% because we mainly do smaller, single to few function apps and our debugging time or the amount of user experienced problems hasn’t increased.

Testing mutable object-oriented code tends to be a lot more tedious than testing immutable functional code. Clojure's REPL-driven development style in particular makes testing a breeze. Often you're just playing around with the code live and saving the results but there's also Midje for writing more formal test suites.

Property based testing, e.g. http://hypothesis.readthedocs.io/en/latest/ for Python, can help get lots of coverage with less effort. Works better in some domains than others.

I can't say percentage because sometimes I spend days on tests, sometimes I write no tests. (https://www.youtube.com/watch?v=Vaq_e7qUA-4&feature=youtu.be... is my attempt to explain how I decide.)

I don't know, but I write unit tests and my code is being written with them (Kotlin primarily). Every new feature starts with tests being written and by the time they're done the code is implemented already.

Practise. Yes I also experience the same thing: sometimes writing spending more time with the test code than the code under test. TDD is very helpful to focus me as a dev and get to the end of the story fast. Early, while infrastructure is new and features are new the test code takes longer. Over time tho as each story becomes more well understood, tests and copy and paste - you build a corpus of scenario variations from different tests you can paste together to build the requirements you need.

Also consider whether you really need a test: https://robdmoore.id.au/blog/2015/01/26/review-of-jimmy-boga...

Not nearly enough

Reply on news.ycombinator.com