Hacker News Reader

Thankfully we don't have to. Hillel Wayne links to a few of the studies that have been done on TDD [0]. It doesn't have a conclusive effect on error rates in software. While I do write tests before I write code, more so in a dynamic language without a strong, static type system; it appears that there isn't any correlation to a reduced number of bugs.

But I still do it. And I think that's because that while I may prevent a few obvious errors with the practice the benefit I get is from the informal, automated specifications of the units of a module that I'm building. I also get the benefit of a continuous refactor cycle. Less experienced developers wonder how I write clean, simple code: I refactor, a lot, all the time, because I have tests checking my code.

If it's functional correctness we're after TDD is only one, small, piece of the puzzle. A strong, formal specification will go a lot further to ensuring correctness than fifty more unit tests.

[0] https://www.hillelwayne.com/talks/what-we-know-we-dont-know/

I haven't written tests as part of my regular workflow in... years now, but one of my earlier mind blown / level up as a developer was when I wrote tests, which made it required that my code actually had to be clean and conform to a load of best practices that weren't in my mind's eye yet.

The other day I was doing a part of Go Blueprints which touched upon TDD; one thing it highlighted was that if you use TDD you should only write until your test is green, no more - this (supposedly) avoids you writing too much code. It has some merits. I'll need to do a lot more TDD and co to actually see it though.

Don't you think that if tests are effective, they may let you go faster?

To put it another way, it is not surprising that error rates are similar since, presumably, you keep coding till you produce an error, fix it and then repeat.

The question is how much error free code you wrote between the errors. If testing -- or any technique for that matter, like relaxing, jogging, discussing, planning -- reduces errors, it will manifest as more functionality per error.

"Effective tests" are a bit like the sufficiently smart compiler, or No true Scotsman fallacy.

Tests either take a long time to run because they're integration tests in disguise, or they mock and stub module boundaries and are an impediment to design refactoring because lots of tests are invalidated.

I tend towards favouring unit tests for leaf modules, especially those that get reused a lot, and fewer but rich (and slow) integration tests for a whole stack of functionality, end to end, where the test doesn't need much rewriting even if the design changes substantially.

Unit testing everything usually undertests the composition of modules. You can fool yourself (especially using code coverage as a metric) that unit testing makes it easier to have complete testing, but actually the space of code under composition of units is much larger, and unit tests don't really test that at all.

Depending on your use of the term "unit" and "integration" (testing taxonomy is bad), I tend to do the opposite, but with the addition of acceptance (user feature testing) to tie the whole thing together.

By this I mean that I unit test (as driven out by Discovery Testing i.e mocking collaborators) down the dependency graph until I hit a leaf node, which I will integration test against real external systems or black box unit test if the logic is self contained.

This sort of drives out a world where slow tests are kept to the leaf components, or the e2e acceptance, and everything else is quick with confidence in all the paths.

It certainly does lead to a world where refactoring is more costly because of the burden of these mocks but this isn't trying to follow Chicago TDD where the refactor step is part of the cycle, instead looking like London TDD where the testing puts pressure on the design itself, so refactors are far less common.

> (testing taxonomy is bad)

So true, it is terrible!

What I say is:

Unit tests - small tests that show the intent of one line of code or small set of lines, what the developer thought it was meant to do - can't prove correctness of an entire process

Integration tests - test one specific process works (login success, login fail, etc.) starts to prove correctness of one part of an application

Acceptance tests - larger, test that processes interact together and prove that multiple processes work

Production tests - (roll out to x users and monitor or something similar) - prove that the entire application works, in production.

Tests go from: Small to large, fast to slow, no risk to complete risk

I take the perspective that tests - and particularly unit tests - are a living specification for the software that you know cannot be out of date. If you write tests in a way that you understand the business purpose of the system, you are providing a significant part of the documentation while also providing a regression suite that is automated.

There are other ways to handle this. You can have a design document that is continually updated. Some teams can do this well. You can use literate programming a la Knuth. (Many code bases have abbreviated forms of this.) You can assume that the small number of developers who have been around for these decisions will never leave the company and forgo all of it. (I do not recommend this.)

So, what is your preferred alternative to unit tests as a specification? (And if your set of unit tests don't provide clarity to the source code, that may be a source of the frustration.)

I'm wary of "testing theater" (in the spirit of "security theater"); but I've come to think of testing as similar to two-factor authentication: it doesn't guarantee correctness, but it does reduce the likelihood of bugs and regressions, especially during refactors.

I think the other benefit of testability has nothing to do with the tests themselves, but rather the discipline of writing testable code: in general, writing code that is easy to test will tend to be higher-quality and easier to reason about.

The thing I'm not fully sold on is mocking, which ends up being a huge timesink, and may or may not improve reliability since you're testing against a fake system and not the real thing. I vastly prefer a combination of small functional/unit tests, and E2E integration tests in a real environment (cypress/etc); the uncanny valley in between has a poor ROI IMO.

This is a good point. Just like with anything in engineering understanding the process is more important than just following some steps because you think you should.

When developing software there are many steps before testing even occurs to catch problems, and the earlier you catch problems the better. Coding standards, having requirements, and peer reviews all are important too.

I find tests useful for having a "checklist" of things to do before releasing a new build. In robotics automated tests are especially helpful since there is a lot of code which only happens in certain physical conditions which are hard to recreate manually (i.e. in a low battery condition the robot should do this behavior). But just having the checklist is more important than how you execute it.

> This is a good point. Just like with anything in engineering understanding the process is more important than just following some steps because you think you should.

Music to my ears. We've been all at some point or another being followers to the cargo-cult. But taking a step back and evaluating things from first principles greatly help getting one's self out of being stuck in a rut. At the end of the day, there are no silver bullets. Everything has trade off, just pick whichever is the least worst :)

Spot on with cost, as with everything you have to be pragmatic.

Tests are great for:

* High risk items (large consequence when it goes wrong)

* Documentation

* Weird unintuitive things

We had a C# project recently that needs to detect changes between a DTOs properties. At implementation, all the comparisons would be done over value types, but if someone added a reference type that didn't properly implement equality this would silently fail (likely for months). Good case for adding a test that ensures the change detection works for all properties.

I used to be a TDD zealot. In recent years, I've taken a much more selective approach to test coverage. I typically focus my testing on pieces of code that contain business logic whereas I used to test everything. I've also found automated UI testing is not worth the squeeze and I've had better luck just looking at impacted objects and manually testing those.

I'd be interested to hear if anyone has automated UI testing tools in place that are easier to write test cases for than to just do the manual testing.

I’ve had really good luck with react UI testing using Jest and React Testing Library and throwing in UI screenshot testing.

At my previous job we eliminated selenium from our testing suite. We found that good UI unit and integration testing caught 99% of the bugs and the bugs that did make it through most likely wouldn’t have been caught by selenium so the added time wasn’t worth it.

IMO, snapshots, like Jest/React, are one of my favorite ways to test UIs. They require minimal effort and address the major points of testing a UI.

Actual pixel "perfect" UI testing should still be done from time to time in a real browser. It's nearly impossible to properly capture some of the layout differences/bug that can come simply from a new browser version.

We ran screenshots and looked for a percentage same. Because 100% is almost impossible. Where it saved us over snapshots was we were using MUI and they introduced a breaking UI change that our snapshots didn’t catch the change. And the screenshot did because the menu was no longer hidden. But the code hadn’t changed.

I'd recommend taking a look at Cypress. https://www.cypress.io/

Maybe you can help me understand this.

Since you don't write as many tests, that means you're not actually testing all your code branches because tests incur technical debt after all. So does this mean you test every single branch manually? just don't bother with it at all? Do you just have a few integration tests and they break and you spend a good chunk of time figuring out which logical branch broke?

What happens if you make a typo, comment out a piece of code and forget to uncomment , etc?

I'd love to write less tests but don't know how to do it?

I write tests when I know the answers before to some complicated code.

A good example of this is writing some code to tell if a point is inside a triangle or not. Setting up the test case is easy with three points for the triangle, a point or two inside, points outside, etc... Writing the code is simple enough, but possibly prone to flipping a sign for a slope, or maybe translating the equation wrong.

If I'm writing a web app, I just don't bother to write them before hand. I mostly code until I get what I want and then write the test. I think of this as "nailing it in place". I want to make sure through the test that this code continues to produce what I expect. I'll write as many tests as I feel are necessary but probably not explore every branch.

Yeah, as soon as code has more than two "real" branches, I don't trust myself to manually test them all. One of them will be broken quickly if I keep hacking in a particular branch. (This is, secretly, also an argument for writing code in sufficient generality to avoid this phenomenon in the first place.)

I also never trust a test that passes the first time I run it. I am both terrible at writing correct code, and completely normal in that regard.

I believe he's using the term branch different than you : 'alternate code path'

No, alternate code path is exactly what I mean. What's the functionality I'm coding in support for today, in addition to the stuff I was supporting yesterday? How do I know the stuff from yesterday still works? TESTS!

Most of the typoos and obvious errors are caught by IDE thanks to intelligent code completion systems and linters. If you're writing in a compiled language - compile step might catch some of the more obvious errors too.

I manually test my current branch manually while developing and do sanity checks when it's merged somewhere. I do write tests for some cases, but rarely, only to save time when I have to test against a wide range of input parameters.

> What happens if you make a typo, comment out a piece of code and forget to uncomment , etc?

Thanks god this is handled out of the box by any compiled language. I can't imagine myself writing tests to catch non-logical bugs.

It's weird you bring up global warming when there is scientific consensus that it is not only happening but man made. Doesn't seem like a good analogy for something you want more data on.

I believe they're using it as an example of a thing that people have very strong, immovable opinions about, making it difficult to discuss. It just so happens that in the case of climate change one side is just objectively wrong, and its opinions are factually incorrect... but that just underscores the OP's use that it's impossible to discuss in public with them. They hold strong opinions in spite of overwhelming evidence against them, indicating strong feelings that will dominate any conversation you try to hold.

I accept the reality of global warming (I'm almost a single-issue voter on the subject), but I look at it from a Bayesian point of view: if we compared two universes, one where AGW was real, and one where the scientific consensus was mistaken (not exactly unprecedented), popular opinion would probably vary by no more than 1% between the two.

Humans tend to start with moral intuitions and tribal affiliations, and then cherry-pick data to support them; that irrational force is at work even in the cases where tribal values-signaling happens to align perfectly with reality.

Have you ever wondered why it became partisan to begin with?

Lots of other issues are not partisan like "smoking causes cancer", everyone generally agrees that it does, even many years ago before the "science proved it".

There is some science that is so obviously manipulated by power and money everyone knows something is wrong, but can't do anything about it.

I believe in this case the partisanship comes from less from the topic itself, but if you look into the proposed solutions, they all tend to shift money and power in a single direction and don't have much to offer in regards of actually changing the climate.

I suspect that's a product of anything that becomes political: whether it's climate change or border security, the incentives are more towards theater than results, creating an attack surface for economic exploitation.

At this point, I think the most efficient (and least corruptible) solution is a carbon tax + dividend [0], AKA a Pigovian tax [1]. It's not as though markets don't work; the problem is that externalities, ecological ones in particular, are not priced into the system, meaning we're just passing the bill to our descendants, at a phenomenally high interest rate.

https://en.wikipedia.org/wiki/Carbon_fee_and_dividend

https://en.wikipedia.org/wiki/Pigovian_tax

Do you think corrupt governments, that also happen to be the largest polluters in the world will participate with a carbon tax honestly and in good faith?

Yes, I suppose that is a risk; as with our (allegedly) progressive-taxation system, a carbon tax could invite loopholes and creative accounting, both in public and private sectors. It would take a significant groundswell of grassroots political will to apply a carbon tax fairly and universally; perhaps the dividend would make it a little more salient and less abstract, making it easier to calculate that company/department X is taking $Y out of each citizen's pocket, as opposed to more convoluted federal spending, regulatory capture, etc.

> It would take a significant groundswell of grassroots political will to apply a carbon tax fairly and universally;

This is where carbon tax falls apart. There is ZERO insentive for poor people in high polluting countries to make their lives worse for the benefit of the world. None.

Therefore, carbon tax will only ever be paid by the "rich", but then we /know/ the rich dodge taxes. So the carbon tax falls on the middle class in the rich countries.

Now the politics are more clear. Is it any surprise that the middle class don't want carbon taxes? Only the middle wealthy (doesn't affect their life much), the poor (they don't pay any taxes) and the rich (actors, politicians, etc...).

This is why Trump is popular, a whole class of people is being dumped on, it's not a conspiracy from the oil companies, it's ideology based on fantasies that dump all the costs on one group of people.

A new plan is needed, carbon tax will _never_ work.

It's a legitimate concern; but the majority of carbon-producing activity is on the production side, and can be taxed from the producer, or even as a "carbon TCO" VAT on the consumption side. While tax dodges are a risk (particularly in the open, as big corps lobby for exceptions), this makes it much more universal than something like tax havens for capital gains by a small number of the uber-wealthy.

I would consider the world-wide implications to be a bigger concern: if the tax is too high, some producers might feel incentivized to move operations to a country without a carbon tax. Though I generally think tariffs are a bad idea, a carbon tariff that essentially covers the difference (and goes to the same dividend pool) might not be the worst idea.

Yes, the perverse incentive of the oil lobby is transparently staggering. Even if one is charitable, that the lobbyists believe their own talking points, "it is difficult to get a man to understand something, when his salary depends upon his not understanding it".

That said, it didn't help the politicization of the climate for Al Gore to have become its face, thus associating its brand with Democrats, for all his pleading that "this issue is moral, not political". He meant well, but it was painfully naive.

There are success stories in the past about how things have improved. And it seems like the only way was when leaders didn't frame the issue to focus on themselves or their party, but about the people and the country.

From what I understand about TDD it's more about getting the design right then testing for bugs.

TDD is also a design tool. I agree for newly written code, yes.

Writing a test before fixing a bug reveals the bug and solidify a proof that the fix addresses it adequately. So in this context, it's less about the design (since it's already there) and more about fixing and proving that the fix works.

TDD is about getting design right, but it's also about ensuring that the design's functionality is defined, demonstrated and protected from regression.

I've honestly thought I was crazy for thinking that required tests are ridiculous. Thank you for confirming that I'm right in my beliefs.

At least 50% of my last job was writing tests, and the snail's pace of their dev process was the main reason I left.

> Anecdotally, I've worked for companies that test super heavily, and I've worked for companies that had no automated tests at all.

How did refactoring work at that latter company?

Those companies were mostly C#, with a bit of Ruby and JavaScript. The C# type checker generally made refactoring a non issue. In the case of JavaScript and Ruby, it was not much harder. Mostly find and replace. Sometimes a bit more work, depending on the complexity of the change. In general, the code bases were pretty well designed and modular enough that refactoring was pretty simple.

I tend to be a repl-driven developer, and I also use the product as I code. If you've ever watched Jonathan Blow programming on YouTube, that's similar to my process.

I find the majority of issues as I go. The bugs that slip through tend to be the surprising edge cases where I think I'm just modifying a single vertical, but there's some bleeding over into other verticals which I'm not using while developing. These sorts of bugs can usually caught by a handful of basic integration tests.

very good point.