Hacker News Reader

I'm still firmly in the "no" camp on testing private methods. By definition, anything your class does in the real world is done via its public interface; I'm not sure why I need to care about what's done under the covers.

I've actually moved further and further away from unit testing over the years (after being a pretty big TDD fan for a long time). In terms of bang for buck, integration tests across your public API are the best IMO. You're testing how your API is actually used. The problem historically has been that they're fragile to refactoring and difficult to run, but with the right tooling you can get around that.

I find the opposite wrt. refactoring - that is to say integration tests are _more_ robust to refactoring than unit tests, and that's one of the reasons I strongly prefer them.

Unit tests are deeply coupled to the internal structure of your system - refactoring often implies changing unit tests, which opens the door to bugs where the code and test both change to match each other.

As you say, integration tests validate your public API, which from a 'correctness' point of view is really the only thing you care about, not the internal structure of the system. That's why I love integration tests, you can make sweeping refractors without needing to change the tests, because the test will still tell you whether the behaviour of the whole system is correct.

Yes, especially for some unit testing is done like "atomic testing" (test the smallest possible part, if they could they would test individual lines), but turns out that the putting together is another crucial part with often not less complexity and where the corner cases come in. In that sense even not much like the unit vs integration testing difference & debates, because that line is blurry and just depends on your definition of an unit: It may be the single heavy algorithmic function, it may be the whole state machine, it may even be two heavily dependent and cooperating tasks in an embedded RTOS... and you always know you are in trouble if people discuss more what an unit test is and not is (instead of the value whatever thing provides).

I "love" btw those code bases rich with tests but 99% of the time not finding a bug, but breaking on the tiniest detail change.. not supporting refactoring like it should be but making it slow and everyone wanting not to refactor because of the crap test rat tail.

Integration tests are robust with respect to process flow/business logic, but (can be) fragile with respect to the mechanics of calling the API. Back In The Day that used to be a real issue because the tooling for writing API integration tests was separate from the tooling for writing the API itself.

These days (at least the way we do it) the integration test code gets refactored alongside the rest of the code, and compile-time errors catch 90% of issues.

That's part 1 of what sold me on integration tests. Part 2 is the ability to simply and efficiently spin up a DB per test, removing the need for mocking the persistence layer.

I’m wondering if the software world has something to learn from digital design here (for once), where there’s a huge emphasis on code coverage collection.

You really shouldn’t care how a function is tested. That there exists a test for that particular function isn’t particularly useful measure of how well the function is tested. So you need a methodology that tells you which of your functions have been covered by your tests. If it’s not covered you should make a judgement about how to get that coverage.

Getting your private methods covered by the public interface is good because it encourages you to write thorough tests for those public methods that covers all the ways the method might be used.

I agree a lot with this, and want to add that if you have a lot of complex private logic in a class it might make sense to extract an abstraction from it (if possible) and make it its own class for which you can then test the public (previously private) methods.

First thing I'd say is that there's no "one size fits all" approach. The right answer depends on the language, the project, the requirements, the amount of churn, the timeline, etc. So I 100% believe you when you say you've found success by moving the focus toward implementation tests. =)

But in my experiences, I've had major issues with a reliance on integration tests. This is almost exclusively in Rails.

- They're slow, because there wasn't an easy way to stub out the slow stuff

- If you stub out the slow stuff now you have no coverage for the stubbed stuff, unless you also have unit tests covering the guts of the app

- When integration tests fail, it can take a lot of work to investigate the cause, as opposed to granular unit tests

Our default approach is to mock and stub as little as possible, but of course there are things we just have to mock, like external service calls. However, we also have UI automation tests that run on a fully deployed and integrated system, which mostly covers that.

Slowness is only a problem when it's a problem, and can sometimes be solved with better tooling. I'd feel more confident in a system with 1000 integration tests that take 2 hours to run over one with 10,000 unit tests that take 10 minutes to run.

On your last point, this is positive and negative. Yes, it can be harder to find the issue, but IME integration tests actually find the issues that will cause things to break in production, and it's easier to find out why a test is failing than why an error is being logged.

Agreed, there isn't unlimited time to perform testing. Sometimes you also know when you're writing tests that will never help find a bug, and that's a waste of time.

I always focus on integration testing. If I have time I'll write unit tests, but there needs to be a good reason. I'll unit test an area that has proven to be buggy even with integration tests, but this is rare.

Finally, some code is just really complex and/or critical. These are good candidates for unit tests.

Agreed, too :)

The one place I still use unit tests is around calculation algorithms where you want to push a lot of different data through. You could do it with an integration test, but the test run overhead of say 100 parameterized integration tests is far too high. In these cases the algorithm will still be exercised by an integration test, but the nitty-gritty will be done in a unit test.

Integration tests are great, but the very real downside is that they tend to be much more complex to write than unit tests, and can in some cases require setup and tear down overhead that makes it impractical to lean on them for the majority of testing logic.

Unit tests for each unit of functionality, and a small number of integration tests to validate all works well together.

Again, I think it comes down to tooling. Our app is basically a document management system with quite complex business rules. I can write a test that takes a document from inception to completion in about 15 LoC because we've put time into our test framework to provide sensible abstractions over our API.

Setup and teardown can be an issue, but we've found a good balance. We spin up a clean DB for each test in less than a second. Running out full suite does take a couple of hours, but that's what CI servers are for.

I think unit tests can be divided into two camps as well: there are the trivial ones that are very close-coupled to the implementation, like, check if my ad hoc stack implementation will actually add the element and return sensible outputs. Something like this can easily be a private method as well, and I think they make sense to include right next to the function itself. Some languages allow these reality checks right in the documentation itself - and I don’t see who would it hurt. They are closely coupled to the implementation so sure, they will have to be rewritten with it, but that’s completely fine.

The usually called unit tests indeed should not directly test implementation details, like private methods, but then we have to find a different name for the former because I do think they are useful sometimes

> Unit testing private methods.

Adding the single biggest reason I have encountered: Because the overall functionality is too complex to test through the API alone, so the alternative is to extract part of the "private" functionality to a separate class that would then be public itself, making functionality public that shouldn't be. By that, the discussion about "testing private methods" ignores the elephant in the room, which is that the language used doesn't have a rich concept of public vs. private in the first place.

As evidence for that I'd like to mention that I never had that problem in Rust, because I can simply extract the private functionality into helper functions within a private sub-module and unit-test them there, without that functionality becoming public to the rest of the code.

If your class is doing so many things that you can't reasonably test everything but through the private methods, how about extracting a helper for those things which has testable public (or package private, if you have to) methods!? Did I just sneakily bypass the rules, or make the code better overall? I don't know anymore...

> If your class is doing so many things that you can't reasonably test everything but through the private methods, how about extracting a helper for those things which has testable public

When your solution to not wanting to write tests for important behavior is to trick yourself into wanting to by adding extra layers of code for no other reason, maybe the rule stopping you from doing it was wrong.

I acknowledged right away that it was a hack. And if it wasn't clear, by that I meant that the private method rule shouldn't be hard. It's can still be a smell, though, but I'm trying to be pragmatic.

It's not about that imo. It's about having tests that are limited in scope as much as possible, so if they fail it's much easier to understand the problem.

Ideally tests should be run in a specific order, from most low level to most high level. So tiny basic functions first, then functions that use those functions, then functions that use THOSE functions, etc until you have end-to-end tests.

Unfortunately I know of no testing system that has such a hierarchy.

Basically every decent test framework has a concept of test suites which can be used for this purpose.

Could you name one concretely and show how a suite will do this?

I agree that if your private functionality has grown complex, that's a strong "code smell" that perhaps it needs to be extracted.

But yeah, then you've changed it from private to public.

In my experience refactoring and making things public is an entirely reasonable thing to do!

I often find that when I have a very complicated structure with lots of behavior, there’s some other cohesive unit of functionality living inside. That part is often private, but extracting it into a separate structure with public interfaces makes complete sense. From the perspective of consumers of the original class, all of this is still implementation detail that can’t be directly accessed. But there’s nothing wrong about accessing that underlying bit directly through its own public interface.

Private methods can benefit from knowledge about the class they live in, and be only good enough for their purpose in that class. When elevated to public methods, you may want them to be more defensive than when they were private, which is an investment that doesn't pay off in itself. People may also be tempted to reuse your public methods now, even if they were never intended for that. Java has package-private for that (I guess). I'm not a fan of package-private, but then again, I'm writing small micro-services and considering the code inside this service "friendly" to my class, i.e. not abusing public methods without looking inside them to see if they're up for the job.

That's not really what I'm talking about at all. Private methods are often useful for handling internal details. But often when you have a lot of private methods, you're bundling too much functionality into one object. I have found that it is very common that I get significant improvements in readability, testability, and comprehensibility when I decompose a large structure with a lot of private methods into a few smaller structures that operate on each others' public API.

Why you should test private methods: To make sure they work correctly. This may be significantly easier than testing the public methods that use them.

Why you shouldn't test private methods: Because the implementation might change, obsoleting the tests. But everything might change (except the end-user requirements, haha), so by this argument you shouldn't test anything except what is in the spec/what the user will see.

The only benefit of not testing private methods, is the same benefit that not testing anything else brings.

I've actually become a bigger fan of unit testing private methods over time. :shrug:

Yeah - the alternative is often to make your non-test code worse to make it more testable.

Test private methods when you have sufficient behavior in them that it's worth testing in isolation.

This is all there is to it. The dogma around not testing private methods is entirely unconvincing. I have some very complex private methods that it’s just easier to write some thorough unit tests for, and it guarantees the contract my public methods rely on. I really do not see why that is controversial.

If you don't have time to write good tests, maybe consider not writing any tests. I've worked for many a startup where we didn't have any tests because we rewrote everything every few months.

And that was ok because everyone was happy to have a lot of shit written fast.

I guess maybe what I'm trying to say is that shitty tests are sometimes worse than no tests.

I don't agree that private method tests are shitty.

However, I do agree that maybe there is a time during which it's OK to just not have tests. If you are just spitballing it, prototyping, code jamming, maybe even blasting out an MVP for alpha or beta testers. Sure.

Part of mastering a craft is making reasoned decisions about breaking rules.

I can see a process of triaging and deciding that some code isn't worth testing (yet), but the broader idea of "this is just a prototype", I'd like to push back on, unless you work in the rare place where they actually start over after the prototype. "Blasting out an MVP," you say? That should be production code then, with production-grade testing. I too work for a startup, and well-tested code allows us to release often and with confidence. I am humbled every day by unit tests slapping my wrist when I attempt something stupid.

> unless you work in the rare place where they actually start over after the prototype.

Every line of code will last some amount of time. Some lines will be maintained in production for decades. And others will be thrown out in a few weeks. For example, if I’m mocking up a UI, the css I write won’t outlast the mock-up. Or doing a game jam. Or prototyping a different way to structure some code.

Unit tests are useful for long lived code but they slow down your capacity to do scrappy iterations. Whether you need to prioritise short term velocity or long term reliability depends on the needs of the project you’re working on.

Code just tends to live longer than developers often anticipate. They put more trust in their organizations than is warranted about how much time they will be given later to go back and clean things up. (This probably happens because the original work is over-estimated and then the project is delivered under in a crunch, instead of having some breathing time to clean things up.) But again, this is all situational, you may work in different circumstances than I ever have.

I have given up on being annoyed or proselytising. It's much less stressful.

I write the documentation that I would like to see myself.

> Code alone can only tell you the "what." If the "why" is not obvious, comment it.

But isn't that the common view? I think most programmers would agree that the "why" deserves documenting. The problem is that in the heat of the moment, most forget.

Just to intensify the sacrilege here: I’ve been happy testing private behaviour with tests directly alongside the code. This way you can easily sweep away the test code if you change the implementation, and it serves as decent documentation for complex implementation details. The tests obviously disappear in production builds either way.

    I’ve been happy testing private behaviour with tests 
    directly alongside the code.

I'm intrigued but I don't understand. What does this look like? What language is this?

In Clojure, for example, tests are defined by a macro that decorates a function with certain metadata marking it as a unit test. So they can live and be run from anywhere, but get stripped out of proper builds. You can also go one further, and wrap a function definition in a `with-test` block, so any assertions you make against the function are kept directly alongside its definition, which is a nice form of documentation.

What this means is that you can have tests for implementation details, that are cordoned off from your main test suite (which defines your code's real requirements and public API). The benefit is that if you change implementations and suddenly a function is no longer needed, you can just delete the test code right there in the same file, so refactoring is very slightly easier and more obvious. You can also decide whether they should be run during your build process or just at dev time.

That's interesting, thank you!

No one has ever said that you shouldn't comment your code.

"Comment the how, not the why."

When you read software engineering books like Clean Code, they are generally heavily in favor of writing good comments.

When you read programming books, like "Learn Language XYZ" books, there are generally no comments whatsoever because there is no need. Which makes sense in the scope of such books, but I think it accidentally sets a precedent for eschewing comments in the minds of many.

But.

Out in the "real world?"

I'd say 90-95% of coders don't comment a damn thing. At least that has been my experience working in the industry since the 90s.

Some coders are in the obnoxious and toxic "code should be self-explanatory" camp, which is objectively dumb. Code can't explain intent, such as business rules or weird shit you're doing to work around weird shit in external libraries or APIs or hardware bugs.

A greater number of coders are of the mindset that inline comments are bad and that things should be explained in the commit message and/or pull request. This is more noble, but I think it is not nearly as practical as inline comments for a variety of reasons.

Many coders also believe that comments make code harder to read. I find this baffling and dumb. Get a bigger monitor or get an editor that lets you collapse/hide comments with a keystroke.

A great deal of people have. Uncle Bob and his substantial following, for example.

Many of his followers perhaps, but at least in the Clean Code book there is a long chapter on comments, where about half is dedicated to "Good Comments" that include many reasons for when and how to comment code. Yes, it has a section titled "Comments Do Not Make Up for Bad Code"... but he also writes about how there "are certainly times when code makes a poor vehicle for explanation" and that some comments are "necessary or beneficial" like "Informative Comments" or "Explanation of Intent" etc. That chapter is one of the few I remember as somewhat useful in the book (I am not much of a follower or fan... but I did enjoy his Clean Architecture book).

I have a tendency to comment code that isn’t optimal and will need to be refactored for performance at a later date.

Not sure how many times I’ve been looking into a new performance issue as we grow, and come across one of my own comments.

Sort of falls into the Good/Fast/Cheap trifecta. When you have to do the Fast/Cheap bit, note when you know it’s not optimal so you can identify it quickly when you or someone else comes back to it.

Unfortunately there is/was a very loud group that said exactly that. The opinion was that code should be self documenting.

I have heard many people say this.

That's backwards. The code is the how, the why is what comments can add.

I don’t think you shouldn’t comment your code, but in my experience, good code doesn’t need much, if any, comments.

This is because I define “good code” as simple code. If code gets to the point that it needs to be explained with comments, then there’s a good chance there is opportunity for simplification.

In other words, if I can’t read code and follow what it’s doing — or why it’s doing it — then I won’t ask them to add comments, but to instead refactor/simplify the code. (This is assuming the code can’t be made more readable with simple name changes, which a lot of times is all that is needed.)

Yes, there are times when the “why”, even with simple code, needs to be explained. Comments in this case are fine. But ideally this is the exception and not the norm.

There are numerous cases where code needs comments. People who think identifier names are all you need are frankly frustrating. It forces co-workers to do reverse engineering any time they want to understand the code.

The most obvious case are API docs which includes internal APIs (means: any public method, in my codebases). Comments are much faster to read and understand than the source code, can be quickly pulled up by using the right hotkey in your IDE, and can be browsed via generate API doc pages in a browser. I don't want to read your function to figure out what it does, I don't want to guess how I'm supposed to use it from the name. I want proper docs.

The second most obvious case is where the reason for a piece of code isn't obvious. Typically this means bug fixes or workarounds, but can also be performance optimizations. If something isn't explained by a comment then sooner or later it's going to get accidentally changed. Unit tests aren't a complete replacement because you don't necessarily know what tests are for what lines of code, and often it's hard to unit test an obscure bug fix anyway (e.g. workarounds for OS level problems).