Hacker News Reader

Comment by tayo42 | Hacker News Reader

Unit tests catch that kind of stuff

The code works perfectly - there is no issue that a unit test could catch... unless you are spying on internally created objects to a method and verifying that certain functions are called some number of times for given data.

Sure and you can do that

Trying to write the easiest code that I could test... I don't think I can without writing an excessively brittle test that would break at the slightest implementation change.

So you've got this Java:

    public List<Integer> someCall() {
        return IntStream.range(1,10).boxed().toList();
    }

    public List<Integer> filterEvens(List<Integer> ints) {
        return ints.stream()
                .filter(i -> i % 2 == 0)
                .toList();
    }

    int aMethod() {
        List<Integer> data = someCall();
        return filterEvens(data.stream().filter(i -> i % 2 == 0).toList()).size();
    }

And I can mock the class and return a spied'ed List. But now I've got to have that spied List return a spied stream that checks to see if .filter(i -> i % 2 == 0) was called. But then someone comes and writes it later as .filter(i -> i % 2 != 1) and the test breaks. Or someone adds another call to sort them first, and the test breaks.

To that end, I'd be very curious to see the test code that verifies that when aMethod() is called that the List returned by SomeCall is not filtered twice.

What's more, it's not a useful test - "not filtered twice" isn't something that is observable. It's an implementation detail that could change with a refactoring.

Writing a test that verifies that filterEvens returns a list that only contains even numbers? That's a useful test.

Writing a test that verifies that aMethod returns back the size of the even numbers that someCall produced? That's a useful test.

Writing a test that tries to enforce a particular implementation between the {} of aMethod? That's not useful and incredibly brittle (assuming that it can be written).

You are correct and the objection is just completely invalid. There's no way anyone would or should write tests like this at the client level.

I think they are just arguing for the sake of arguing.

You mention the tools you can use to make it happen.

I think we're at the point where you need concrete examples to talk about whether it's worth it or not. If you have functions that can't be called twice, then you have no other option to test details in the implementation like that.

Yeah there's a tradeoff between torturing your code to make everything about it testable and enforce certain behavior or keeping it simpler.

I have worked in multiple code bases where every function call had asserts on how many times it was called and what the args were.

In functions that you write, that might be possible.

How would you assert that a given std::vector only was filtered by std::ranges::copy_if once? And how would you test that the code that was in the predicate for it wasn't duplicated?

How would you write a failing test for this function keeping the constraint that you are working with std::vector?

    std::vector<int> doThing(const std::vector<int>& nums) {
        std::vector<int> tmp1;
        std::vector<int> tmp2;
        std::ranges::copy_if(data,
                             std::back_inserter(tmp1),
                             [](int n) { return n % 2 == 0; }
        );
        std::ranges::copy_if(tmp1,
                             std::back_inserter(tmp2),
                             [](int n) { return n % 2 == 0; }
        );
        return tmp2;
    }

I know how I would do it in python. This is built into the stdlibs testing library, with mocks.

Maybe dependency injection and function pointers for the copy if function. Then you can check the call counts in your tests. But idk the cpp eco system and what's available there to do it.

The python code would be

    def some_call():
        return [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
    
    def print_evens(nums):
        for n in filter(lambda n: n % 2 == 0, nums):
            print(n)
    
    def func():
        filtered = list(filter(lambda n: n % 2 == 0, some_call()))
        print_evens(filtered)
    
    if __name__ == "__main__":
        func()

How would you write a failing test that prevents the list from some_call() from having the same filter applied to it twice?

You would use something like this

https://docs.python.org/3/library/unittest.mock.html#unittes...

Then you would make the mock filter with patch and test the `func` function

psuedo python code would be

   @patch(builtins.filter)
   def test_func_filter_calls(mock_filter):
     mock_filter.return_value = [2,4,6]
     func()
     mock_filter.assert_called_once_with([1, 2, 3, 4, 5, 6, 7, 8, 9, 10])

That wouldn't fail though. It was called only once with [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]. The second time it was called with [2, 4, 6, 8, 10].

Likewise, if some_call returned [2, 4, 6, 8, 10] instead, it should only be called with [2, 4, 6, 8, 10] once then.

However, the purpose of this test then becomes questionable. Why are you testing implementation details rather than observable? Is there anything that you could observe that depended on the filter being called once or twice with the same filter function?

Did you try it? If it doesn't work there's also called once if you scroll up on the doc

And as far as whether it's a good idea or not, I generally wouldn't, but was saying when it is important then you do have these tools available,llms aren't the first thing to check for these mistakes. It's up to the engineer to choose between trade offs for your scenario.

Yes. The test passes. https://imgur.com/a/4qlTKlc

To try to monkey patch this in, you would need to also assert that it wasn't called with [2, 4, 6, 8, 10].

At which point, I would again ask "why are you testing that it _wasn't_ called with a given set of values?"

The comment at the root of this is "Unit tests catch that kind of stuff".

... But unit tests aren't for testing internals of implementation but rather observable aspects of a function.

Consider if the code was written so that it was

    def print_evens(nums):
        for n in nums:
            if n % 2 == 0:
                print(n)

instead (with the filter being used in func())

This isn't something that unit tests can (or should) identify. It would come out in a code review that there is redundant functionality in func and print_evens.

Using ChatGPT or another tool to assist in doing code reviews can be helpful (my original premise).

https://chatgpt.com/share/697a64a6-33c0-8011-a0f8-ca4fec74ab...

ChatGPT properly identifies the duplicated functionality (even though the code is using different idioms for doing the filtering for even numbers).

The test pass because your patching print_even so the 2nd filter is never called.

Which I guess, idk maybe think through the testing more and your code more before jumping to conclusions about how things are?

Testing is one tool you have, and it can test the internal like this. Obviously there's a use for it if its in the Python stdlib

This is in mockito

https://stackoverflow.com/questions/39452438/mockito-how-to-...

this is in google testing library for cpp

https://google.github.io/googletest/gmock_for_dummies.html

> Specify your expectations on them (How many times will a method be called? With what arguments? What should it do? etc.).

If you never heard of this, I guess you learned something new? Im not a tutor though. I would read the docs more and experiment. Maybe chatgpt can help you with how tests can be written.

With Mockito, I can mock the returned result of someCall().

However, it also means mocking list.stream() and mocking the Stream for stream.filter() and mock the call stream.toList() to return a new mocked object that has those mocks on it again.

I could catch the object passed in to printEven(...) but that has no history on it to see if filter was called on it before.

Trying to do the filter(...) call would be especially hard since you'd be parameterize it with a code block.

And all this returns to "is this a useful test?"

Testing should only be done on the observable parts of the function. Does printEven only print even numbers?

The tests that you are proposing are testing the implementation of those calls to work in a specific way. "It must call filter" - but if it's changed to a different filter or if it's changed to not use a filter but has the same functionality the code breaks.

Inefficient? Yes. Bad? Yes. Wrong - no. And not being wrong it isn't something that a unit test could validate without going unnecessarily into the implementation of the internals for the method. Internals changing while the contract remains the same is perfectly acceptable and shouldn't be breaking a unit test.

youre verifying std lib function call counts in unit tests? lmao.

You can do that with mocks if it's important that something is only called once, or likely there's some unintended side effect of calling it twice and tests woukd catch the bug

i know you could do it, im asking why on earth you would feel its vital to verify stream.filter() was called twice in a function

You're not verifying the observable behavior of your application? lmao

what happened to not testing implementation details?

A redundant filter() isn't observable (except in execution time).

You could pick it up if you were to explicitly track whether it's being called redundantly but it'd be very hard and by the time you'd thought of doing that you'd certainly have already manually checked the code for it.

How would you suggest tests around:

    void func() {
        printEvens(someCall().stream().filter(n -> n % 2 == 0).toList());
    }

    void printEvens(List<Integer> nums) {
        nums.stream().filter(n -> n % 2 == 0).forEach(n -> System.out.println(n));
    }

The first filter is redundant in this example. Duplicate code checkers are checking for exactly matching lines.

I am unaware of any linter or static analyzer that would flag this.

What's more, unit tests to test the code for printEvens (there exists one) pass because they're working properly... and the unit test that calls the calling function passes because it is working properly too.

Alternatively, write the failing test for this code.

Idk how exactly to do it in cpp becasue I'm not familiar with the tooling

You could write a test that makes sure the output of someCall is passed directly to printeven without being modified.

The example as you wrote is hard to test in general. It's probably not something you would write if your serious about testing.

In C++, the code would look like:

    #include <vector>
    #include <iostream>
    #include <algorithm>

    std::vector<int> someCall()
    {
        return {1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
    }

    void printEvens(const std::vector<int>& nums)
    {
        std::ranges::for_each(nums, [](int n)
        {
            if (n % 2 == 0)
            {
                std::cout << n << '\n';
            }
        });
    }

    int main()
    {
        std::vector<int> data = someCall();
        std::vector<int> tmp;

        std::ranges::copy_if(data,
                             std::back_inserter(tmp),
                             [](int n) { return n % 2 == 0; }
        );
    
        printEvens(tmp);
        return 0;
    }

---

Nothing in there is wrong. There is no test that would fail short of going through the hassle of creating a new type that does some sort of introspection of its call stack to verify which function its being called in.

Likewise, identify if a linter or other static analysis tool could catch this issue.

Yes, this is a contrived example and it likely isn't idiomatic C++ (C++ isn't my 'native' language). The actual code in Java was more complex and had a lot more going on in other parts of the files. However, it should serve to show that there isn't a test for printEvens or someCall that would fail because it was filtered twice. Additionally, it should show that a linter or other static analysis wouldn't catch the problem (I would be rather impressed with one that did).

From ChatGPT a code review of the code: https://chatgpt.com/share/69780ce6-03e0-8011-a488-e9f3f8173f...

> You could write a test that makes sure the output of someCall is passed directly to printeven without being modified.

But why would anyone ever do that? There's nothing incorrect about the code, it's just less efficient than it should be. There's no reason to limit calls to printEven to accept only output from someCall.

Reply on news.ycombinator.com