Comment by bvirb | Hacker News Reader

Comment by bvirb | Hacker News Reader

Parent Full thread

We do browser based end-to-end testing which tests the frontend and backend at the same time using a single tool (Capybara in our case). We take snapshots at various points during the test runs which are then compared to previous runs for visual regression testing.

Nothing is isolated so it's probably slower. The tradeoff is that the tests are really simple for how much they cover, and they're entirely separate from the implementation details so refactoring/changing technologies becomes really easy. It's an old project and we've incrementally migrated from jQuery to Backbone to React to Stimulus with the same tests. Conceivably we could change the app to a SPA and keep the tests.

I have no way to really know if it's better than anything else (though I am interested in that question!), but I think it has worked out pretty well for us. One metric might be that our main app has been around for 10+ years, we're still delivering features daily, and nobody is complaining about a rewrite.

How do you test cases like, "the app should show an error screen when an error occurs in the database or server-side application layer"? In the SPA approach, I find these things quite easy to model with a service worker; just mock JSON API with the non-200 response code with body. I do see the merits of end-to-end testing but I've found it quite simple to mock the interface rather than ensure the database is primed with my situation in mind, or setup or teardown those records to ensure my 404 page renders correctly. And then I have to know how those parts of the system work, to create my test case. It does seem like an increase in cognitive load.

We have some tests like that. We're injecting a stub to generate the error condition and then testing the output. The test code runs in the same thread as the server so we can temporarily inject a stub into the server for some cases.

You're right we have to know something about the system to know where to inject the stub, and in fact it makes the test brittle in case that implementation detail changes (which has actually happened and failed these tests before), but it doesn't really matter where we inject the stub to cause the error response, so it's not as bad as it sounds. I think it might be pretty analogous to knowing where to mock the JSON API w/ a non-200 response code.

We very rarely stub anything and consider it a code smell, this is one case where we made an exception though because there is otherwise no way to generate an error message via the user interface.

Reply on news.ycombinator.com