Comment by dpc_pw | Hacker News Reader

Comment by dpc_pw | Hacker News Reader

Parent Full thread

I very dislike Cypress as well.

This even model is sooo confusing, clumsy and error-prone that writing any non-trivial test is a challenge. Whole product uses async/await or Promises and suddenly you have to switch to this baroque model just to write a test. Reusing logic between production code and tests is pretty much impossible.

Dynamic pages etc. often confuse cypress. I'm actually not a frontend developer, so I don't have a lot of insight while exactly that is, but I just know that it has been a huge drag.

Other than that, it would be OKish. Being able to test the UI is great, but I would advice everyone not to base the whole testing strategy on it. It's just too heavy, slow and unreliable. It doesn't not scale even moderately with the product growing.

e2e testing is better done on API level, IMO. You'll want redesigns, changes of UX, and suddenly your changes will require updating tests, which otherwise (if done on the API level) would require no modifications.

It’s possible (and indeed uh ... recommended) to write frontend tests that are resilient to DOM changes by not depending on things that are not essential to the functionality being tested. Eg select a form field by its label, not by its position in the DOM, and use specific data attributes if there’s no better choice. Then when you gut the HTML for UI overhaul, if the functionality hasn’t changed, the cypress test doesn’t change, you just make sure you attach the attributes to the right new elements.

Of course if the functionality changed (like you split something up into 2 pages) yeah you need to update your UI tests cause you are testing a different experience now. But they are UI tests, so yeah. That’s appropriate.

I dunno. It sounds like you found other parts of it unwieldy as well, but the idea that the tests are inherently brittle is wrong. Time spent maintaining UI based testing is saved many times over when compared with the manual clicking around that it saves imo.

related: testing-framework has been adapted for cypress. It encourages this kind of thinking and testing.

I've used cypress at work but have suspended it since we used it end-to-end and it clogged up backend databases.

Can’t you set up a test database instead of testing against production? And set up timeouts to control the tempo of the tests if necessary?

> Then when you gut the HTML for UI overhaul, if the functionality hasn’t changed,

How often website redesigns are only about html? Usually everything changes. New flows, new dialogs, new toolbars, new frameworks.

Often the web UI gets and Android UI and iOS UI siblings. Gets replaced altogether by a completely different UI, developed by a different team.

APIs change much less often, and can be shared between many UIs. You can also version them. How are you going to version a web UI? Like reddit with "turn back on classic look"? How long can you keep the old UI around just because you have 1000s tests around that use it.

> but the idea that the tests are inherently brittle is wrong

I don't think so. Tests are great calcifyiers (check my blogpost about it if you want). If you test though your UI, you're calcifying your UI. The more you test though it, the more you calcifying it. No way around it.

https://dpc.pw/be-warned-tests-are-great-calcifiers

I’m not against testing APIs. But testing APIs does not tell you if your latest CSS change broke pointer events on your “add to cart” button, so you can no longer receive orders, etc.

If you are wholesale redesigning/rebuilding a whole site and all the workflows yes of course you need new tests. But also your users would get pretty impatient if entire ways of working are changing all the time in a way that makes testing a giant moving target.

The purpose of a UI test in my mind is to make sure that the core business things a user is supposed to be able to do, are doable. In the context of your blog posts, I think those things should be calcified.

Like, I want a test to fail if I remove a workflow that used to be there. I want a test to fail if a form field suddenly has no label. There should be those alerts when functionality that was previously understood to be correct has been changed. Lightweight UI tests with some easily re-approved screenshot diffs goes a long way.

The idea behind the code being async is that you don't know how long things will take, especially talking to a backend, or even sometimes DOM manipulations in dynamic pages like you mention.

Having said that, I find parts of it v confusing (although not the asyn stuff) especially around assertions - there seem to be multiple ways to do things and I'm never clear which is which.

As a sibling reply said, try to write cypress (or other similar product) tests not to rely too much on layout - I use html data attrivutes rather than text or Dom based selectors, so the tests can survive most redesigns, and if they don't, it's because the user journey has changed enough that we _should_ be changing the tests.

There are much simpler programming models to deal with long-running actions, like processes and threads. The essential point of async is to be thread-like, but with lower overhead; or in environments where you cannot use processes and threads - but there's no very convincing reason for testing to be it. And choosing to prioritize dealing with runtime overhead over dealing with coding complexity isn't an obvious win. Note that promises make sense for processes and threads too; the key question is whether you really block, or virtually block.

Nothing wrong with async code; but it's often sold as the "obvious" solution to, well, several things happening and some of them taking long - and it's just not that great a solution.

Would threads or processes be a great way of simulating how a user interacts with a web app though?

I tend to think of them clicking things and waiting for a response then moving on to the next thing when that happens (or failing the test when it doesn't) which seems to fit well with an async model.

Cypress _does_ do a lot of magic behind the scenes to stop you explicitly using async/await or promises in the code... Which occasionally bites it.

Especially when you have to use a .then to use values you pull from the Dom.

You can wait in a threaded model too, right? The only difference is that if you do so, you're blocking that thread. The pain comes when you mix the two models, because async often deals really poorly with blocking. In general, the differences between the two models when used cleanly are quite small. When most waits are no-ops, threads have significantly lower overhead; but when waits are common, especially if long-lived, async overhead is much lower.

In fact, if you want to minimize the chance of deadlocks, using threads or processes for every action is safest; although of course that means you may get issues with concurrent modification. But though that risk is serious, it's worth noting that async code doesn't actually solve it; after all, every await may mean a context switch, and thus if you have shared mutable state living across an await, all bets are still off. async-only code with exactly 1 thread only provides safety to the extent that modification is encapsulated and that the module within which the internal datastructures are updated does not use async. And that's not nothing; it's kind of as if a java object was synchronized on all methods. But it's also not reliable enough that you can just forget about concurrent modification, either.

But if all you want to do is automate a browser, those overheads are completely irrelevant, and this story about concurrent modification likely is too. You could use either model equally well. To be explicit: there's nothing about promises that means you need to use async. It's only once you start chaining a bunch of them that you enter async territory, but why would you ever do that in a threaded model when automating a browser? The browser is so heavy weight, just fork a bunch of processes already, and KISS: no need to worry about any of this, at all.

Reply on news.ycombinator.com