Writing end-to-end tests is always a sucky and flaky experience, and it always ends up slowing down CI. With that said, I've used testcafe without any big complaints.

My general rule of thumb is to try having as few end-to-end tests as possible. It's fine for an e2e test to cover multiple aspects of your application. Trying to write e2e tests as if they were unit tests just leads to sadness and infinite CI builds.

I don't think there's any value to running all your tests on the browser. It's much faster and simpler to run the bulk of your tests on node, with a fake DOM environment.

If you're interacting directly with the DOM, you might want to consider breaking that off as a separate lib. From the lib's repo you can have all of its tests run on a real browser.