Comment by eyelidlessness | Hacker News Reader

Comment by eyelidlessness | Hacker News Reader

Parent Full thread

I’m not a fan of Jest, but this is good news anyway. If nothing else, establishing clear dedicated stewardship is a great chance to see if some of its limitations can be overcome.

I don’t mean to criticize anyone’s prior work but I know it’s better to be more specific, so my short list of problems I’ve experienced with Jest and reasons I won’t consider it for projects currently:

- Its mocking model is coupled to CJS semantics and static analysis and hoisting patterns built on Babel, and it’s hard to work around that especially in legacy code.

- Its isolation model is inherently prone to memory leaks and fragile tests, directly related to the above point. If you `import 'anything'`, in a test module or its dependency tree, you might think you’re initializing once but you’re initializing for each test. If that initialization even exposes a way to teardown it’s just sitting there waiting for you to know you need it in an afterEach, otherwise they run perpetually and never get GCed. Really surprising things like loggers and other seemingly passive libraries which you might develop internally suffer from this. This might all sound like whining, but it can make a large test suite basically useless because you can’t even observe the code that’s still running.

- I’ve watched ESM support improve and I’m impressed by the effort, but from my observation the above issues are major barriers to making its mocking and isolation strategies viable. ESM isn’t the reason they’re challenging, but cache invalidation of stateful imports is an exceedingly good way to reveal them.

Hi! Christoph from Jest here. Your feedback is great, and 100% valid. Jest was built a long time ago when ES modules did not exist, and we are still carrying some of that legacy baggage around.

The good news is that we have never been shy about making breaking changes and we are working on cleaning the house and making many legacy components optional, all while bringing the existing community with us.

As for mocking, you don’t have to rely on Jest’s inbuilt mocking libraries and you can use the ones you like better.

If you care more about raw performance and ES module support and less about isolation, check out the jest-light-runner: https://github.com/nicolo-ribaudo/jest-light-runner

We also mentioned it in our Jest 28 blog post: https://jestjs.io/blog/2022/04/25/jest-28

I’m wondering if it’s time to consider taking a big step and making this runner the default, and give people the optionality of isolation. However, in my past experience at large companies (both first-hand and second-hand experience), the lack of isolation in tests led to major reliability problems with testing infrastructure. I’m still feeling like it’s the better default today, but maybe we should have a serious discussion about Jest’s next set of defaults.

> As for mocking, you don’t have to rely on Jest’s inbuilt mocking libraries and you can use the ones you like better.

I was preparing myself to reply “thanks for the kind and receptive response, but…” I’m glad I read further before drawing a conclusion.

> The good news is that we have never been shy about making breaking changes and we are working on cleaning the house and making many legacy components optional, all while bringing the existing community with us.

> If you care more about raw performance and ES module support and less about isolation, check out the jest-light-runner

It looks likely this addresses the problem I was going to raise! I never wanted to use module mocking in the first place, and never did. But Jest at the time I used it didn’t give me an option to opt out of its isolation model and still imposed all of the problems whether I used module mocks or not. I think this (runner which doesn’t support it) is a great thing to provide, and yeah if you’re okay with breaking changes it’s a much more sensible default.

Thank you for not just the kind and receptive response, but also for showing clear direction from problems I raised! I’ll keep tinkering on my test framework as a hobby project because there are already major perf and semantic wins in it. But this definitely makes Jest a more viable day work consideration for me.

Edit to clarify: I agree isolation between tests is crucial, and isolating modules is an important part of that. It’s just… not what Jest is actually doing. It wasn’t when I was using CJS, and it’s much more challenging to mitigate with ESM. Jest very reasonably clears the require cache between tests, but that’s actually breaking isolation when modules have private state initialized on require. If I remember correctly the ESM solution is the same as my naive one: tack a query parameter on when re-importing.

Realistically, the only current viable solution to isolation is worker threads and WASM/NAPI hacks. Hopefully better isolation primitives are coming up through the standards process.

That seems great, definitely going to check it out!

Do you know if there's a way to use that runner per-file or per-project within jest? Seems like that's the only way it'd be possible to smoothly migrate in a large codebase without rewriting all tests at once.

Jest supports multiple “projects” in a single invocation, and they can all use a different reporter.

You can define a base config for your repo, and then inherit two sub-configs from it each with different runners and mutually exclusive selection for tests (by suffix, folder, or any convention that works for you).

See https://jestjs.io/docs/configuration

Thank you! We actually do use the "projects" feature, but we've found a fair number of options don't work when defined at the project level (a good example is `coverageThreshold` - we weren't able to make per-project thresholds work, at least in jest 26).

Glad that this one has the ability to be configured more granually - that'll come in handy for migrating gradually.

Reply on news.ycombinator.com