Hacker News Reader

Comment by johnnypangs | Hacker News Reader

What do people thing of playwright vs cypress? I've been considering using playwright instead as it supports more browsers and I feel like it's easier to do production monitoring (by putting it in a aws lambda or using checkly)

- Cypress: https://www.cypress.io

- Playwright aws lambda: https://github.com/PauloGoncalvesBH/running-playwright-on-aw...

- Checkly: https://www.checklyhq.com

View on HN

My main issue with Cypress was the fact that it required a shift in mental model, i.e. queuing commands rather than executing them, thenables that aren't actually promises, running as client side JS as opposed to in a controller script. There were a ton of sharp edges and required a lot of education to adopt.

Meanwhile Playwright's API is just promises. Everybody knows how to compose promises.

The Cypress approach does in theory completely eliminate latency-induced flakiness, but in my experience, Playwright/Puppeteer's usage of bi-directional real-time protocols to communicate with the browser makes latency low enough that latency-induced flakiness is no longer a practical concern, especially when paired with the ability to selectively run test logic in client-side JS in the page with 0 latency.

Selenium did suffer from latency-induced flakes all the time due to its slow uni-directional request/response model. I personally believe the Cypress model is an over-correction for Selenium PTSD, and isn't making as good a set of tradeoffs compared to Playwright/Puppeteer.

The irony is that the Cypress model is extremely similar to the Selenium Version 1 model. We abandoned that approach in Selenium V2 (WebDriver). Time is a flat circle.

I am leading a transition at work from Selenium to Cypress precisely because of our frustration with the WebDriver model. I don't consider myself an expert enough to know which approach is right, but could you elaborate on why Selenium chose to divert in V2 to what it is today?

This. Similarly how cypress does clicking with JavaScript which leads to weird bugs rather than with the debugger - similarly to old versions of selenium

We see a strong adoption of Playwright. It’s the default we recommend to users now. We also support Puppeteer, but its development is lagging.

Having said that, I would love to support Cypress if I had infinite time and focus.

Side note: Selenium is not on the menu, even with its large install base.

We are aiming for where the puck is going and it’s going to Playwright.

Full disclaimer: I’m CTO at Checkly.

The checkly folk recently published this

https://blog.checklyhq.com/cypress-vs-selenium-vs-playwright...

Cypress vs Selenium vs Playwright vs Puppeteer speed comparison

Why was Selenium off the menu?

Selenium is slow and based on webdriver. I think most consider it legacy at this point. Most new projects tend to use Playwright, Cypress, or Puppeteer for E2E tests. All three options are much more performant and reliable.

I've been out of this space for a while, but isn't WebDriver the only cross-browser spec for browser automation? The last I knew, the browser vendors were removing other automation hooks for security reasons and heavily influenced the WebDriver spec.

Well, playwright is cross browser too.

I've only been using/playing with it for a few weeks, but I find it more contemporary and less clunky than selenium. And it's lighter and runs faster. I dislike passing around a driver to tests and the playwright built-in testing (which I think is wrapping the expect library) is pretty nice.

I haven't fully made peace with the selector API in playwright, which uses a lot of experimental css selectors and a custom DSL, but these are minor learning curve issues.

I'm sorry if not clear, but I was asking about the WebDriver spec [1]. The terminology gets confusing because WebDriver was a tool that then merged with Selenium and Selenium implemented the WebDriver wire protocol. If Playwright isn't using WebDriver, is there a new browser automation API?

Edit: It looks like it's using Chrome's DevTools protocol. I'll have to read up more on that. I thought Google deliberately didn't want that to be used for automation, but I'm probably thinking of something else.

[1] -- https://www.w3.org/TR/webdriver2/

I'm not sure if it's using (or can use) CDP for all automation or for chromium, but CDP became my go to for headless browser work in the last six months, it's bypassing the abstraction of puppeteer and has a lot more language support.

But it's not itself a test framework, nor is puppeteer, so it's nice having a more modern toolset on top.

fwiw, Selenium (in v4) now supports CDP. From the website:

"WebDriver Bidi is the next generation of the W3C WebDriver protocol and aims to provide a stable API implemented by all browsers, but it’s not yet complete. Until it is, Selenium provides access to the CDP for those browsers that implement it (such as Google Chrome, or Microsoft Edge, and Firefox), allowing you to enhance your tests in interesting ways."

https://www.selenium.dev/documentation/webdriver/bidirection...

WebDriver through WebdriverIO is actually really solid and I think more teams should give it a try. My understanding is you don’t need any Selenium tooling if you use it but can interface with projects like chromedriver directly.

It’s faster than cypress and you have access to a massive ecosystem as well of things that speak the WebDriver protocol, as well as the ability to do hybrid and native mobile testing (through Appium).

I’ve been really happy with it and we’ve decided to go all-in on it for apps in the Ionic ecosystem.

I saw puppeteer is actively developed, wonder where the 'lagging' implies.

doesn't playwright use puppeteer for chromium-based browsers(even edge-based), I thought it's just a wrapper for puppeteer with extra support for firefox.

How about Python bindings? Selenium is the only decent option in that space so far, are we wrong?

looks like playwright is bridging this gap too:

https://playwright.dev/python/

Playwright Python is excellent

Any insights on the flakiness of Playwright vs other tools you’ve used?

Cypress was such a pain that I simply wasn't writing or running tests just to avoid it. It was incredibly slow, had major bugs that are till open after multiple years, and worst of all, in headless mode it was passing tests that should have been failing (and were failing in headed mode). The only reason I discovered it was because headless mode seemed to run unusually fast.

I switched to Playwright 6 months ago - I'm much happier now and the switch also allowed me to delete 3/4 of the helper code that I needed before.

On my team we evaluated Cypress and Playwright and landed on Playwright. Some features it has that Cypress didn’t was Safari support, support for cross domain tests, and support for tests that need to open multiple browser windows (for testing collaborative editing).

I love Cypress. there are definitely limitations though like iframe support or visiting two separate domains, tab support

https://docs.cypress.io/guides/references/trade-offs#Permane...

- Cypress does not run on M1 natively.

- Playwright is more lightweight. Can be good or bad on what you expect. But I definitely prefer Playwright.

Cypress was terrible choice for us where flow in the system is managed by multiple users - as cypress can work only on single open window/tab, you had to effectively duplicate, triple etc. each test. Playwright also allows trivially control/crosscheck backend/whatever as you're in nodejs context, not browser context.

Playwright is definitely better IMO. Cypress is overengineered.

I wouldn’t say cypress is over engineered. Just a byproduct of its time.

What do you mean?

I highly recommend codeceptjs. After 4 years of using test cafe for e2e testing, codecept has proven to be much more pleasant to use.

There is also gauge/taiko https://taiko.dev/

We recently picked playwright for a project because we could do more with the "mouse". With playwight you can click coordinates on the page. I did not find an easy way to do this using cypress. Aside from that, playwright seems to be so much faster.

Reply on news.ycombinator.com