Ask HN: How Do You Do IE, Edge, and Safari Tests for CI? | Hacker News Reader

Ask HN: How Do You Do IE, Edge, and Safari Tests for CI?

14 pointspaulddraper8 comments

Currently, I have dockerized Chrome and Firefox tests. Run locally, run in CI...easy.

But now I'm looking at the default browsers of the most popular desktop OS's. And it looks bleak.

How do people run IE, Edge, and Safari tests? As far as I can determine it's absolutely impossible to even run Safari in AWS (the current location of my CI infrastructure). There's other services but I hesitate because they look rather expensive when I compare them to my current setup.

Any suggestions? Third-party services, open source tools...whatever. It seems crazy it's so hard to test in web dev 2019.

View on HN

I also use BrowserStack for web browser (desktop) and mobile apps device front-end based testing and have generally had good experience technically, with 'Browserstack Local' I found getting both web and mobile apps access to internal staging environments was straightforward. Unfortunately I have found their billing system quite inflexible. We had a minimal annual plan and wanted to create a new plan on the same account but this time on monthly billing but we could do not do that because we already had another subscription on annual. Even after speaking with support and explaining we were looking to increase our spending by 10X multiplier we could not do this and were told to create another account, not ideal because then there are multiple api keys.

As for the test executioner/runner I have a custom Node.js application using Mocha test framework, and for reporting I use Tesults (https://www.tesults.com) which I recommend to everyone of course because because I think it's the best way to consolidate and report test results and I'm the founder :) Send me a message for anything reporting related and I'll do my best to help.

BroswerStack. There are a few competitors to this, but sorry can't remember their names. Also Selenium. You could run Windows VMs on Azure for testing IE/Edge if you wanted and run Selenium on them.

Another route might be Jasmine tests, run them on Karma. Run them on a physical machine with the target browser. Not so great for CI but OK if you are doing things on a shoestring and want a quick check before you deploy. You could even run them on a public URL, go to a mates house with a Mac / Windows PC, and just visit that URL and wait for the green light to appear.

I use Firefox, the other dev uses Internet Explorer, and the last dev uses Chrome. All kidding (but not kidding) aside. We’ve had luck with Selenium. As for the platforms, we actually have a Mac Mini, 2012 MBP, and two iPads that we use for testing Safari and Safari Mobile. Unfortunately that part is mostly manual since few of our customers use Safari.

A quick search for "Cross browser testing CI" brought up quite a few options. Sauce Labs seems to have an automated plan (https://saucelabs.com/pricing)

Have you had good experience...flakiness, coverage, availability, etc?

We use webdriver.io (https://webdriver.io/) to test FF, Chrome, and IE. We don't test Safari.

BrowserStack- It's expensive but it does it all.

SauceLabs, it ain't cheap, but easier than dealing with the licenses

Reply on news.ycombinator.com