Hacker News Reader

If you prefer coded tests, there is no good alternative to the de-facto standard of using Selenium and/or Webdriver. But as you said, this means that creating a test setup is a development project by itself. So for our team, screenshot-based web automation tools like Kantu (https://kantu.io - or Sikuli https://sikulix.com for the desktop) are the best alternative to Selenium, especially for non-standard and animation-heavy web-app/pages like the one you need to automate.

I find that screenshot-based test cases are very easy to read and maintain - even and especially for the non-coders on your team. New button? Just take a new screenshot. Everyone that can use Snagit understands that ;)

I did an experiment with the Chrome Debugging Protocol the other day. It allowed me to write and run acceptance tests in pure JavaScript with no Selenium, Java or binaries (except for Chrome of course). See working proof-of-concept here: https://www.npmjs.com/package/websteps

>I like my web app acceptance tests to be code, readable and maintainable, not UI/OCR thingies... but I've seen so many bugs like "this element is visible and clickable, but Selenium says it's not, because the 3d-css animation + dom-manipulation voodoo of the tested app got selenium completely confused"... and I'm not even mentioning web code with events-on-SVG-nodes or webgl here...

I just write javascript directly for automating situations like this.

I see this as somewhat analogous to the abstractions provided by an ORM. It won't provide every edge case but that doesn't mean you should toss it out entirely - just bypass the abstraction in the (minority) of cases where you need to.

If you need to bypass the abstraction too much of the time that indicates that something is a bit wrong with your front end code, just as having a database that you could almost never use an ORM would indicate something fundamentally wrong with your data model.

I do wonder about the accessibility of pages like this. If it confuses Selenium, can screenreaders cope?

Ugly and simple truth: no. In real life, in a customer centric approach, you can choose to ignore even 50% of your users anyway, from a marketing pov, if you decide those are not the users you care about (hint: you care about those willing to pay way way more - like when you make an overpriced iOS app but completely ignore Android to give another example). Why care about... less than 1% of users, when you already chose to ignore an even larger percent in your other approaches? Fair? No. Makes sense? Yes.

Expose all data trough a clean and documented API. If screenreader accessibility matters, someone will eventually pay you to develop an alternate interface using same API, or an open-source will be developed at some point...

Solution to accessibility problems is usually: (1) expose your data though an API, (2) wait to be paid to develop a truly accessible alternate "light" UI (if it ever happens), but don't cripple the "main ui" (well, actually more like "have it like the customer wants it" ;) heavy animations and fx are "crippling" in my personal view too...).

I'm just saying that ignoring users and ignoring testers has considerable overlap in this regard.

(And apart from general and professional ethics, local laws might disagree with the assumption that you're free to ignore users for that reason.)

> (I know the "real solution" is "use a frontend framework and write front-end-unit-tests instead", but usually you're writing the acceptance tests for something that already exists).

Eh, GUI tests definitely have their place, although they can be far less specific than your unit tests. They're pretty great for quick and regular confidence checks.

Anyway, TestCafe says it doesn't use Selenium, but I haven't tried that yet: https://testcafe.devexpress.com/

I was about to suggest TestCafe even though I haven't tried either. After reading a bit, it seems to work with a small proxy that injects a script to trigger events on the page via the normal DOM api's. If might be a good solution if you're targeting modern browsers.

You may like something like Codeception: you code your tests and it drives Selenium. Code is in php but I'm sure you can find the same kind of things in your prefered code language.

For a more visually oriented IDE Sikuli seems to be a good start but last time I checked there was no easy way to automate its use for CI.

How about casperjs?

casperjs is similar to Selenium but it only works with phantomjs. That's useful in the sense that you can do a quick test to see if your code works in a browser, but it's a different prospect to Selenium. Selenium (with WebDriver) works with all the major browsers. If you're automating testing a website/app then you need to be testing it in lots of browsers.