Ask HN: How has AI changed your QA validation workflows?

The feedback cycle got a lot more segmented, as we try to catch different types of issues and get feedback as early as we can. For example, UX review is now before everything. Claude builds static demo pages for how the app will look once the changes finally land, and then there's an audit script which runs through those to measure contrast/visibility/accessibility for all elements in focus/hover/dark and light theme etc. when that passes, a human looks at the demo pages to check if the design makes sense, and iterates on that. once a human is happy, the implementation work starts. before AI, UX review was usually part of exploratory testing after implementation, but now it's both cheap and necessary to do upfront. The demo pages become a part of the spec, and they are a baseline for automated testing at the end.

Lots of rules and checks evolved from common feedback and now work in custom linter rules, custom checkers, custom-built rule engines to prevent Claude from making similar mistakes.

Similarly to you, we have a "demo video" skill that records a flow through the app demonstrating newly build functionality so a human can look at it at a higher speed and notice if there's something obviously wrong, before doing exploratory testing.

I've written two QA skills for our app: 1) flow discovery, and 2) e2e implementation.

The first one scans the app and detects critical flows, happy paths, unhappy paths, invariants. Writes an invariant matrix across app domains and surfaces, and maps acceptance criteria to each.

The second actually implements the flows as e2e tests (Playwright for web, Maestro for mobile). Tests are run locally before a PR is submitted, with a video recording. The video + screenshots are posted in the PR body. A rule in CLAUDE.md says "No PR w/o visual evidence!".

CI runs the suite depending on touched code.

It's nice to lay back and watch the runner click through your app.