Hacker News Reader

Comment by zdragnar | Hacker News Reader

There is certainly a false sense of security in test-only approaches. You only have tests for what you expect to go wrong, so every wrong assumption you make becomes a bug. Dedicated QA can help catch these, and are practically a requirement if your customers aren't content with things randomly breaking.

My previous job was just that... one QA person for a 50ish person team, focusing only on writing user acceptance tests. The leads thought that if we just had enough coverage, we could deploy continually without needing QA reviews, and were continually surprised at the number of bugs getting through.

That isnt to say that automated testing is bad, but rather that it is easy to be over-confident in what it will do for you.

View on HN

>There is certainly a false sense of security in test-only approaches.

Assuming you mean automated-tests-only, then sure. But not only are full-stack approaches to testing encouraged by TDD best practice (i.e. unit to integration to e2e/BDD), it's also still compatible with doing functional QA, and I would recommend it. So I don't think we're in disagreement here, I think moderate amounts of functional QA and TDD are both parts of a healthy testing culture. (I'm a software testing and verification enthusiast, and formal methods have not yet exceeded the utility of these practices. When and if they do, I expect them to be far superior to both in most aspects, and fully replacing TDD with something like proof-driven-development.)

Not to mention that adding tests to a codebase vs starting with TDD from the ground up tend to result in pretty different results over the same time period; the former takes a longer time to start fully reaping the benefits.

Google Play allows for different "tracks" for publication: internal testing, closed alpha, closed/open beta, and production.

My approach is to keep releases in sync with Git branches (master is to production, staging is to closed alpha, etc). Although I don't often push things to master, I publish on any of these tracks at least once a day. I'm looking into the Apple ecosystem now, so I'm not sure if I can use a similar approach (my main concern is having a Mac as CI server).

If you already happen to be running Jenkins you can run an agent on the Mac and tag iOS builds to always run there. I’ve had decent experience with that approach, using Fastlane to build/upload different releases based on git branch like you describe.

Keep in mind you’ll need a graphical connection for occasional updates, which is likely different from the rest of the build system. You may also need it to fetch an Apple 2FA code every 30 days, that’s used to refresh your token to upload releases to App Store Connect.

Reply on news.ycombinator.com