In my case it's not the CI that's the bottleneck. It's the human testing side. Does it work, sure. But does it actually do the thing we want (and more importantly) does it do it in a way our customers will understand and actually like?
Engineers seem to fall within a spectrum between "building" (enjoy the task of designing and programming something) and "completing" (enjoy seeing a finished product). AI enthusiasm correlates neatly with that dimension.
QA is about quality. That it has been increasingly used to mean "repetitive manual testing only" is part of the very trend you decry of qa being undervalued.
Fundamentally QA is about two groups of people collaborating in an adversarial process ("you build it we break it") to achieve a common goal: a better product. QA by definition requires a relationship of equals, or the process ceases to be adversarial and becomes useless ceremony. If you're not able to assemble two distinct groups of people, one person can wear both hats (ie you test your own software) but you'll have more blind spots.
How much of QA is done by a human or automated, is and has always been an implementation detail.
If you're in charge of QA, you're in charge not just of finding new problems, but preventing regressions as well. And you're responsible for embedding as much of that work into the devloop itself, so that builders can find and remediate problems earlier and with less pressure on your limited time. How do you achieve that? automation.
I started my career doing QA and 99% of what I was doing could be automated by an AI today. Today I still do QA on my own product- I just spend all my time on the remaining 1% and my product is better as a result.
Despite all that, it was understood if you want a high quality product you will be spending more than half of your development budget on testing.
I have long ago concluded that it is not possible to test my own code - and nobody else can test their own code either. I know how to test code - like you I started in QA - but I can't test my own. I have too many blind spots because I'm too close to how it works.
I actually think that for a lot of software, AI can really help with that blind spot, even beyond regressions, in ways that don't replace human testers but are complementary.
For starters, not all software is used directly by humans. A good chunk was already consumed by APIs, and now an even larger chunk will be consumed by AIs. In those cases, it's very possible that AIs will be better at QA than humans, even for original problem discovery. After all, they are the users.
Even for human-facing CLIs (the kind of software I develop these days), it's trivially easy for AIs to interact with the software, and in my experience the newer models are shockingly good at understanding the principles and conventions of CLI usability, at least in the Unix world. I routinely run CLI changes through a gauntlet of AI agents, and their feedback is genuinely good.
TLDR I think in these debates over how automatable QA really is, we often tend to forget that there is a lot of software out there, and not all software is tested the same way.
Quality assurance != Manual testing.
For all the vibe bros "!=" is "not equal"; a little programmer humor.
What you cannot automate are finding the "unknown unknowns". That is things you don't know to look for. I have yet to see a project where humans running the automated tested code cannot find large large number of bugs that slipped passed all the automated tests that already pass.
You absolutely automate every test you can. It is boring and tedious to run through a manual test plan. What you want is to tell the testers "find what is wrong with it" and let them figure where/how to look. The act of finding new ways to try things is a large part of what makes manual testers better than automated tests.
BTW, PR review is a flavor of quality assurance activity. People are arguing it can be replaced with AI review.
AI can likely do a lot of QA work, but I still want a real human to test the code. At least code that humans are expected to use.
What exactly are you using to back up this claim?
> QA reviews [...] fully automated
The idea is to drive actual testing from this, but in this era, I think it's interesting as a way to use AI to generate tests, and to cross-check those tests with the natural language descriptions, in a bit of a cycle that helps refine the highest level definition of the software.
Once that's nailed down, the implementation is just details.. normal engineering concerns like maintainability etc notwithstanding of course, but you can trust more and more the AI agents to get it right. The design specs being natural enough for humans to deal with but interpretable/specific enough to actually generate tests is pretty interesting for the bottleneck you are talking about, I think.
At some point, with all this velocity, human users become the bottleneck, unable to keep up with and learn all the changes and new features. Luckily, there's a simple solution: just replace the human users with agentic AI users.
> Henry Ford II: Walter, how are you going to get those robots to pay your union dues?
> Walter Reuther: Henry, how are you going to get them to buy your cars?
Because every developer is now a slop cannon by default, by default users will experience churn and whiplash, and things will break all over the place. As you point out, this is bad. It's also impossible to fix without deploying agents on the QA side. Like it or hate it, agentic testing is inevitable to protect users from the churn and noise caused by the slop cannon. I don't think that replaces test engineers at all - if anything it makes the job more fun. If you've ever had to keep playwright tests in sync with the target manually, and kept the CI environment up to speed with toolchain changes, you know what I mean.
Whether the "slop cannon by default" situation could have been avoided in the first place, is another question... But we're here now and there's no going back. Might as well deal with it as best as we can.
TLDR: it's not all bad :)