Hacker News Reader

Comment by diekhans | Hacker News Reader

My view is that automated testing is not a substitute for QA, but an additional tool. It lets QA focus on harder to automated tasks.

For unit tests, a developer is going to try something out anyway, so capturing it in a unit test for the future should be just a little extra work. Also, writing a unit test means the developer has minimally used what they are writing.

Higher-level (system) testing, especially with GUIs, can be more work than the value added. It is a cost trade-off, but ultimately, a human adds value not matter how much automated.

AI will help too, but these are also tools. Drop QA and you are trading off costs for quality.

IMHO, not backed up by research.

View on HN

Absolute one is not a replacement for the other.

I worked at a place that did pretty strict TDD and had a dedicated QA person embedded on each team. Our high-level systems tests severed more as a smoke tests and only ever tested the happy paths. Our integration and units tests of course covered a lot more, but QA was essential in covering corner cases we never thought about as developers.

There is one programming language that fascinated me (maybe it was Ada) where it tried to have some basic tests inline with the code, by defining basic guidelines for legitimate results of the function.

For example, you could make a function called `addLaunchThrusterAndBlastRadius` (I know it make no sense, but bear with me), and then right alongside declaring it was an integer, you could put a limit saying that all results that this function can return must be greater than 5, less than 100, and not between 25 and 45. You could also do it when declaring variables - say, `blastRadius` may never be greater than 100 or less than 10, ever, without an exception.

I wish we could go further that direction. That's pretty cool. Sure, you can get that manually by throwing exceptions, but it was just so elegant that there was just no reason not to do it for every function possible.

Modern C++ supports this pretty extensively via the type system. You can define/construct integer types with almost arbitrary constraints and properties that otherwise look like normal integers, for example. The template / generics / metaprogramming / type inference facilities in C++ make it trivial. Some categories of unsafe type interactions can be detected at compile-time with minimal effort, it isn't just runtime asserts.

This is common in C++ for reliable systems. You infrequently see a naked 'int' or similar (usually at OS interfaces), almost all of the primitive types are constrained to the context. It is a very useful type of safety. You can go pretty far with a surprisingly small library of type templates if the constraint specification parameters are flexible.

(This is also a good exercise to learn elementary C++ template metaprogramming. A decent constrained integer implementation doesn't require understanding deep arcana, unlike some other template metaprogramming wizardry.)

You can do that in Swift (and, I suspect, lots of languages).

Swift has a fairly decent assertion/precondition facility, as well as reflection[0]. They also have a decent test framework[1] (and I know that some folks have extended it, to do some cool stuff).

Some of these add significant overhead, so they aren't practical for shipping runtime, but they can be quite useful for debug-mode validation.

Assertions are a very old technique. I think I first encountered them in Writing Solid Code, in the 1990s. Back then, we had to sort of "roll our own," but they have since, become integrated into languages.

Of course, all the tools in the world, are worthless, if we don't use them.

[0] https://developer.apple.com/documentation/swift/debugging-an...

[1] https://developer.apple.com/documentation/xctest/

> Of course, all the tools in the world, are worthless, if we don't use them.

True...

I wonder if there would be any way, to simplify the syntax, and require basic assertions to be on every function. There might be a super easy cop-out like `any`, but at least by being forced to type it, you become aware of what it means and that it exists.

Almost like:

`public any int addXandY (any int x, any int y) {`

I also wonder, if there could be such a thing as an `assertion exception` (or whatever it would be called). Maybe it would just make things a mess, but I'm just thinking out loud. Basically, you could have a function that behaves a specific way 90% of the time, but for that 10% of the time where it doesn't work, you could pass that assertion exception to override. Maybe that would just be awful... or it would keep functions much cleaner?

Maybe you wouldn't even call it an exception. You'd just have multiple sets of assertions that could be applied to each function call.

I just had another thought. What if you could have a bank of assertions? Like this pseudocode:

```

assertion acceptableBlastNumber (int x) { x < 25; x > 5; }

assertion acceptableBlastRadius (int x) { x > 500; x < 1000; ! (x > 750 && x < 800) }

assertion acceptableBlastAddedNumber (int x) { x < 1025; x > 505; }

public acceptableBlastAddedNumber int addBlastNumbers (acceptableBlastNumber int x, acceptableBlastRadius int y) { return x + y; }

addBlastNumbers (10, 720) => 730

addBlastNumbers (26, 750) => Exception

```

Though I suppose that this is getting really close to just... classes. It would just be a little more... inline? Less complicated because it would never hold state? Though I suppose, this would also mean your class can just focus on being an objec, and not on having all the definitions for the things inside it, because you can have an <assertion> <object> rather than just <object>.

Preconditions and postconditions around procedures. I thought that it was an innovation from Eiffel, though wikipedia lists Ada as an influence, so maybe it did originate there!

> My view is that automated testing is not a substitute for QA, but an additional tool. It lets QA focus on harder to automated tasks.

This is my take as well. Automation is great for a lot of the repetitive work, but humans are better at creatively breaking stuff, handling UX testing, and improving and enhancing the automation itself.

Reply on news.ycombinator.com