Hacker News Reader

Comment by ambrop7 | Hacker News Reader

This is bullshit. You can write tests for TLS libraries just fine. And yes, I'm saying that without ever doing any programming of TLS libraries (just using them).

For certificate verification, you craft various correct and incorrect certificates to cover the various scenarios involved in certificate verification.

For communication related code, you talk to the tested code over TCP or internally as a transport layer implementation (assuming the TLS library supports user defined transport layers). Same as above, you identify the various code paths and attempt to test them all.

> for the same reason that the kernel TCP/IP stack isn't built with TDD

I believe the primary reason why testing kernels is hard is that they are intended to run on real hardware and often with strict timing expectations. This is not the case for TLS libraries, which normally run in user space.

View on HN

If you knew every possible incorrect certificate format, or even every possible form a valid certificate could take, your comment would be more compelling. But of course, you don't. Nobody does.

I think this is unnecessarily pessimistic. You know what validity checks the code performs. To catch this bug it would only have been necessary to make each of them fail and make sure the failure led to rejection.

It's true, that doesn't prove the implementation is perfect, which seems to be what you're holding up as the goal. But it would have caught this bug.

If you're writing that code, you SHOULD know every possible incorrect certificate format. If you don't, you should learn from the relevant standards and other materials, or admit that you're not capable of writing that kind of code.

But in this case this is not about that. It's about testing code already written. All you need to do is go through that code and verify that every branch does what is supposed to do. Which is pretty straightforward, if time consuming.

> If you're writing that code, you SHOULD know every possible incorrect certificate format.

If we define a correct format there are likely an infinite number of incorrect formats, no? A test explicitly checking for this bug would prevent a fix from regressing, but it seems to write a test that exploits this bug before understanding the bug itself would require quite a bit of luck.

EDIT: I'm re-reading the initial advisory and trying to decide if this applies to any cert with a ROOT CA fail or just a specifically crafted one. If it's the former my initial comment is garbage.

If you classify incorrect formats by what they get wrong, you should be able to limit the number. For example, in terms of C strings one invalid format is 'embedded null'. It doesn't matter if that null comes from jpg data or mp3 data because those distinctions don't exist at the 'C string' level.

Reply on news.ycombinator.com