Hacker News Reader

Comment by frogulis | Hacker News Reader

Aw man, I was nodding along with the "most examples suck" section and then... it ended :(

I'll give you the good example I've been doing for the last two decades: testing a compiler.

The complexity here is the complete opposite of the simple toy examples. What are the edge cases of an optimizing compiler? How do you even approach them, if they're buried deep in a chain of transformations?

The properties are simple things like "the compiler shouldn't crash, the compiled code shouldn't crash, and code compiled with different optimization levels should do the same thing." This assumes the randomly generated code doesn't touch undefined behavior in the language spec.

Here's a recent example of a bug found by this approach. The Common Lisp code stimulating the bug has been automatically minimized: https://bugs.launchpad.net/sbcl/+bug/2109837 with the bug fix https://sourceforge.net/p/sbcl/sbcl/ci/1abebf7addda1a43d6d24...

Unfortunately I have no idea what I'm looking at here.

The function constructs two lambda expressions (source code for anonymous functions) that should be equivalent. One has some extra declarations. It then compiles the two lambda expressions and calls the compiled code on the same arguments, and gets different values (which is the bug).

Here's a great talk by John Hughes, one of the authors of QuickCheck, with real-life examples:

https://www.youtube.com/watch?v=zi0rHwfiX1Q

Thanks, that's gotta be one of the best talks I've ever watched. A passionate speaker, talking about fascinating, useful tech, giving specific real world examples of utility, AND showing how to actually apply it for interesting "dirty" situations.

They should put some good ones in the article.

Reply on news.ycombinator.com