I assume that the random trace generator produce structured output and not random garbage that's rejected most of the time, right?
Yeah, the generated traces are 'sensible' by construction (they don't compute anything sensible, but they are well-formed).
Thank you, makes a lot of sense. I was working on a structure aware fuzzer, and needed to know if there are even projects out there that don't use some form of grammar to produce output.
Downvoted for a question, people these days...
This is very interesting! Were these bugs reported to the Python tracker? Also, what are the actual implications of these bugs in real code? What happens when such a bug is genuinely hit? Are there just performance penalties or are there actual miscalculations in the end result?
Note that these are bugs in pypy and not in "Python" i.e not in the dominant python implementation.
Right, thanks for the clarification!
There would be an actual miscalculation, yes. which is a really annoying kind of bug, also because these jit bugs tend to go away if you turn the debugger on, making them even more frustrating to track down.
The ones we found so far are all extremely unlikely to occur in 'regular' Python code, because they require the use of internal pypy specific numerical operations.
Have you tried using "holes" in the Z3 solver to generate faster optimizations or guards yet?
Yeah, that's my current project, already found a bunch of missing small features in the optimizers.