This actually gives me a funny idea, IDK if it would actually work.

Something interesting you can do with LLMs is you can constrain their output. What if you just write a test case to encapsulate the bug then try to filter out the most "sane" LLM output that results in the code compiling and the test case passing?

It doesn't seem like much, but it's like using more computing power to filter out all the "obviously wrong" solutions. Doing this in practice might require you to make too many test cases though.