Were you using o3-mini before o3 came out? I wonder if o3-mini has had the same issues if it is from the same series of models.
That said: so far, I'm putting up with it because o3 is smart.
That test harness can be a human checking the results, it is more work for the human but it will solve more problems than without the hallucinations.