The problem is the cost of doing this. If it gets overturned, the defendant shouldn’t be the one to bear the cost. I’d be happy with either 1) the patent owner or 2) the patent office, who didn’t do its job properly.
Your comment seems to relate more to unfairness in [lack of] awards of costs in USA legal proceedings.
If you sue me for something I didn't do then it shouldn't cost me anything; that seems reasonable whether the domain is IPR or any other aspect of life.