Convicting the guy is one alternative, but there's another (non-mutually exclusive) option: punish whoever conducted an illegal search. If a warrant legitimised the search, punish the judge instead. If the judge didn't know any better because he was deceived, punish whoever lied to the judge, and so on.
"Might not get a conviction" is a negligible deterrent against police overreach.