It seems like you’re just being obstinate. At some point in the game, you’re going to be so unlucky that it’s improbable you don’t succeed. If I’m making an absurd claim, you’re doing exactly the same thing.
You’re not making an absurd claim at all and I agree with it. But you’re still using the same term which denotes the need for luck, that term is improbable. By saying it’s improbable you’re not saying it’s a certainty, so therefore luck is still a factor in your success.