We tend to romanticize Chess as being a hyper-intellectual game where if you are good enough, you should be able to beat anyone worse than you 100% of the time. The truth is a lot more subtle. Chess is indeed a perfect-information game, so there isn't true luck involved, but due to the fact that we can't just search the entire game tree, we introduce several elements of uncertainty. Namely, that past a certain point, you must play your opponent as much as the board. Players (and AIs) naturally have certain strengths and weaknesses, and the ability to steer the game into positions that play to your strengths and opponents weaknesses is often just as important as your overall game strength. Some of the best players in the world are known for how intensely they prepare for specific opponents.
In short, Deep Blue had 'preparation' against Kasparov from all his game histories, while Kasparov did not have the same benefit. While switching the code could be considered analogous to switching to a whole new opponent (though likely only a small change in practice), the overall unpreparedness was probably the larger factor. In my opinion Kasparov was a stronger player than 1997 Deep Blue, but this asymmetry probably cost him the series.