It's not just that it introduces uncertainty by trying to predict the future, it's that it contends that most important issues are the ones that effect people in the extreme distant future--which is the time that's the most difficult to predict, meaning the predictions have the largest possible uncertainty.

In other words, Longtermism tells you to give preeminent weight to matters you have no way of knowing the nature of or whether you're improving things or not. At that point, you might as well switch to astrology.