> However, in a few cases, they seem to hit the nail right on the head
> Does anyone have any insight as to why some predictions turn out right? Is it just luck, or is there some greater principle that can help us find out what's ahead?
Individuals don't make a lot of forecasts/predictions, but society at a whole does. People who read into these forecasts get "informed" about the potential future, and perhaps even inspired (ie. a self full-filling prophecy; I'm not arguing that's the default though).
The default is as follows: if "one" (see above) makes a lot of forecasts, the likelihood one of them hits the nail on the head increases.
Luck is when a beginner with a crossbow hits the bullseye with one shot. Its just that if you shoot with a shotgun, you're gonna hit something, somewhere. But we're ignoring all the fails here, the misses. They're not relevant so they get ignored. History does essentially the same. The winners write history, and this is an example of that.
We should also not forget that a detailed, technical prediction by say a scientist is something different than say a suit who's trying to just sell their book with a bunch of rough predictions. Ie. not all forecasts/predictions are equal.