Just a thought: Whether a prediction is objectively good or not is different to whether the prediction is seen as good or not. If you can think of a mechanism to figure out whether a prediction was good or not after the event has happened, the system could then make a second order prediction about whether that user's a good predictor or not. Keep me up to date if you manage to solve any of these problems.