You also have to know you want to solve a problem.

Once you get to the point where you have any hypothesis whatsoever, no matter how weak, a systematic approach (saving examples as test cases) helps to avoid confirmation bias and makes testing further hypothesis less costly.

Another hard one is when there is a simple, probabilistic rule. You usually end up with an over-complicated rule to cover all your data instead of the true rule. (Of course that gets down to what is at the basis of the probability: are you satisfied with a probability?)