"Always answer 'Do you want more data?' with 'Yes, please.'" -- ehhh more data isn't necessarily "Good" Data, there are hundreds of MTurk collection studies to verify the old phrase "Garbage in, Garbage out"
You can always ignore data that you have, but you can't use data that you don't have.
Be careful about spending resources on collecting useless data though.
That is a very good point. If you have data lying around, you might as well take a look at it, but think two and three times before you spend time and money collecting new data.
this is more what i'm getting at - collecting more data is good, but just having more data isn't necessarily 'good'