With diceware there's a list of 7776 words.
When someone rolls 1,1,1,1,1 and rejects that word they've cut down the word list to 7775 words. That's tiny, but it is a change.
When someone rolls 1,1,1,1,1 and rejects that word they've cut down the word list to 7775 words. That's tiny, but it is a change.
An attacker can under certain circumstances exploit this behavior. For example, if they notice you are picking shorter words or words that consist of certain characters, they will have an easier time to crack the password. Whole categories or patterns can be rejected because of that, reducing the search space.
An extreme version of your method, is picking only the patterns that relate exclusively to you, like picking your favorite music or hobbies. Then the whole scheme becomes useless.