It's more that most humans are predictable most of the time. There's no reason a password cracking dictionary can't contain phrases as well, and most of the sorts of rules based permutations used against normal passwords can be extended to passphrases. Attacking a specific individual also tends to be easier - you can profile them and build targeted word and phrase lists.
XKCD is bad specifically because it is specifically talking about attacking a password via an internet based oracle rather than attacking a hash, but people generalized it. A search space of ten or twenty trillion is laughable when a few GPUs can make a billions of guesses per second against a vanilla hash algorithm.
Ultimately, if you want a password or passphrase that a computer can't guess, you should let a computer pick it for you, or if you're really paranoid, use diceware.