I think most people would upvote a reference to a xkcd based on how relevant they think it is to the post, not how much they love it.