Note that you can substitute all sorts of adjectives for "dumb".
E.g., "those nerds are so unpopular they deserve to get beaten up". Or, "those people are poor enough that they deserve to live next to a toxic waste dump".
Basically, it's a way of taking whatever characteristic you pride yourself on and then making yourself feel even better about it by enjoying when shitty things happen to other people.