There's nothing silly about it. It's just a metaphor. It's more incredulous that people have a problem with figurative speech like this than the comparison itself.