There are certain games that work better with controllers—the best example I can think of is Mirror's Edge. I would be upset if I couldn't use a controller, because sometimes you need joysticks.
If you haven't tried before, I highly recommend doing time trials with KB+M. It's a completely different game.