Set basically has an Ord constraint on the type of things you have a Set of since most of the functions on it have an Ord a constraint, and Monads can't be constrained in what types they wrap. This is also why it's not a Functor; you can define

  setMap :: (Ord a, Ord b) => (a -> b) -> Set a -> Set b

but that's still more restricted than fmap.