I really don't see how this would have a major negative impact on speed. If anything, it will allow CSS files to be smaller reducing the amount of data being downloaded to the client and may actually improve performance.