You should review your contract carefully. Typically the IP is things created "in the context of your work" and that are related to the company's business or with knowledge gained through your work for the company, using company equipment, time etc. So first, you have to violate the specific terms of the contact, and second they need to find out about it and care enough to sue you. People start side projects at work all the time.