Don't take legal advice from anyone but a lawyer that you are paying.
Seriously, this depends on a variety of factors and is especially dependent on your local laws, the specifics of the contract and most importantly, precedent.
Find a lawyer, give them your contract, detail your side project and get an opinion.