> But it would be reasonable of them to expect you to give up something in exchange (e.g. smaller salary).

Except when the side project is useful for your employer and you've effectively contributed free, unpaid labor to developing it.