I think your agreement should say that they have access to the source code if they want it. However, they must not resell any modifications and may submit their modifications to you for a discount.