You can try getting a tighter fit and see if that helps.
I've noticed with any consumer heart rate monitor, if the fit is not tight and the monitor moves even a little bit, the heart rate the monitor reports will be off. I'm assuming the same will apply to the Apple watch.