The question is, are you ok with the encryption being breakable if it makes your watch faster?
Personally, if I'm ok with the encryption being breakable, I'm also ok with not using any encryption at all. I can't imagine a scenario where I would want weak encryption.
Anyway, the way to solve that problem is to get a CPU with AES instructions.