By the distinction that Prof Devlin tries to make, primary school subtraction is typically taught/learned in an 'algebraic' way (logical reasoning to invert addition). This makes it difficult to understand what he is trying to say.
I most strongly doubt the claim that students who are strong in arithmetic find it harder to learn algebra. It's obvious that students with good arithmetic skills are more quickly able to find value and purpose in algebra.