It matters quite a bit I think, because otherwise it's an impediment to communication. Just look around at all the pushback from unnecessary changes to language due to so-called "wokeness". If a term has a coherent conception and is mostly well understood and commonly utilized by people, I don't see the value in trying to change that language.
In this context, Harris is just one in a long line of people leveling this accusation that Compatibilism is changing the definition of free will, where it seems instead that it empirically matches how people actually use this term.
This is not to say that you specifically were making this argument, but it's been implicit (and sometimes explicit) undercurrent in this whole thread.