I think people often confuse the representation for the object itself, and thus create a limited mental model that must be undone later. Analogous to a world in which all cookie shaped objects are edible and delicious. Sure it would be nice to live in that world but it's not reality and a more rich (but less pleasant) representation makes reality more accessible.

I'd say the same thing about any math that talks about "numbers" without defining which number system very explicitly, even at the Kindergarten level. The slop in these early abstractions is somewhat convenient but it erects serious mental barriers against more accurate abstractions.

Look at the sloppy way that many programming languages handle lossy casts for an example of the pernicious nature of the idea that "number" should have a highly intuitive meaning.

On the other hand, imagine all the nice notations/sugars we might invent that are highly intuitive and capture mathematical objects more elegantly than what we are using today. Representations are the interface between abstract concepts and brains evolved for eating, sex and lying.