The distinction is important because thinking about the way you have presented leads to confusion about what tensors are...
In context we often refer to scalars, vectors,matrices as order/rank 0,1,2 tensors (higher order tensors don’t have the same sort of common shorthand). This works fine when you have the context of the underlying vector space, and nderstand the “rules”. Physicists do this a lot, and they often love shortcuts :)
However, there is a growing use/abuse of the terminology (see machine learning) to just mean n-dimensional arrays. The analogy is drawn that a matrix is a 2d version but you can have 3D, 4d, etc. While it’s true that a NxN matrix can represent a tensor (given the context as previously) that misses most of the structure... as such, it’s an unfortunate use of the name.