Well, quaternions form a vector space over quaternion addition. This part is not very interesting. Vector spaces do not describe multiplication of vectors by each other. So, quaternions are not (only) vectors "in the mathematical sense" when it comes to their more interesting properties.
What do you mean? They also form an algebra (a vector space where a "multiplication" is defined).