Matrices for rotation have the problem that you are using a 9-dimensional representation for a 3-dimensional quantity, so it is easy to make matrices that are not rotations (indeed the vast majority of the possible space of matrices consists of transformations that are not anywhere close to a pure rotation). Composing matrices leads to rounding errors. It is more complicated to invert matrices. As you say the matrix logarithm is more complicated. Etc.

If you use a scalar + bivector “rotor” representation, that is only a 4-dimensional representation, which is easy to normalize to unit magnitude.

It's beautiful to see that the logarithm of a rotation matrix is a skew-symmetric matrix. The skew-symmetric matrices correspond to angular velocity, which behave like vectors. You can add and interpolate as you expect. Rotation matrices multiply. Skew-symmetric matrices add. The logarithmic and exponential map moves between the two spaces.

Just letting you know that I think it's valuable to bring up matrices into this discussion, even if they do have problems in practice with rounding errors and efficiency.