It has taken me a long time and a lot of work to understand it to the level I have, and I still have a lot more to learn. But it is a fascinating subject and a very elegant way to represent some very complex optimization problems.
There are other cool things like representing a probability distribution over a 3D rotation with an multivariate gaussian vector interpreted as a so3 or se3 vector. Using this you can for implement a very elegant extended kalman filter estimating a 3D rotation. Which usually can be a tricky thing to do.
In robotics we pretty much only use the SO3, SE3 and SIM3 groups so my knowledge is limited to those groups.
A 3 valued so3 lie algebra vector is however always a valid 3D rotation, no matter what the values are. That makes it an ideal parameterization of the optimization problem.
The mapping from so3 lie algebra to SO3 lie group (which could be represented as a rotation matrix or quaternion) is a smooth differentiable function, which allows us to compute the jacobian we need for the gradient in the optimization.