The Into<SpanArithmetic> argument feels a little too magical.
Since you are not constrained by a trait, I would prefer something like
Span::add_with_reference(&self, other: &Span, reference: impl Into<SpanRelativeTo>)