A related idea are the concept of second class references, as exist in Hylo. There the "ref" is not part of the type, but the way they work is very similar.
Lifetimes give you a lot of power but, IMO, I think languages that do this should choose between either being fully explicit about them, or going "second class" like C# and Hylo and avoiding lifetime annotations entirely.
Eliding them like Rust does can be convenient for experts but is actually a nightmare for newbies. For an example of a language that does explicit lifetimes without becoming unbearable, check out Austral.
Instead of C#'s scope ref solution to having a function accept and return multiple references, another option (in an imaginary language) would be to explicitly refer to the relevant parameters:
ref(b) double whatever(ref Point a, ref Point b) { return b.x; }