So instead of the time there you’d look up the sun position. Not worse just different.
It changes the meaning of time that we all use to communicate.
Edit: originally said they need to be related which is opposite of my intent
We put up with crazy things in an attempt to match solar position. The meaning of time we all use is flawed when I can hop on a plane and go back to yesterday. That's quite inconsistent with the accepted definition of time as unidirectional.