You'd end up with the official calendar day either flipping in the middle of the day, or at a different time in every locale.
And then let "Sunday", "Monday" etc become simply approximate terms. And I never have any idea what day of the month it is so I don't care. (Birthdays and public holidays are set based some recurring feature of the count.)
A refinement of this uses a time period that is approximately 100,000th of a day so then people can easily notice the day. In practice we might use hectokiloperiods instead of periods. Someone will invite a less woeful name.
Nothing about telling time will be particular familiar. Doesn't mean it won't work. But I suspect it'll be awful. I'm fine with daylight savings and all the rest of it. Except days of months. Can't we just say "Monday, third week of June"? or "Wednesday, 39th week"?