The server's timestamp isn't going to be representative of the user's actual input when you're using UDP. Especially over consumer wifi which is hardly the most consistent network, but is still used by a majority.
As I mentioned in my last comment, someone would need to design a system to better deal with this type of stuff. The way things are stem from decades of decisions made, starting with more design constraints than we have today.
I'd recommend checking out Leslie Lamport's work, you can probably derive some ideas. I'm not be prescriptive here because this isn't a space I plan to work in to solve problems since it's just games. Entertainment isn't really my thing, I tend to lean heavily into systems and security knowledge which has served me well broadly.