Maybe, instead of either just declaring what is the "right" way or observing users like some kind of behavioural scientist, we could just go back to, I don't know, asking them.

Innovation comes from experimentation. It's useful to have hypotheses and quantitative/qualitative data to back the assumptions up or disprove them.