It means you are more likely to answer other questions in a certain way. Other studies might even show that you might be likely to behave in a certain way.
> Does it have a consistent effect?
Yes, but like thermometers only work on groups of molecules, the questionnaires are consistent on groups of humans.
> Can I measure the effect? With what? Another questionnaire? So it's questionnaires all the way down?
No, you could easily do a follow up study by finding groups of people that answered the questionnaires in a certain way, and then have them participate in behavioral experiments.
> Well, I know for sure that whatever thermometers are measuring- it's not thermometers all the way down.
The manner in which molecules bump into eachother randomly. The harder they do it the more space they take up. The more people in your group with an agreability score of 1, the more space they might take up ;)
Opinions are not science, maths is. We don't improve the social sciences with more vague intuitions. If you want to read intuitions then maybe read a glossy instead. Science is about making quantifiable statements, and maths is the way you turn samples into those. We certainly don't need any more room for so-called experts to tell us how we should behave in their papers. We tried that, and it was awful.