It is the most important factor, but I disagree that it's that easy to tell (if that's part of what you're implying).
Some people can actually do very well in what appear to be hostile labs. Some needs lots of space, while others do best with close monitoring. It ultimately just comes down to whether the advising relationship works or not, and it's nearly as complicated as romantic relationships.
One thing that I like to stress is that you don't have to like your advisor. You should probably not dislike them, but what matters is how you play off each other and work together.
An example: some advisors will put the people in their lab in somewhat adversarial positions. There will be two or more people working on similar projects, and the more successful of them will start to get more and more of it. For certain types, this situation is actually very motivating and can produce two differentiated projects that support each other. For others, it will undermine their confidence and wellbeing to a terrible degree. There can be a lot of resentment among both parties.
While I would generally consider this a warning sign, you must recognize that this can be a good environment for the right person. The difficulty is that it's hard for you to know what works for you until you do it. There's no best formula, you just have to choose as best you can.