you are right but the aim is to find who the writers consider the "main" character and not who had the most impact on the GoT universe. If we take it in that sense the white walkers are probably the most significant.
Some of the characters who have had the most effect on the events as they are taking place in the books are dead before they start and get very little page time.
This is an analysis of the books, who have just one author.
I would say there is no single main character to the story.
that's also the most likely reason the story (book or TV) is so popular. the fact that there is no distinct good or evil, no main character, no main villain etc.
everyone is grey, everyone can die etc
There are obviously evil characters, obviously good characters, and ambiguous characters. Mostly ambiguous.