The person you're replying is arguing that in practice it isn't "OK", because if "OK" means that you can peacefully live your life as you please, but others are actively engaged in preventing you from doing so, you're forced to care about politics, because others make it not OK for you. Thus only if you are "privileged", meaning that people are letting you be free and giving you your fair share of the resources, can you in practice choose to "not care", since the politics don't affect you negatively.
Also note that politics means: "the set of activities that are associated with making decisions in groups, or other forms of power relations between individuals, such as the distribution of resources or status" and implies nothing about having or not a "Government". That depends very much on the political system itself.