You do not necessarily have 4 grandparents. Try to think of ways in which you can have fewer than 4 (hint: it's a little bit impolite to talk about). Neither do you necessarily have 8 great-grandparents, for the same reason.
If your nearest common ancestor is a great-great-great-great-grandparent, that's not incest (it would most likely not even be known at the time of marriage), but it still leads to large amounts of duplication in your family tree. Every family tree will have many instances of nth-cousin marriages simply because people tend to marry people who live close to them.
It's very hard to calculate what the actual number would be because it depends a lot on where your ancestors lived and how much they migrated over time.
That really depends what you're imagining when you say "incest". In literal terms, yes, we're talking about mating between two people who share common ancestors. Those do not need to be particularly recent.
In genetics there is the concept of the "coefficient of inbreeding" which tracks how much of an individual's genome is identical-by-descent across both chromosomes. (That is, if you inherit a particular stretch of DNA from your mother, and you inherit the same stretch of DNA from your father, and the reason they are the same is that your father and your mother both inherited that DNA from the same person, then that stretch of DNA contributes to your coefficient of inbreeding.)
If you're imagining brother-sister marriages, that is a much higher level of inbreeding than would be common in human history. (Expected coefficient of inbreeding in the offspring: 25% assuming zero inbreeding in the parents, perhaps because they are themselves the offspring of a cross between an African Pygmy and an Australian Aborigine.)
But it is true that different populations have more or less inbreeding than others; we can say that Arabs tend to be more inbred than the French. This is not necessarily because they tolerate incest at closer degrees of kinship than the French. It is mostly because they have a greater quantity of still-relatively-remote incest. (Not the greatest example; Western Europe hugely stigmatized cousin marriage under the influence of the Catholic Church, so in reality Arabs do tolerate closer degrees of kinship in a marriage than the French do. But the concept is sound - inbreeding will "build up" in a population over time.)