you seem to have strong sense of ownership on your land, maybe your ancestors owned that land for very long time, but keep in mind that your ancestors came from somewhere and nobody denied them the right to stay and live
The concept of visas may not have existed at that point in time, but you were at the mercy of those in power. If they didn't like you, they'd banish you from their lands or worse.
The modern rules are entirely based on statistics. How many overstay their visa. How many get denied a visa. Are compromised passports swiftly dealt with. India doesn't live up to the rules for easy visas.