So if we conquer a place and change the people there, it's the new people who decides? or how we proceed?
After 500 years? Yes
Somewhere between is reasonable to debate, but many people won't even agree on the principal of self determination, at least not consistently. Personally I wouldn't put the threshold longer than a human lifetime