Sounds like the UK has effectively given away the atoll to the US. The US is not going to pack up their very important torture base just because the UK asks them to. Denying the islanders permission to return is just a face-saving move to avoid admitting that the UK no longer controls the territory.