However, as far as I know you're right. If there's no clear authority as to who retains ownership and licensing rights to the code, and the contributions made to it, it's going to get messy.
A lot of projects, notably including Linux, intentionally avoid copyright assignment to make it impossible for anyone to relicense the codebase. Making sure that there are thousands of copyright holders from hundreds of jurisdictions, many of who are not easily reachable or even knowable, all bound by common license terms protects the project from situations where some project participants would do something not agreed by the rest, either willingly or because they were forced to (eg. through bankruptcy).
LWN.net has covered both sides of this subject quite well, in recent months, with the tedious process of relicensing VLC [1] and the GnuTLS copyright assignment controversy [2].
The politics of OSS make for some excellent reading.
That only works in the US, there are various jurisdictions where it is impossible to allow third parties to relicense one’s IP in any way they want.
I admit that I'm not familiar with international copyright law. I do know that several GNU projects, and other high profile open source projects, have a policy in place wherein contributors have to agree to a copyright assignment agreement before they can contribute their modifications.
If that's not possible in some regions, they clearly still have some means of continuing to uphold their own IP. I suspect it's not as harsh as completely avoiding contributions from some regions of the world. Unless I'm mistaken.
However, similar agreements (i.e. an artist allowing a record company to use his song however they wanted) signed in the 70s have been found to not extend to ‘digital’ uses, of e.g. music, by German courts as this use case did not exist yet at the time the license was granted.
I’m really saying that this is an annoyingly complicated matter and best avoided by not requiring relicensing – and I’m not a lawyer, of course (and also too lazy to find references now).
And, naturally, there is the issue whether you trust the FSF enough to do the right thing.