Except almost worse. You can choose to not use a piece of GPL software. You might very well accidentally invent something simple that turns out to be patented.
Independently invent something. Turns out it's already patented, within this good faith consortium. Now the only way to use it is by participating. Normally, this kind of thing results in getting sued for royalties.
You have answered it. Free patent, GPL or not, you'd have this problem. And it's even worse on "normal" cases.
I've contributed quite a few patches to open source projects, and sometimes if I'll review other patches I cannot tell the difference between code I've written or that others had written.