LGPL does not bind the author(s) of the software in this way. And since there was a CLA the other authors authorized the main dev to change the license.
If you accepted PRs without a contributor agreement transferring the ownership, you might be infringing on their IP (licensed to you and others).