Can you say for certain that this is not directly caused by a policy of having a contributor license agreement? Have you experienced this on other projects that don't have one? It seems by having a CLA you're limiting your contributors to a subset of developers who are willing to sign one.
(Disclaimer: I personally would discourage developers from signing a CLA unless upstream is paying them to do so, since upstream is going to directly derive profit from it)