I'm pretty suspicious of that survey (the one that always gets cited as proof that Copilot makes developers less productive, which then inevitably gets used to argue that all generative AI makes developers left productive): https://resources.uplevelteam.com/gen-ai-for-coding
If I was running a company like https://uplevelteam.com/ that sells developer productivity metrics software, one of the smartest marketing moves I could make would be to put out a report that makes a bold, contrarian claim about a hot topic like AI coding productivity. Guaranteed to get a ton of press coverage from that.
Is the survey itself any good? I filled in the "request a copy" form and it's a two page infographic! Precious few confirmable details on how they actually ran it: https://static.simonwillison.net/static/2025/uplevel-genai-p...
Here's what they say about their methodology:
> Metrics were evaluated prior to implementation of Copilot from January 9 through April 9, 2023 versus after implementation from January 8 through April 7, 2024. This time period was selected to remove the effects of seasonality.
> Data on Copilot access was provided to Uplevel Data Labs across several enterprise engineering customers for a total of 351 developers in the TEST group (with Copilot access) and 434 in the CONTROL group (without Copilot access). The developers in the CONTROL group were similar to those in the TEST group in terms of role, working days, and PR volume in each period.