Can you cite a significant environmental group that completely opposes all large desert solar projects for this reason? Otherwise this reads like a rather unfair characterization.
In a recent case [1], the Sierra Club successfully argued that a planned solar array outside Hollister should be made smaller (by about either one half or two thirds, it's not clear why each successive cut was made), due to the potential threat to a handful of endangered species in the area. They also indicated unequivocal support for a similarly sized solar array in Imperial County.
Given how much undeveloped desert land exists in the US Southwest, it seems reasonable to build solar arrays first in areas that won't threaten the habitat of endangered species.
[1]: http://www.mercurynews.com/2017/07/21/giant-solar-project-re...