You only have to worry about deaf birds. Drones are pretty loud.
I guess all that noise might affect the life of birds in some way or another.
In my personal experience any radio controlled aircraft coming near a bird's nest will be challenged in the air. It certainly stresses them out in the mortal danger kind of way. It made me change where I do my “Park Flying” before I injured a bird.
I've flown a lot since the late 90's. A bunch of different areas. My dad was very into it. I've never had a bird attack an airplane or helicopter. I've never flown quadcopters/drones much as they require no skill and not fun. The closest I ever saw was a small foamie where a bird swooped towards it. Never came that close and moved on.