Also when it comes to self-defense laws, most states require the shooter to genuinely believe he/she is in mortal danger before shooting. Sure some states have stand your ground laws that allow more, but I would not say it's so clear-cut that people can just start shooting other people or property when they believe their rights are being violated.
It's not the people's decision as to whether a drone is violating the law. If you were arguing having the government have that right or responsibility that's a lot different. But even so shooting down aircraft is not something that should be done likely. There's a reason the FAA makes it completely illegal. Shooting a drone for flying over your property could make it go out of control and injure others.