Also, the US military has been stockpiling kinetic drone countermeasures for about four years now. The idea is you get a hardened, ~11 pound autonomous drone that slams into the target at roughly 90 mph and physically destroys it before returning home. Add on 1-2 year-old US EW technology that now disables autonomous drones (yep, even autonomous drones), and you can establish a very comprehensive defense. The point I'm making here is that the tech is not only possible, but it exists.
Is it perfect? No. Though defense against firearms and explosives today isn't perfect either. Namely because of response times of the countermeasure. So in that sense, we aren't entering a uniquely dangerous situation.
Edit:
I think what will happen is that the first time a UAS is used on civilians, flying drones around population centers will be banned without permit. That way, if a drone is seen flying without a permit, it gets taken down on sight.