You can jam signals they rely on, like GPS and cellular, but there are already drones that are capable of flying on vision alone, so no.
A small-ish plane frame could easily deliver a grenade from sleepy suburbs 10km straight to the Oval Office with only few minutes of warning (assuming somebody could actually detect it in time at all). We're just lucky nobody seems to be bothered to do that.