It's a lot safer if they are flown by AI. Possibly only reverting to manual control in an emergency and then only for a controlled AI assisted landing. For example.
That could prevent the scenario where terrorist with TNT strapped to his belt flies into a buildng.
On the other hand: People could still muck with the hardware and the software. Just making it more difficult doesn't rule it out.