The key point is the trial. You can't hold a trial in a hostage situation. They could have held a trial for Anwar al-awlaki. Once decided by trial that he committed treason, military action could be justified.
The problem is how that trial would be conducted in absentia would have been a farce. The government would also be concerned about its classified evidence. Worse, if the trial led to acquittal, it would have legitimized his cause.
An acquittal is a scary prospect of bringing out other copycats. But more scary is executing someone without trial that would have been acquitted. But we'll never know. They should have held a trial.