>could there be a case where the parties in a conversation are legitimate suspects?

It doesn't matter if they are. No one expects coffee shops to put microphones at every table on the off chance that a terrorist plot is planned there (how many mob hits could have been prevented back in the day).

We shouldn't accept the government listening in on digital communication just because it's easy.