Mass surveillance is completely legal. It's just stupid to say its not.
At the same time, it is expressly illegal in some circumstances; that was the whole core of the Snowden revelations. The NSA and CIA are expressly curtailed from doing that by law — there are cases where they may surveil citizens with a court order, but not "mass" surveillance. There are some restrictions on the military along those same lines.
Keywords: Executive Order 12333, FISA, National Security Act, Posse Comitatus Act