The constitution makes no reference to citizens with regards to what congress can and cannot do. Nearly all these things are couched with "congress shall pass no law" -- the focus is restricting the behavior of congress rather than the citizenry.

This is a totally bunk argument. If you want an argument that does work against the constitution, SCOTUS' "empty shell" or Lysander Spooner's "Constitution of no authority" are of far more weight, but have more disturbing implications (e.g. You can disregard it completely; our system is precisely the "odious arbitrariness" the founders denounced).