While you may believe that current surveillance has gone too far, advocating for no security is much too far the other direction. As usual, the answer is somewhere in the middle.
Without any visas or border control, you have no way to prevent ingress and egress of known criminals, people carrying infectious diseases, or any of myriad other actually real threats. Just because people exaggerate threats like terrorism does not mean there is nothing to defend against.
It is also important to prevent people who have no means of sustenance from moving into your country and becoming a burden. In no way am I implying that exceptions should not be granted for asylum or other exceptional circumstances. However, being in a country is a bit like being in a club -- everyone pays dues (taxes) and there are perks too. Why should club members want to give everyone the perks while they have all the responsibilities?
Your thought that people should try to be closer to one another is admirable, but I think if you reflect on it further, you'll agree that some forms of restriction are necessary.