It's true that now those kinds of people are often directed towards politics and become the elected officials which you assert justify the borders, but in fact most of the borders we live with were never drawn with any consent of the people living there. The vast majority of borders are pure objects of fiat power (especially the border over which the immigration you have a chip on your shoulder about, which was quite literally determined via war).
Herein I'm not advocating charity, which is, nevertheless, a fundamental human virtue. I am advocating that human beings, if they have any rights or dignity at all, have those rights and dignities by virtue of their humanity and not by virtue of where their parents happened to be when they were born. Communities of humans shouldn't tolerate freeloaders, but the people coming across the border are by and large the exact opposite: people willing to work harder for less than many people living here. If you are really so concerned with social parasites, let me suggest you look to institutions like health insurance companies, landlords, etc.