When you live and work in (let's say) America, you are paying taxes and following the law. If suddenly, anyone could come to America, America would probably experience a huge population increase from Africa and Asia. While some of them may be productive people, there will definitely be a lot of them who will not necessarily follow the law or find useful jobs. In addition, if it was so easy to enter America, what's stopping them from entering, committing a huge crime, and then leaving?
Another reason to be against what you are proposing is because it is in a nation's interest to make sure that the people living there aren't poor or starving. If an American doesn't have that many skills and can only work in a specific field, by allowing anyone in the world to compete with them increases the likelihood that they will be out of a job.