I'm not the OP, but I mostly support the idea of open borders and I'm from Canada. Canada has many of the same issues that New Zealand has. We are the second largest country (by landmass) in the world, yet our population is slightly smaller than California. Much of our land is quite rugged, or very cold but our country is still capable of supporting more than the 35 million we currently have.
The best argument that I have ever heard against opening up our border is that we are big, but size does not equal the ability to support a decent quality of life. Some argue that if we open up our borders, dreams of higher incomes will compel lesser skilled people to move to Canada. These people often will not understand basic labour laws like minimum wage, or the need for benefits and so they will drive down the cost of labour. Others argue that adding even five million people sounds like a good idea, but that it would require massive investments in health care, housing, and basic infrastructure.
Like I said, I mostly support open borders but have never been too impressed by the moral arguments. Canada is a great country and I love it dearly, but I have spent a little too much in some bad neighbourhoods and become friends with too many unhappy immigrants. Rather, I would argue that we are an amazing country, but that many immigrants who bought into the hype, came here and live in poverty far worse than that which they escaped. I say 'far worse' because, while our social safety net is fairly good, it involves navigating an intense bureaucracy that is difficult for native English speakers and near impossible for immigrants, especially those with more limited English language skills. And, unfortunately, parts of our population are very anti-immigration and argue that they should 'stay home' instead of clogging up our social services.
Morally, I think that I could just as easily argue that we should stop all immigration entirely until we can do a better job of caring for the immigrants that we already have.
My all-time favourite argument in favour of opening up our immigration system is that it is one of the best chances that we have of saving our manufacturing industry. Since the 2008 financial crisis, much of the manufacturing industry has been in a horrible shambles and years later, factory jobs simply have not returned to eastern Canada. Factory jobs have mostly been a victim of cost and so the theory goes that opening up borders to lesser skilled people will drive down the cost of lower skilled labour. If you drive the cost of lower skilled labour down closer to minimum wage and get rid of the often union negotiated benefits, suddenly the cost disparity between producing in Canada versus China or Mexico looks a little better.
Another argument I love is that the last time our borders were mostly open, we allowed an influx of mostly eastern European people to come into our country. (My last name is Hluska and my great grandfather was one of them). There were some growing pains and for many people (my great grandfather included), the move to Canada was really difficult. However, after a few generations, these immigrants' children and grandchildren have become important entrepreneurs, business and political leaders.
So, the argument goes that in our history, mostly open borders lead to a difficult decade or two, but that the kind of person who will immigrate tends to be resourceful enough to bring enhanced opportunity for many generations after.