I don't really understand your point... that the problem is non-local people making money on housing?

Even if it was only local people making money on housing this would be a net negative to local residents. Housing that exists is placed on the "global" market rather than the local one. Unavailable to anyone but tourists.