This is not even close to the only problem. Our housing stock has not kept up with population growth, which is a big reason why housing prices have been so out of control here. It's not due to a lack of competition so much as regulation and population growth that has outpaced building capacity.
So I'm not sure I can agree that that graph looks fine. It's true that we need immigration to balance falling fertility rates, but this needs to be balanced by the available resources so the cost of living doesn't spiral out of control. Labour shortages can often be addressed in others ways, like automation.