This argument can be made about every technological advancement ever made. "Don't fix it with technology, just throw more people at it". That isn't how you make progress and it doesn't benefit everybody it just encourages stagnation.
The more we have immigrants stocking supermarket shelves and constructing infrastructure the more we can focus our high end R&D on things like biotech, medicine, space, and the like. That’s economics; comparative advantage is real.
If Japan can automate many of these things they will end up being leagues ahead of those who solved it with immigration and will end up selling the solutions to the rest of the world.