* Doing the work in a completely different way would eliminate the need for more people doing the labor. In the case of Japan, there is a lot of small farmland. In the case of the US, farmland tends to be huge. I guess smaller farmland is more labor-intensive. Consolidating smaller strips of farmland into a larger piece of farmland may improve labor intensity. But that means that one person gets to do the farming for a higher margin and everybody else loses their profession.
* Lots of farmland is being worked by elderly people. At some point you can't do it anymore. Somebody not working in agriculture would have to give up their current job and go into agriculture. It's difficult to predict whether that will happen.
* Labor shortage often means "we can't find anybody who is willing to do it for 1000 yen per hour so there must be a labor shortage".
BTW, there are a lot of abandoned houses in Japan; many of them will come with some amount of farmland that could be used, but isn't used.