"It's also very not true that farmers are poor in the US (median household income about 30% greater than general population)."

Most farmers work a "real" job in addition to the farm work.

Then you also have farm workers, who do not own the farm, and they tend to be poor.