I think your statement "We call people who do the farming 'farmers'" is accurate to my usage as well, perhaps with then some nuance to what work is "farming".

I didn't mean to imply that land ownership was central. But maybe "operating the business of a farm" is farming/being a farmer, whereas "doing labor on a farm" is not.

For instance, if person A owns 100 acres of farm land, person B rents that land and makes the decisions about what to plant, when to plant, when to harvest, how to manage the transportation and sale of the product, etc and person C is hired by B to do some work on that farm, I'd probably only call person B a farmer.