People who just inherit a farm and get other people to run it aren't farmers.
The word people use for what you describe is farmhand. It is the farmhand who works on a farm. The farmhand is the agricultural equivalent of a factory worker. Indeed, someone who owns a farm, but does not work on it, would not be considered a farmhand, but they most definitely would be a farmer.
It is technically possible for one to be both a farmer and a farmhand, but being a farmer does not imply that one is also a farmhand. They are distinct roles.