Huh, this statement is so weird to me that it was a "lightbulb moment" about what words mean to different people online. I wonder how much misunderstanding this causes. I'm guessing your background is urban and coastal? I'd love it if you'd share, I'm genuinely curious.
I would never use the word "farmer" if I was talking about an immigrant laborer picking fruit or whatever. I'd call them migrant laborers or pickers or farm workers.
In the standard dialect of people in the 90% of America designated "flyover country", they'd would use the word "farmer" to mean the people who own and operate farms. Generally a middle-class, small-business-owner type person. Likely educated in some mixture of agriculture, business, finance, real estate. They're highly financially savvy, as they have to manage selling futures vs taking the spot price for their crops balanced against storage costs, land rental agreements, machinery leases or repairs, tax and crop insurance complexities, not to mention the science of how to optimize yield on their specific land.
Calling the laborers "farmers" is like calling an uber driver a tech worker, at least in the dialect of English I speak.