Because they are not silent. Or sometimes are not. Inverters do have quite large fans.
Farm-scale irrigation is not silent.
Crop Dusters are not silent.
Combines and other tractors are not silent.
Burning fields are both not silent and release a tremendous amount of sooty smoke that spreads far beyond the boundaries of a farm.
Farms make a lot of noise.
Depending upon their other priorities, they may be upset about the loss of hunting access as well. Understandably, people putting up solar arrays don't want people firing guns in the middle of their arrays.
If I were to hazard a guess every person complaining would happily suffer the 'consequences' of a solar farm not being near their neighborhood.
It really should be a no brainer compromise to zone solar as industrial so they're not near where people live. There's in practice infinite amounts of land you can get zoned like this. Living to electrical noise sucks in a way living need next to a wind farm doesn't.
You won't have to hear it, you won't have to look at it except as way off in the distance, you won't have to worry about whether or not your buddy's farm is gonna get taken over by one when they run into financial troubles. Out your backyard you get to look at mostly pristine farmland and wilderness. During this time where there's political will and capital to just ban them outright I think this relatively small concession will make folks not put up too much in a fight as long as it's kept out of sight out of mind.