That doesn’t work as well for stuff that you wish other people would find noxious but they don’t: neo-Nazis probably would rather see more neo-Nazi things rather than fewer, even if the broader social consensus is (at least on the pages of the New York Times) that those things fall under the uselessly vague header of “toxic.”
Even if the recommenders only filter out some of what you want less of, ditching them entirely means you’ll see more of the portion of the slop that they’re deprioritizing the way you want them to.