You cannot conflate dataset engineering, machine learning, and researchers as if they're a single discipline and people. This is precisely where the disagreement stems from.

Dataset engineering is an important field of research. It is also an important part of the modelling process. Neither of these things are included in model research. Researchers who design computer vision models and other novel mathematical models can be held ethically responsible for the societal results of the novel part of their research but if we agree that their field of research is valuable for society then they cannot be held responsible for the misuse of these models by people who skip or fail integral parts of the modelling process.

You cannot ask a researcher to invent a new more performant computer-vision model, invent a way to correct for bias, and design a fair dataset all in one. That's three entirely different careers.