The problem with dark matter is that there's no (working) theory on how the dark matter is distributed. It's really easy to "explain" gravitational effects if you can postulate extra mass ad-hoc to fit the observations.
Not even anything that extreme. What's ruled out is interaction via electromagnetism (or if you want to get really nit-picky, electromagnetic interaction with a strength above some extremely low threshold).
Besides the idea 'not all mass can be seen optically' is not that surprising. The many theories on what that mass might be are all speculation, but they are treated as such.
Both of these are pretty much ruled out though: you can't plausibly add enough brown dwarfs, and if it's black holes then you should see more lensing events towards nearby stars given how many you'd need.
But they're both concrete predictions which are falsifiable (or boundable such that they can't be the dominant contributors).