No the biggest problem is men actively harrasing women because they can easily get away with it. Ask any woman you know about their experiences some time.
All the big metoo cases, for example, are not people who didn't know. They are people who were activly predatory over many years. The accidents aren't what are being targeted here. It's mostly men activley and knowingly abusing the fact that they can get away with it.