Similarly, when someone is reporting something we know to be impossible or extremely improbable from other considerations, we are not accusing them of something when we say that their experience or memory of that experience was delusional (or mistaken, depending on the details). Again, I was insulting when I called such people "delusional", as it implies they often have such hallucinations, which of course I can't know and don't believe - and, again, for this I apologize.
Now, the major difference to accusations of sexual assault is the plausibility of the claim. I of course do not personally know if Harvey Weinstein assaulted any of the women that accused him. However, I know that such accusations are painful and risky for the person making them; and I know that sexual assault is something that can absolutely happen; so, the witness testimony carries a lot of weight.
If on the other hand the exact same women accused Harvey Weinstein of stealing their souls through satanic rituals, I would not think much of these claims, and I would believe, and feel justified in believing, that the women are either lying or have had some hallucinations that have convinced them of this (or are having false memories).
Of course, if you tend to believe that aliens (or angels, curses etc) are plausible, you may lend more credence to these testimonies, even without scientific style evidence for what may have happened.
I still believe that comparing the certainty we can have that sexual assault is a real thing that real men and women may experience to the certainty that aliens (or demons and ghosts) are real is deeply insulting to victims of sexual assault.
I also don't believe there are millions of people claiming to have experienced alien abductions. Looking around a bit, I assume this claim is coming from a Roper Poll that found 119 out of some 6000 respondents had experiences which were considered typical of alien abduction, which would be extrapolated to 3.7 million out of the 185 million people for which the poll was representative. Crucially, the respondents were not claiming that they had had an experience of being abducted by aliens, they were claiming that they had had some experiences like "waking up paralyzed and feeling a presence in the room", "finding puzzling scars on your body", "seeing unexplained lights in a room" - all of which require a significant jump to conclude "ALIENS!". In the best case, they could be used to claim unknown phenomena are real, but to pick any specific posited phenomenon would be deeply wrong: these are as likely to be signs of aliens as they are of being fairies or ancestor spirits or mind/body dualism or anything else; including altered mental states (especially as the poll didn't even ask about the respondent's belief that the experience was real - for all we know, some of those 119 people could have sought psychiatric help themselves after these experiences).