We know SSRI's really do cause permanent sexual dysfunctional in a small minority of people, small enough that this side effect doesn't come up in traditional FDA tests.
If a side effect is extremely rare it would be impossible or at least impractical to prove in a population.
Grandparent could be right or wrong about how the drug affected them, maybe their brain suffered from other issues and the timing of the medications was purelycoincidental, but if they are correct, your dismissive response is exactly what we'd expect given when they are saying sounds unusual/ improbable.