I usually rephrase it as “getting run over by the lottery” or “winning a bus”. Just to be different/humorous/less morbid.
(I do also like the lottery version and use them basically interchangeably, though the shorthand is always "bus factor" for me)
For them, hearing "bus factor" may make it through the barrier and result in an involuntary creation of imagery in their head of someone getting gruesomely hit by a bus, and then the corresponding emotions they would feel (or simply just the emotions, without the imagery).