A lot of EA messaging tries to work around this by trying to avoid guilt trips or demanding that you do everything you possibly can, since asking too much is a good way to end up with nothing. See Giving What We Can and similar.
But a lot of the justifications for this kind of approach can very easily imply a much higher bar than anyone can meet. Taking global health as an example- millions of preventable deaths a year, and if you work yourself to the bone for years, you can only hope to partially mitigate it.
Even if the messaging says, "hey, just do what you can sustainably and don't worry about being perfect, because that's way better than nothing," the logic behind it is based on nothing more than observations of actions and their consequences. There's not a specific threshold where your job is actually done, and 'failure' to meet the impossible bar generally means mass death.
For someone outside EA, it is very easy to go from that observation to "these people are telling me I am basically a mass murderer for not helping, and even if I help, I'll still in practice be a mass murderer because I'll help suboptimally, gee thanks."
Even if very few identifying with the EA community would endorse that phrasing or the implied moral judgment, I can see how people could end up feeling that way. Not sure what to do about it.