> Roughly speaking, the latency of systems like object storage tend to have a lognormal distribution

I would dig into that. This might (or might not) be something you can do something about more directly.

That's not really an "organic" pattern, so I'd guess some retry/routing/robustness mechanism is not working the way it should. And, it might (or might not) be one you have control over and can fix.

To dig in, I might look at what's going on at the packet/ack level.