At the very least, every wall outlet and power bank I’ve seen has text somewhere that tells what it outputs, with the powerbanks aswell having capacity, both amp hour and voltage. Though I will say the text is rather small.
I mean, I can work out what they probably are with a little general knowledge and arithmetic, but it isn’t as clear as you suggest.
It might also support higher voltages (up to 20V), though if it doesn't specify I wouldn't assume. If you need higher voltage, you'll have to look for a battery that specifies it. (And hope that the package isn't lying.)
I've never seen a vendor that specifies 3.6V, the best you can hope for is that they give mAh and Wh so you can derive it yourself. If we need the Wh rating to keep the mAh rating honest, the mAh becomes redundant before the Wh does.