Battery I am using is this: https://tech.scargill.net/ravpower-the-ultimate-pi-ups/

It indeed helpfully maintains stable power output when being plugged and unplugged from mains itself. But only on one of the output ports. Also this is not always true when there's active power consumer on the second port. Other than that, I'm happy I found this powerbank model.