Good points. It seems you might enjoy Fibery[1], it addresses most of these problems (and has internal whiteboard as draw.io replacement as well). But I’m biased as a Fibery founder.
But... even after looking at all the four separate landing pages I have no idea what exactly fibery could do for me