set browser.ml.chat.enabled to false in about:config
I don't know why they defaulted to placing it right next to my back button, but you inspired me to check and yes, one can right-click and either remove that dumb Sidebars button or hide it in the overflow menu