The space where tabs go (when horizontal) isn't removed in vertical tabs mode. So using vertical tabs doesn't save any extra on vertical space, only uses additional horizontal space.
edit: the latter (or Sidebery does do it but my previous solution didn't and this is now redundant, I suppose), I have:
#TabsToolbar {
visibility: collapse;
}
#sidebar-header {
display: none;
}