I discovered this recently when I logged in to website X in a Private window, then closed the window without explicitly logging out, and returned to X later that day. I had another Private window open (with a completely different site). I was still logged in to X! because that second Private window had preserved the session.
This means, of course, that sites you visit simultaneously in separate Private windows are not isolated from each other either.
I still am mostly happy with Firefox and use it daily, but this was a completely unexpected and unwelcome behavior.