Edit: I guess I'm misremembering how Chrome works, or maybe they changed it since I've last used Chrome seriously. I'll leave the comment so the replies make sense.
Original comment:
I'm kind of surprised that Mozilla went with a per-tab approach here, since they went with per-window for private browsing (while Chrome did the opposite: per-tab private browsing and per-window profiles).