Gamma curves do not affect black and white themselves, only intermediate grays. It’s true that grays are usually used to draw antialiased black and white lines, but we can also think about “ideal” (axis-aligned, pixel-centered, non-antialiased) black and white lines. By framing this as a question about gamma, you’ve implicitly assumed that “ideal” black lines on white would have equal perceived thickness to “ideal” white lines on black.
This is not the case, as typographers have known for decades. The right way to draw perceptually equal-thickness black-on-white and white-on-black lines is to vary the line width (perhaps even by more than a full pixel if the lines are thick enough!). Gamma only comes in afterwards, to help us reproduce the varied widths accurately, and the accurate way to do that is in a linear color space.