> For dark mode, the site shows white text on pure black, to help shave energy use on OLED displays. For light mode, the site shows black on pure white.

That's exactly the problem. Too much contrast is a thing with text and can make a website uncomfortable to look at (for me at least). It's hard to find a source for that, but most people seem to agree (see ~95% of other sites).

That's not a problem on commodity LCDs (maybe the one you're using) because "black" is fairly bright, but on OLED especially that's almost infinite contrast.