Remapping only L in Oklab will be much more efficient than using the standard combination of invert() and hue-rotate(180deg), because it keeps hues correct while messing up saturations and making reds darker. With a and b remaining the same, any red highlights remain red. What about images and graphics? Remapping lightness in an image will make it seem like a negative image, hence you wouldn’t use image XObjects, but then diagrams that have white backgrounds end up as white boxes on a dark background.
Currently I remap all backgrounds, text, and vector graphics, and I leave raster images alone. This works great for documents whose only raster images are photographs; some documents do have other raster images that might even have white backgrounds instead of transparent, so that's not ideal, but it seems to work decently well in practice.