X absolutely supports depths of 1 and 4 bits per pixel, along with 8bpp, with its VGA server: https://www.xfree86.org/4.8.0/vga.4.html
I hadn't thought of 1bpp because it isn't interesting on VGA hardware. But you're absolutely right about 4bpp having been a thing.