yep, was going to say that this is probably the real source behind the mystery. Early OS X is basically NeXT when Jobs came back to Apple. Doom was developed on NeXT, so id Software and Carmack already had a history of using BSD. They also released Quake for Linux around 1996, which may have used X11 (I cannot recall, maybe it just depended on 3dfx/nvidia without X11 at the time... it's been so long)
The original quake on Linux used svgalib and was done by Dave Taylor and GAlexand (EFnet #c??)
There are writings floating around from Carmack that he was playing around with the OS X prototypes back when it was called Rhapsody in the late 90s.
I also thought I read somewhere that Carmack persuaded Apple to bet big on OpenGL for OS X. (Ironic now that it's deprecated.)