Theoretically you could boot X without a hardware framebuffer emulator. X supports rendering to normal RAM as well (for example, for VNC connections).
Actually, if you go read my source code, you'll see that the emulator DOES in fact emulate a framebuffer. In fact I even have code in place to output the image. I just didn't connect a graphical LCD to this particular built. It is, however, supported.