While developing some application, did you ever feel that your SDK is lacking some feature which would make your app development much easier?
So think the video driver as library and the windowing system as an application using video driver library.
So think the video driver as library and the windowing system as an application using video driver library.
However, as tinco explained[0], it seems that there is no "standard graphics api" for linux, and the "link" between window servers and graphics drivers is hard coded. So you do in fact have to patch the video driver to provide support for a new window server (on Linux at least).