I wouldn't go too far past that if your goal is to get the hang of it. SDL and the various GUI libraries abstract more away but the Win32 API is easy to work with and a good starting place. X11 has another layer of complexity as it works via sockets and that should be ruled right out.
Having said all of the above i also wouldn't do it beyond learning. I've made my own game engines from scratch, i've written games in assembly, got them down to a few KB in size. No one plays them and i'd have to link to webarchive to find them. No one wants to play a very simply game that took a lot of effort to make that doesn't even work without emulation these days. It doesn't even look good on a resume.
You know what i actually use for productivity these days? Unreal or Unity.
This is just for learning purpose. I have a fetish about assembly language and want to use it as a daily drive for a certain project. Definitely not for work or anything money related!