The magic comes from the hardware being just a bit too spartan. Had it incorporated a bit more dedicated purpose, ease of use type functionality, the ultimate limits would have been significantly lower. (potentially, depending on how buggy those would have been)
The first thing one does is get a kernel running. That's the display, rendered line by line.
Depending upon what one's vision for the display is, that kernel may be simple, flexible, or very complex, timing dependent.
When it's all said and done, some cycles are left during the blanking period, mostly the vertical one, for game mechanics.
Once, during a project, I ran out of bits. Literally, every single bit used. Was thought provoking.
Another thing, I consider pretty amazing is the compiled BASIC available. One enthusiast saw that a lot of the code could be generated, while retaining a fair amount of display capability.
https://www.google.com/search?q=batari+basic+games&sa=X&rlz=...
What happened was kind of unexpected. Ordinary people could grok the very simple language and limited environment.
The same thing happened for them as it did for assembly language programmers, and that is the stark limits made for a keen focus on the game mechanic. Limit choice, and people finish projects, and or execute on visions.
A ton of goofy, often pretty fun, games got made.