If you want to understand how the GPU driver thinks under the hood read through https://fgiesen.wordpress.com/2011/07/09/a-trip-through-the-...
If you want to see the OpenGL state machine in action, check out https://webglfundamentals.org/webgl/lessons/resources/webgl-...
I'd not come across the state machine site before, it looks pretty useful.
It's more like you have a fairly high number of cores each of which consists of a large number (typically 32 or 64) of ALUs executing the same instructions in parallel.
This means that while you cannot execute a thousand different instructions in parallel you can in theory run somewhere between tens and hundreds of different instructions each across 32 or 64 different sets of input data.
The GPU does not need high single-thread performance so it has lots of simpler and smaller cores, and as a result the total IPC goes way up.
And then of course the GPU has dedicated hardware for operations like rasterization and texture interpolation which the CPU lacks. On the other hand, the CPU needs to run an OS so each core has support for virtual memory, interrupts and other types of instructions that are either not needed on the GPU or at least do not need to be supported by each one of the little cores and are handled in a more global manner, making these cores even smaller.