Remember all those components can't "talk" directly to other components. They must all go through the CPU. So if your graphics card want's to make an internet connection, it must go to the CPU which will then go to the network device.

So if you don't have control over a peripheral (say your GPU for example) then yes, it could be doing things you have no control over. But it can't interfere with anything else unless the CPU says so.

But if you don't have control over your CPU, the "central" processing unit, then it's game over.