>Is it because we can just yeet waste substances towards the sun?
It is actually quite expensive to get to the sun.
It is actually quite expensive to get to the sun.
However, this also means that to hit the Sun, you need to remove more or less all of the energy that Earth starts you with. This is the key insight - if you do a bit too much or a bit too little, it slingshots around the Sun (getting close to it but not going into it) and stays in orbit rather than landing in the Sun.