Plenty of late 2600 games look tons better than early games. If you look at Combat vs late life Activision games like Pitfall! or Keystone Kapers, it's a huge difference in visual quality.

It's still nothing compared to early NES games, of course. And late NES games certainly got a lot nicer looking.