I think I see what you mean, but isn't it true that if there is light shining on the object, then the lamp must be in the light cone? It is literally impossible for something outside the light cone to have any physical impact on the object at all.

We do know the sphere of causality already, no computation needed: it is the sphere defined by the distance light has had time to travel. You don't need to simulate anything outside that zone to know that it is impossible for matter outside it to physically impact the center.

Think of it this way: if you want to simulate Earth through the year 2000, you know you don't need to simulate Alpha Centauri after 1996, since it is more than 4 light years away. You can know this without doing any computation at all.