In the eye, cells only fire if the signal between the neighbors is great. There is also higher resolution concentrated at the center and drops off as you move away. Additionally the brain processes signals in context of the last signal it received. So if you put that all together, on a small screen the gaps are small enough that in the center of the eye, fovea, it will be processing more red than brown. You can also see the ball is actually not just brown. Its shaded which reduces the "fire near highly different neighbors" effect. The red though is solid.

There is also the fact that the eyes resolution drops off towards the edge. This makes the effect even harsher in the corner of your eye. As your eye moves from ball to ball your brain will be processing the new higher resolution versions based on the previous versions you saw in the corner of your eye. That effect fades away but your eye is actually always moving doing something called saccades, on a small screen with many prominent points it will do it a lot.

With a big screen alot of that goes away. The gaps are larger. The brown fades more slowly from light to dark so is percieved as a stronger color. Objects are larger so saccades will be less effective because they will be searching fewer objects rather than many in a tight screen. Lastly the lower resolutions at the edge of your eye will pick up more brown as well.

So there most definitely is physical aspects.