It is unable to find edits: https://twitter.com/mengdi_en/status/1763338245696274469
It hallucinates changes badly: https://twitter.com/thomasahle/status/1763408041960231010
To be fair, finding diffs is an odd way to use LLMs, but does show the highly variable abilities of these models over bigger contexts.
In other models, reasoning performance declines with more tokens (no data yet here for Gemini 1.5 Pro): https://arxiv.org/html/2402.14848v1
Note in that last chart that significant declines in reasoning are apparent over just a few thousand tokens.