The article doesn't even profess to answer the "how" only the "what" as evidenced by this quote:

> we asked the Google DeepMind project team to explain what long context windows are, and how this breakthrough experimental feature can help developers in many ways.

At this time we can only speculate on how they achieved it. The best guess so far is RingAttention. See https://github.com/lucidrains/ring-attention-pytorch