The source file I linked in my initial comment is honestly the most succinct way to understand how this works, but the TL;DR is that there is a NoPE layer interval parameter passed to the transformer block implementation. That defines how frequent a "no positional encoding" layer is used. The NoPE layers use the global attention mask, which is a traditional application of attention (attends to all tokens in the context window). The other layers use RoPE (rotary positional encodings) and a chunked local attention mask, which only attends to a fixed set of tokens in each chunk.

There is a wealth of literature to catch up on to understand the performance motivations behind those choices, but you can think of it as essentially a balancing act. They want to extend the context length, which is limited by conventional attention compute scaling. RoPE on the other hand is a trick that helps you to scale attention to longer context, but at the cost of poor retrieval across the entire context window. This approach is a hybrid of those two things. The recent Cohere models employ a similar methodology.