It's closer to a counter-flow heat exchanger, but you could still have air only go in one direction at a time. Say your indoor temperature is 20 degrees and the 3 heat exchangers are at 25, 30, and 35 degrees, and outside is 40 degrees (I'm thinking in Celsius, though 40 is a bit extreme). You blow air out until they cool to 20, 25, and 30 degrees. Then you blow air the other way until they heat back up to 25, 30, and 35 degrees, with the air coming in being somewhere in the 20-25 degree range instead of 40 degrees that the outside air is at.

This assumes that the heat exchanger has just enough thermal capacity so that raising/lowering its temperature by 5 degrees would get the air to the same temperature. In fact, it might be easier to imagine if the air doesn't blow continuously. Each chamber could fill up with air and wait for the heat exchanger and the air to get to the same temperature, before moving the air to the next chamber.