Switching/routing usually requires significant information processing (e.g. decode packet header, match destination address against routing tables, etc.). This necessitates 10k or more gates. All-optical computing can't deliver this level of integration density, nor the performance at reasonable power levels.

Maybe there will be some smart way to pre-encode routing information onto packets to reduce processing requirements, but I doubt that such a network could scale.