The fundamental driver behind Moore's Law is shrinking the circuits. The problem is that the circuits are made up of atoms that deliver electrons, and electrical circuits fundamentally cannot go smaller than a single atom, because electricity is movement of electrons between atoms.

We could, in theory, use other technology (some sort of photon-based computer?) to get smaller (and therefore faster) computers, but that wouldn't have the Moore's law mechanism.