If reality had an infinite amount of detail (i.e. matter could be arbitrarily small), we could make storage media as dense as we liked by encoding ones and zeros as the presence or absence of tiny bits of matter.
The alien's stick is a version of this, albeit an exponentially inefficient one restricted to codewords of the form 1111...0000.
In practice, if atoms are about N orders of magnitude smaller than macroscopic objects, we can fit (very roughly) 10^N bits of information in an object, and the alien's method can only fit, as you said, roughly N bits.
Of course, existing storage methods are somewhere in between, because 1 gram of storage media can hold way more than 23 bits but way less than 10^23.
(I'm handwaving past some important distinctions, like the distinction between the size of atoms and the level of detail in the physical world. In classical Newtonian physics, things can be made of particles but the particles can have perfectly continuous positions, so that there's still no ultimate limit on measurement detail. Quantum physics changes this -- although this gets complicated because of the holographic principle; many physicists think the ultimate information limit grows like the 2/3 power of volume, instead of linearly...)