I just meant that if you make your imperfections smaller, but then make the whole system smaller to take advantage of that, and then zoom in on it, the imperfections look just as significant, relative to the whole, as before when everything was bigger.
A few factors:
There's always some finite transition region between the fully-exposed area and the fully-unexposed area of the polymer resist area. With optical lithography, to get the smallest features possible, we're basically trying to make a shadow edge that's perfectly sharp and vertical through the thickness of the resist, by pressing a transparent mask with a pattern of thin metal on it right against the resist.
The thicker the resist, the blurrier the shadow down at the substrate. This blur takes the form of partially-exposed resist, which will develop faster than unexposed resist but more slowly than exposed resist.
With electron-beam lithography, there's the focus and control of the e-beam writer, but the sharpness and precision of the exposed pattern also depend on electron scattering and charging effects in the sample being exposed. Again, thicker resist makes the pattern less sharp as it needs a higher electron dose and will suffer more scattering.
We use a chemical to dissolve the exposed resist (there are processes where it's the unexposed resist that dissolves instead, but that's a tangent here). It takes time to dissolve the resist right down to the substrate, and as it's doing so, it's attacking the edge of the pattern too, albeit more slowly, making corners rounder and changing the vertical edge profile of the resist.
Say we get a decent pattern in our developed resist, and the next step is a liquid chemical etch. The etchant will dissolve the substrate at a certain rate. But it won't only etch straight down. If it etches equally in all directions, it will round the corners and etch under the resist, which makes the etched-away areas bigger than the pattern in the resist, with the rounded vertical profiles we often see in device cross-sectional images. This is assuming perfect resist adhesion.
A really tiny wet-etched pit has to start with a really, really tiny resist pattern and may well consist mostly of rounded undercut.
Fluid dynamics and chemistry in confined spaces often mean some areas etch faster than others and lines can get wobbly (this can be a factor in development too).
If, instead of etching, we want to deposit material like insulator or gate material, the developed edge of the resist has to have a profile that the deposited film cannot just run continuously up, or when we try to "lift it off" in the unwanted areas by dissolving the remaining resist, the patterned film will tear unevenly (at best). An overhanging resist profile is best for this. It's very difficult to lift off a deposited film that's thicker than the resist, so in general if you're depositing a film, you're starting with the challenge of a thicker resist layer.
Now imagine you've done a couple of steps on the devices already. You have perhaps some etched features, ohmic contacts, and an insulator, and now it's time to pattern some gates on top. The devices are now 3D. The resist is thicker in some places than others. The insulator tends to charge up in the electron beam. It's all the same principles again but a little more complex.
There is a plethora of techniques and chemistries to mitigate the issues encountered at every step, and you mix and match the things you are allowed to do and can afford to do and the skill level of available personnel to get the best result you can. The more automated you can get it, the better, because it's crazy how tightly every environmental variable has to be controlled to make a complex process reproducible, but in academic labs you still have students with gloves and tweezers and beakers and stopwatches, tweaking their process and device design to make something new happen.