Fortunately, I have a physics degree so I am quite sure I am not being misled by the textbooks.

We will face different challenges at different scales, of course, but even as recently as 2016, researchers using a scanning tunneling microscope crafted a 8,192 bit message using less than ten thousand atoms. So, no, I see absolutely no reason why we couldn't have robots within an order of magnitude of the human cell -- we have bacteria which are much much smaller capable of a variety of functions as well as reproducing themselves

You might look into Feynman's "There's Plenty of Room at the Bottom."