I'm sure they could, but that makes attacks only slightly more difficult.

It's fairly trivial to look at common key blanks and visually identify the bitting (the "combination" of cut heights necessary to move the tumblers in the lock to the shear line). Sometimes the key has the bitting stamped right on it, numerically, too.

A stranger could use your local hardware store to copy your key, and this has been possible for decades.