Unfortunately while you shouldn't try to sync with others publically is a good theory, in the current implementation just opening the share sheet to share via messages or some other app will trigger AirDrop discovery. So you'd need to actively turn the feature off.

In some ways it seems having the "Contacts only" mode enabled is almost worse than having it set to everyone. The obvious problem with having it set to everyone though is you can easily get airdropped things by people around you. Which can be a great joke but since it displays preview of image receives can also be used in harmful or abusive ways.