Using the app Streetcomplete, it's really easy to correct OSM data like street numbers and opening times around you as you walk.
Then mapped crusaders like yourself could first focus on adding numbers in the middle of streets, and then gradually, in binary search fashion, continuing to halve unnumbered street sections.
Once someone has made that effort, then it isn't too much extra effort to do all the house numbers. So, no need to restrict oneself to the middle of the range.
OSMAnd's address search feature already allows you to choose from among the house numbers that do exist for a street, even if the whole street isn't mapped, so it is often easy to navigate to the general ballpark of your destination.