It's actually not illegal in most places.
None of the 50 states have a blanket law prohibiting riding on sidewalks, so it's purely a matter of local law. It's illegal in some cities, not in others, and almost never in anything smaller than a metropolis.
Some states do have laws about what constitutes legally riding a bike on a sidewalk. For example, Oregon requires bicyclists to yield to pedestrians, and to generally go slowly, slowing down to walking speed when crossing a crosswalk or driveway: https://www.oregonlaws.org/ors/814.410
Common practice seems to vary a lot by area. Riding a bike on the sidewalk in SF would be unsafe, unnecessary, and against city law, but in suburban and rural areas with high-speed-limit roads (~50-55 mph) without bike lanes or paved shoulders, but which do have sidewalks, it's quite unusual for people to ride in the road. In suburban Houston, for example, I would be really surprised to see a bike in the traffic lanes on something like Texas Hwy-3, but it's not that uncommon to see bikes on the sidewalk.