First off, the exposed engine isn't what makes a vehicle loud, unless you count the valve train noise and primary chain noise which honestly is negligible compared to everything else. What makes a Harley loud is the fact that it doesn't have a resonator muffler but basically a pipe with a baffle. Example of a resonator muffler: https://www.quadratec.com/sites/default/files/knowledge_base.... Example of a motorcycle muffler: https://thekneeslider.com/images/hddynamicexhaust.jpg
Second, a modern stock Harley tops out at 79 dB which is below the EPA requirement of 80 dB. This is no louder than a modern car. In California specifically this is also enforced and you can't (to a first approximation) register a vehicle that exceeds this (CARB regulation).
Most Harley you hear that are loud use aftermarket muffler or full exhaust systems (larger diameter, no catalytic converter, etc.) which are labeled as "for closed course only" because they aren't street legal. Still, motorcycles by and large can outrun cop cars so nobody bothers pulling them over. Also, most police officers don't carry dB meters to be able to determine if after you registered the vehicle you swapped the exhaust system.
Now for the reason: yes it makes a huge difference in the feel of you on the bike. The sound is half the equation. And no that doesn't justify bothering all your neighbors, but it is a visceral feeling you can't get any other way. Some people want it and will break the law to get it.
The good news for you is that bikes are getting quieter and more efficient and most people are realizing the effects of hearing loss from long term exposure to loud exhausts. However you will never get everyone to stop: there will always be a small minority that wants a loud vehicle and people who don't participate in this activity will always have a hard time understanding why.
If you want to have an experience, take a week off, rent a muscle car, and ride it cross country on a road trip. It'll be life changing, I promise.