What those criticisms seem to be neglecting is that those--particularly for apartment buildings--these temporary visitors impact beyond just you.
In Australia, for example, one useful metric used by people when assessing apartment buildings to buy in is to look at the ratio of owner-occupiers vs renters. Generally the more owner-occupiers, the better the building.
Renters don't tend to look after properties, tend to be less mindful of behaviour that impacts the neighbours (parties, etc), etc.
This goes even moreso for really short-term tenants. There have been examples of this already (like the woman who had her house trashed in SF after renting it out on AirBnB). Such extreme examples aside, you are raising the risk of damage or theft to the property of others by bringing a stream of strangers into your home.
Yes most of the time it works out just fine but sometimes it doesn't and your neighbours didn't sign up for you opening a hotel.
Would people feel the same way if someone was operating a brothel in their house or apartment? After all, it's their property and they can do with it as they please right? What about operating a bar or a club?
This is not as one-sided as some suggest.