That said, just asking people with apparently inappropriate dogs what task the animal has been trained to perform would be a first step towards removing fake service animals. If they can't answer, I assume there is something you can do, but I don't know what that is exactly. It may depend on your state.
In the case of a specifically badly behaving fake (or even real) service animals, section b of the same document says: "A public entity may ask an individual with a disability to remove a service animal from the premises if - (1) The animal is out of control and the animal's handler does not take effective action to control it; or (2) The animal is not housebroken.", so a dog barking or pooping in a business can definitely be ordered removed.
"Public entity means -
(1) Any State or local government;
(2) Any department, agency, special purpose district, or other instrumentality of a State or States or local government; and
(3) The National Railroad Passenger Corporation, and any commuter authority (as defined in section 103(8) of the Rail Passenger Service Act)."
I believe the ADA is https://www.law.cornell.edu/uscode/text/42/chapter-126
[1]Businesses may ask if an animal is a service animal or ask what tasks the animal has been trained to perform, but cannot require special ID cards for the animal or ask about the person's disability.
I find it mildly annoying to see dogs in restaurants and grocery stores, even more so in the work place, but I've heard of a lot of people abusing these laws here in LA to be able to get into apartment buildings that don't allow pets.
As a neighbor, if you're animal isn't well-trained, I don't want to live near it in an apartment building. Were I someone allergic to pet dander, I wouldn't want to unknowingly live somewhere that a dog/cat had lived for years no matter how well you cleaned it. Especially at the expense we have to pay for rent in this city.
I think taejo has it right in her/his comment about how US owners act about this stuff:
"I don't think this would fly in the US, where cultural attitudes tend to be less "we've all got to do our bit to make society pleasant for everyone" and more "it's a free country and this is my dog, I'll train it if I want to"."