While I agree that those terms are unclear and might mean different things to different people, I think "banning" them is solving a non-problem. I don't know anyone who would ask a developer to "rewrite the code to not be ugly" (without providing specifics). I think it's a useful informal term for when specifics aren't needed: "making this change will require us to touch the X module, which is really ugly, so factor that into your time estimate" (if both developers know the X module, this needs no further explanation). In other words, I've not encountered any discussions where "ugly"'s ambiguity has caused problems.

Sometimes "ugly" really seems to hit the nail on the head, like when code fails to communicate what it does or why it works (e.g., due to an implementation that looks like it resulted from brownian-motion trial and error).