It's probably because you have not heard it often enough to just not think about it. If I were to describe a sound as being rectangle, it just doesn't make sense and I have to think about it and try to interpret it. The extra effort in trying to decode the meaning is just annoying. In contrast, you've probably heard the phrase sharp sound. It's equally as technically meaningless as crisp sound, but you probably get it and aren't annoyed by it.