You shouldn't be sharing any air with you neighbors, so I doubt ducts are to blame. You can build highly sound-proof timber-framed walls, but most people don't because it's more expensive. Like everything in modern construction, the cheapest permissible option usually wins.
> The apartment buildings I've lived in without central A/C (in Atlanta and San Francisco) were consistently quieter than the ones with central A/C.
When were the units built?