Non-chilled dragonfruit would be probably quite bland, chilling completely changes the flavor, the cooler the better. It's same for me with gummy bears, I have them in fridge, like them chilled, but at room temperature they are meh, so can eat them only for short time after taking out from fridge.

As I said, if you wouldn't know bad durian reputation and just had blind test, you would probably not notice particularly bad smell, it has some distinct smell/fragrance, but I wouldn't call it bad. Some Czech cheese and many other European cheese varietes have way more unpleasant smell than durian, if I had to choose in what smelly enviroment I wanna be durian would for sure beat most of this European cheese.