But you also want to know why it classifies a person as sad/happy/etc.
Then you make the model have 2 outputs. The first is just a classification of the emotion as you'd have normally. While the second is which parts of the image contributed most to the classification - e.g. a heatmap over a smiling mouth in one case or over the squinted eyes in another.
You can do this with pretty much anything.