Absolutely yes people are. I'm active in weight loss communities to support my own weight loss, and yes we are careful about our fat consumption.
The health difference is large for someone trying to lose weight. A tablespoon of butter isn't that filling, but contains about 150 calories. That's equivalent to a whole pot of non-fat yogurt or two eggs, both of which are more filling and give you more protein.
When making eggs, I try to use about a teaspoon of butter, which still gives some butter flavor, but lets me save more calories for eggs.