Not sure I follow. It's amazing how these little bits of social knowledge ferment the flavor of the story. It's a lot like learning to taste beer or wine. Once you take the time to sit down and learn to pay attention to your senses, you can identify things like acetaldehyde, diacetyl, and mercaptans. It's fun teaching people how to sense things they weren't aware of before.
The author seems to have done a little trick. By assuming this little fact, it serves as a rationalization for Dad's emotional neglect. It explains his behavior in retrospect, and I get why that makes sense. But that doesn't mean it's true, it doesn't have any predictive power because it reverses subject and object. "We need work" is a social fact, nothing more, and it's only true as much as it's believed to be true. To me it stands out like pickle in pancake, maybe it can for you too.