I think authors from the 50’s/60’s/70’s often wrote shorter books because they worked their way up writing for pulps and magazines. After the market for that stuff dried up authors seemed to start writing more open ended stories.