If you go on big rides in the middle of nowhere then you can't really afford to have components fail on you. I always take a few tools, small parts etc. but if something like a pedal or stem fails I'm walking a long way home. Quality components are made of different materials (e.g. higher grades of Alu) and manufactured using better processes (e.g. cold forging, not stamped) so they are stronger and lighter. I also don't pick the very lightest components since I need durability.

Aside from that, the joy of riding is what gets me out. Many cyclists choose tough hill climbs for added fitness gains and challenge. There is a good kind of difficulty (steep grade somewhere picturesque) and a bad kind (unresponsive bike that doesn't roll well)