The point is that the things that come easy for us are usually those that we value the least, so, after we put in a lot of hard work and time on something, and we had a great accomplishment, we'd say "if it wasn't hard to do, it wasn't worth doing." It's a self pat-on-the-back.

But I totally understand how if you applied this saying to EVERYTHING then it wouldn't make sense, or be seen as "narrow minded" I suppose.

I mean, fishing is pretty damn easy, but every trip I take is worth taking for the peace of mind.