> If I start lifting 20lbs and 3 years later I still lift that of course I won't lose weight.

I don't understand, your body optimizes, and uses 20% less energy. However, you are still using the 80% which is more than not doing anything at all don't you?