This is not always true.
Your buoyancy decreases as you go deeper (and have more water above you, causing more downward pressure).
At a depth of 30 meters (100 feet), the pressure is 4 atmospheres, 4 times that of sea level.
Furthermore, not everyone is positively buoyant. The less fat one has, especially if not wearing diving suit, the more likely staying afloat/level to become harder.
As the depth increases, lungs and all bodily cavities shrink, further adding to one's density.
So, at 30 meters, it is actually quite a struggle to swim back up, even more so if you've lost your fins.
As for weights, I used to work out and exercise daily. During that time, I also dived few times a week spanning various sea conditions throughout the year and never have once taken weights as I was already negative.
Fast forward 2 years to today, I put on 15kg (33 lbs) and I need 10kg weight to enjoy my dive. But on the upside, I'd probably skyrocket to the surface if needed, lol :D