> The reason is a doctor told my parents not to feed me any sugar until age 2-3

What I take from this is that the actual reason is, "I had parents that were concerned enough with my nutrition to not feed me much sugar as a child, and those healthy eating habits probably didn't stop after age 2-3. I now cannot gain weight, even if I eat what I perceive as a lot of food (because I grew up eating healthy food and portion sizes)."

I was one of those "I can't gain weight/put on muscle" people for a long time. I eventually tried eating literally double what I would normally eat, and as expected, I started gaining weight.