Not sure how familiar you are with the rules of D&D, but "lesser wizard" in this context means suboptimal stat bonuses, not limitations on magic.
It shouldn't limit you from using magic, and it doesn't, and that is good.
But, sometimes you will want to do things other than just casting spells (especially if you have run out of spells or if there is anti-magic preventing casting spells), so you can decide if you want to be good at one thing, or good at other things too but perhaps not as much.
But it does limit how good you can be at magic since you can't get as high magic stat, hence lesser wizard.