I remember there is a study about the alignment cost. Basically the more restrictions and limit you put on a model, the worse its general performance becomes. Things like a ban on violence, race, or any other sensitive topics effectively throttle or change how the model "reason" or connect information within its network of parameters and result in degraded capacity.

I wonder if this is the reason behind all of this.

Edit: the study: https://arxiv.org/pdf/2308.13449.pdf