The social aspect is definitely important. Being someone who can work well with others is essential. Being able to communicate complex tech things to non-tech people is very valuable for career advancement. If this is a skill you lack, work on developing it.

If your manager thinks you suck, then yeah, you will not be getting a raise. In fact, they may be looking for a way to push you out. However, if they do think you are good, then understanding where you are pay-wise relative to the industry can be a good argument to make.