One side (the mainstream) claims vaccines are effective and safe. The other side, claims that no, we do not in fact know that the vaccine is safe and therefore you should not get it and take your chances with the disease.
The problem is that we don't in fact know that the vaccines are truly safe for everyone, because mRNA technology (while incredible) is new, but from my personal interactions I've observed that most intelligent people see that getting the vaccine increases your expected quality of life given you've never had COVID before. However, those who don't believe this will take any claim made by the mainstream (vaccines are safe) as evidence that they are hiding something, further deepening their suspicions.
This is a feedback loop; the more people that refuse the vaccine, the more the mainstream will push for more vaccinations, causing more people to refuse the vaccine. All because the nuance of "we don't know that this is as safe as taking a walk in the sun, but we have solid evidence that it improves your E[QOL]" isn't easy to communicate.