Parents get more time off than non-parents and that's unfair? That's an understandable perspective but here's another one.
1) Children don't get to choose whether to be born. Infants and post-partum women are vulnerable people and giving parents time off helps protect these vulnerable people. Looking at this from the perspective of what's best for the child it's clear that paternal leave is very important.
2) People have a choice of where to work and at least in the US, employers are free to choose compensation that they feel will attract the workers they want. I know several people who have chosen where to work based on paternal leave policies.
3) Life is unfair. If I made a list of injustices in the world an infant spending a few extra weeks with daddy isn't even in the first 1000.