Freedom of religion is something else than making exceptions to other laws because of religion.
Freedom of religion is the right to believe whatever you want and not be discriminated because of it. There needs to be a freedom of religion because there is such a strong tendency to discriminate against minority religions.
Similarly the law needs to be the same for men and women, but the tendency to discriminate is so strong that there needs to be a law against discrimination based on gender.