I'm no longer convinced that "priests abuse children because they're celibate" is an adequate explanation. It seems plausible, but it lacks the ability to explain the rates of sexual abuse of children by non-celibate priests, teachers, family members, etc. It is also my understanding (but I am not expert, nor do I have any personal experience here) that sexual abuse is usually more about power dynamics than the sex itself; while a lack of a sexual outlet can be a driving factor, it often isn't the main one.
Okay, now everything below is completely unrelated to the child abuse problem, just for clarity.
I'm generally pretty negative on the wealth and power of the Catholic Church, I think it's kind of gross that an organization setup for spirituality is so damned rich in this world, but I actually do see some sort of argument for a celibate clergy in the context where that norm was first created. Remember that the Catholic Church is effectively a product of the late Roman Empire. As the Western half of the empire fell, the Catholic Church stepped in and took the form of imperial civic functioning; a basilica was originally a Roman public building, not a religious one[0].
In this context, it was considered normal for the son to inherit the wealth and job title. After all, wasn't Constantine followed by his sons? The issue here wouldn't be priests inheriting their local church as private property, most early churches were private property given that they were clandestine rooms in the back of personal homes, but the real issue is the possibility of bishops and similar inheriting their seat and peeling their entire bishopric away from the church that they were working on centralizing. The early days of the church were very concerned about unity, see the Council of Nicaea, and similar issues were happening to the crumbling empire with frightening regularity at that point. When this norm was set in ~304 CE, everyone had just lived through the "Crisis of the Third Century"[1] a scant 20 years earlier, which was largely characterized by uncertain rules of succession and breakaway states (among other things). The risk of a son inheriting a title and then trying to turn it into a personal fiefdom was not an abstract concern for these men, they had just lived through it.
This first rule of celibacy was unevenly applied, and was a mixture of a requirement or a social norm depending on the exact diocese[2]. When it became a universally enforced rule in the 11th century, I think the church had a lot less to fear from bishops going rogue, and the whole thing looks a lot more greedy than anything else. Needless to say I think the entire practice is indefensible now, and probably actively harmful as the church sends out celibate priests to advise people on matters of marriage, family, and sexuality: issues that priests have no personal experience with. This is really obvious when you consider official Catholic policy against birth control and the American Catholic lay population that has a mere 0.1 more children per parent compared to their Protestant peers (2.3 vs. 2.2)[3].
0 - Obviously the Catholic Church took a bunch of holidays from the official cults of Rome, Christmas most famously. But the functioning of the Church is quite different, down to the architecture and organization. A Roman citizen from the Late Republic would probably think that a modern Church is a public building and not a religious one; Roman temples were basically places where religious artifacts were stored, and ceremonies were performed outside in the view of the gods, unlike the Church where ceremonies were performed indoors. Constantine's involvement in the early Church certainly was a factor here too; when he started giving stuff to the Church, including access to imperial infrastructure (roads, messengers) and buildings, they all took the form of civic structures because he was the head of state, and not a religious figure (the title of Pontifex Maximus having lost most of its religious meaning at the time, and was effectively just a title the emperor held).
This imperial connection helped the early church greatly. Not only did it help it form an effective bureaucracy, but it also endeared it to various ruling dynasties in the former Empire. It's now believed that the conversion of England to Christianity that started under Æthelbert of Kent was partially because as a Christian king he would gain access to the administrative capabilities of the Catholic Church, including the last remaining bits of Roman imperial authority that had long left England by this point. Obviously his Christian wife probably didn't hurt either.
1 - 49 years of military conflict, with 26 various men having claim to imperial title and the empire being temporarily split into 3. Plenty of emperors in that time had a time in office of days or months, not years. The men running the Church in 304-306 CE, when the Council of Elvira first enjoined priests to celibacy, would have lived through the latter parts of the Crisis (it ended in 285 CE).
2 - Another Roman civic term, specifically for portions of the empire. The early church just copied these from the empire directly, not just the word itself; the first church dioceses were the same as the civic ones. This would be a bit like if a new religion called its areas "counties" or "states".
3 - https://www.pewforum.org/2015/05/12/chapter-3-demographic-pr...