In case of an open source project one of the main problems with LLMs is that if a contributor does contribute infringing material, inadvertently or not, the project will be left holding the bag for it. The contributor may disappear because all that's known about them may just be an alias and an email address. This is especially true if a project doesn't have the backing of a large, well-funded organization that can fight out legal battles on behalf of the contributors. The legal situation on Internet-trained LLM usage isn't clear and I can totally understand a small, underfunded project not wanting to go up against large corporations. I think respecting the project maintainers wishes in this case is the right thing to do.
I've seen several examples where an LLM was used as a tool for accessibility (vision assistance, etc) and I do think that that's going to be one of the areas where it will be most valuable to society as a whole. Unfortunately, the legal system moves slowly and it will be a few more years until this mess gets cleaned up. I also think you are doing the right thing and I wish more people would follow your example. If a particular project deems the risk of using an LLM too high, contributors should follow the wishes of the maintainers and not contribute to that project with the usage of an LLM. Depending on the country, petitioning lawmakers may also help move the needle in the desired direction.