Have you tried mold [1]? In my experience, it makes a dramatic difference to link times.

[1] https://github.com/rui314/mold/