As a person who currently uses freetype: It's complicated, it's kinda slow, it's buggy, and it's unsafe. It's the most popular choice for good reason but there's definitely room for improvement. I have a couple TTF files here that cause off-the-shelf freetype to crash in release builds but not in debug builds...
Parts of it have aged poorly, as well - for example you might get the impression that it offers kerning support, and while that is technically true in practice it is not because most kerning data is not stored where FT thinks it is, so you'll ask it for kerning data and get nothing. IIRC the solution for kerning with freetype now is to have a shaper like harfbuzz do it for you since the shapers understand the new kerning data.