It's very annoying. Especially since the reason I use gnome-characters is because I'm in the middle of typing something and I need a character that's not easy to type on my keyboard, so it's already a distraction from what I'm trying to do.
As an aside, I've been trying out Ubuntu for the last 6 months, and this is one example of the lack of polish I'm seeing. I don't remember installing gnome-characters explicitly (it's possible I just forgot doing so), but there are two versions on my machine, one of which happens to have this crazy startup lag. Another annoyance is that firefox came installed as a snap, which meant that all of my downloads went to something like ~/snap/firefox/Downloads instead of the expected ~/Downloads. I don't remember these kind of things happening when I ran Fedora...