In the end I had to remove all messages, and process them using Claws client, which gives much better control and a feedback which messages were transferred and which failed. The only issue I had is that some messages were transferred multiple times, but Claws even has a solution for that by using "delete duplicate messages".
Basically, I did it semi-manual, since it was the only way it worked.
Clicked to "get all messages" on the Gmail account. This is important. Otherwise, it would need to first fetch messages before doing the copy.
For smaller labels / folders, I was just right-clicking label and choosing copy. It would then create a folder in the new account and copy all messages. For labels with a lot of messages, I would create a folder manually, open a label by clicking on it, select manually let's say 500 messages, and calling copy. If copy is successful, I'll mark copied messages in another colour, and proceed copying the next bunch.
In case of any error, and there will be errors, I would just retry copying. That way, I ended up with duplicated messages in the target folder, but that was easy to clean up using Claws option to remove duplicated messages in the folder. The errors may be due to network timeouts, rate limiting on both Gmail and new provider side.
It may be tedious if you have many folders with many messages. I wanted to avoid using any external tools for the purpose, and was too lazy to try to program something. YMMV.
I also took the chance to actually clean and reorganize my mail archive as a part of the process, and ended up not transferring / deleting two thirds of my mail archive during the process.
The general problem is that Mailfence has what I guess you could call DDoS protections in place and opening too many simultaneous IMAP threads trips their defenses. I have my ~/.mbsyncrc set to only use one thread when writing to Mailfence to avoid tripping the trigger and it runs like clockwork now (cron timer doing IMAP sync writes from the primary). $0.02, hope this helps.
I wouldn't trust Thunderbird for more than a few dozen messages I could hand-verify. For anything more, use a dedicated tool that provides logs and can gracefully restart.