Nevertheless, Firefox is part of the problem. And performance is still sub-par since I applied the fix (although not quite as bad).
Gotta love the people that answer "works for me".
They are the biggest reason why I use an extension to stop or unload background tabs.
Switched to uBlock origin and removed the other two, performance is normal again.
On linux I also saw a lot of benefit to making sure hardware acceleration was force enabled (it's disabled on the platform by default for some reason; I also had to start optirun-ing it on my laptop). Firefox has literally never been faster or behaved better since making those two changes.
Startup times are only comparable when you start Firefox for the second time and at least for me, Firefox is faster then. Rendering and the developer tools however seem to be slower in Firefox indeed, sometimes significantly.