While not the issue you're running into, I think VP9 is required for anything over 1080P on Youtube. I used to have to run h264ify[0] because of certain GPUs not supporting it, and it putting it all on my (also aging) CPU.
The only thing you can really do is try enhanced-h264ify[1] and play around with the codecs to see if one works better for you with Firefox.
I also have an Intel 11th gen CPU, but desktop form of 11900K, so I may play around with this in Firefox to see if I can duplicate your results but I also have a Geforce 1060FE. I leave Intel Driver Support Assistant[2] installed on all of my machines. There is a conflict recently between it and the driver version that MS is pushing out over Windows Update, so I have hidden that driver for a couple months now. But you may want to at least give that driver a try as well and see if it changes anything.
I've consistently used Firefox since 2002, never having moved to Chrome, but made the switch to Microsoft Edge last month. There's a few reasons for that, which I won't bore you with here. But if you do decide to make the leap, I can say as a very longtime and formerly extremely dedicated Firefox advocate, I don't regret it.
Hopefully something here helps you, if not, Edge is Chromium Done Right.
[0]https://addons.mozilla.org/en-US/firefox/addon/h264ify/
[1]https://addons.mozilla.org/en-US/firefox/addon/enhanced-h264...
[2]https://www.intel.com/content/www/us/en/support/intel-driver...