Not exactly a scientific experiment, but based on this video it seems like fire would have significantly less visible intensity for it to make a meaningful difference.

https://www.youtube.com/watch?v=JBL3NdIK4DI