NIRCam has quantum efficiency of 70 to 90% depending on light frequency. https://jwst-docs.stsci.edu/files/97978241/97978242/1/159607...
MIRI has average efficiency of about 50%. https://jwst-docs.stsci.edu/files/97977640/97977649/1/159607...
Internal noise of electronics is not significant compared to thermal noise at operating temperature of MIRI (7 Kelvin).
Further small improvements might be possible in the future but no 'orders of magnitude' can be expected without going into even colder temperatures, that would require total redesign.
At the end of the day, efficiency can be offset by simply taking longer exposures so it is nice to have but does not have to be super optimised. Achieving diffraction limit is much more important and JWST's camera do that.
https://jwst-docs.stsci.edu/jwst-near-infrared-camera/nircam... https://jwst-docs.stsci.edu/jwst-mid-infrared-instrument/mir...