There are several spectral renderers out there, such as Weta Digital's Manuka. I don't know if they bother with parts of the EM spectrum that are outside of the visible range though. I imagine UVs can be important to model in some situations.
IIRC the issue is that if you can ignore fluorescence, then reflection is simply an element-wise multiplication of the incoming light at the wavelengths under consideration[1] with the reflection coefficient of the material at those wavelengths. With fluorescence, that turns into a matrix multiplication, with obvious speed implications.
If only a single wavelength is considered at a time, then the wavelength must change upon reflection, otherwise there's no way for the fluorescence to occur. That can also have performance issues, for example conversion coefficients to/from regular color spaces needs to be recalculated.
At least that's my understanding having worked on a physically-based renderer which did do spectral rendering but not fluorescence.
[1]: using for example binned wavelengths or stratified wavelength clustering.