They must be shutdown to be refueled regularly, but it can be planned for.

Refueling usually takes a long time, minimum three weeks, and a month is common:

https://www.outagecalendar.com/

Forever? No. For the design life of the reactor? Possibly, for a purpouse designed reactor type.

If you want it today, there's the nuclear radioisotope thermoelectric generator, in use since the 50s and used on many spacecraft.