It's been studied in the past (NERVA, et al.), and does seem to offer a specific-impulse advantage in the extremely high temperatures achievable by routing propellant through the reactor core. (So-called "closed-cycle" systems heat reaction mass outside a fairly conventional primary coolant loop; "open-cycle" doesn't
have a coolant loop, but rather ejects hot coolant directly as reaction mass. There was also a nuclear-powered cruise missile design, I think called "Project Pluto," on this theme, although not directly related and so far as I know only studied in theory.)
It does of course still require reaction mass, but theoretical efficiency seems higher than other feasible systems; it wasn't really explored much in the old days due to the general post-Apollo builddown, and also due to the concerns of radioactive debris in case of mishap and radioactive exhaust if the system were used as a launch motor. (If not used to launch, it would need some orbital assembly and a lot of heavy lift, given not least the massive shielding required for remotely survivable crewed use.)
Another fission-based propulsion system, not relying on a steady-state reaction, was also studied under the name "Project Orion." I'll leave that one as a diversion for the interested reader, mainly because it seems unlikely anyone not already familiar would believe a word of any description I could give.