That's one thing I have been thinking about. I think Haskell and other statically typed functional programming languages would benefit from a more interactive REPL like experience for "non-pure" programming where you are sort of changing the wheels on a moving car.
This has a very reactive flavor, so I think you could have a sort of functional reactive approach where your hot-swappable definitions are represented as behaviours over time.
How to make that get out of your way until you actually want to swap things out, I have no clue. I think it will involve some sort of effect system, but I'm not sure if that will solve the problem.