Selling Lisp by the Pound

I've been fascinated by the interactivity of Common Lisp for a while, and would really like to be able to, say, redefine functions of a running program in other languages as well. I feel like it would be extremely beneficial for debugging long-running programs.

However, I have never actually been able to program in Common Lisp without repeatedly getting to the point where it seems I have to restart the whole system. I suspect because somehow the state of the code and the state of the system got out of sync. So until now I have always given up, and used C-like languages after all, because there are more well-tested libraries available, and because I never managed to use the full potential of Common Lisp anyway.

I would really like to learn how to use Common Lisps full potential though.