The key in situations like this is to always stay professional. Stop working immediately and make it clear that you'll start going again as soon as the check clears, but stick to the high road. The client will turn petty on you, will come up with all sorts of slanderous attacks on you, your work, and your character. That's fine. Let him vent. Then calmly explain once again the concept of "work for hire", and how you're perfectly happy to continue doing work for him, provided he pays his bills.
This may or may not work. On the off chance he pays up, you get to put him on weekly invoices payable immediately on receipt. If it doesn't, chances are you're out a bunch of money. Either way, try to suppress your natural geek sense of vengeance. You're the professional here. Never act otherwise.
Now, as a pro, you do have one option. Since your invoices are itemized and dated, and source control is itemized and dated, you're well within your rights to accurately roll back the deliverable to the state it was in after the last paid bill. Tell the client that you don't want him to pay for any work that he's not 100% satisfied with, and as such you have retracted your invoice as well as the work it covered. You'll keep the associated changes archived in case he wishes to purchase them in the future. You wish him the best of luck going forward and regret that you'll be parting ways.