Huh! I actually started working on a system that does something very similar, which ended up being morphed into Bitrated after a few people convinced me that it wouldn't work.
This idea has been brought up in the Bitcoin community from time to time [1] [2], usually referred to as mutual assured destruction, risk deposit or nash equilibrium.
I personally really like this idea and think that it could work well with rational players.
The problem is that there is room for abuse - one party could tell the other party "okay, I'm not going to finish that deal with you. we can both either lose everything, or you pay me 70% of the risk deposit and get to keep 30% of it".
In the long run, rational players should always refuse to such offers. If scammers would know that everyone always refuses, they'll quickly get into a loss and stop doing that. The problem is that I'm not sure how people would react in such situations - why not take the 30%?
As to your idea of publishing history - that could work, if that system could somehow know who's a fraudster and who isn't. Its kinda hard to tell by just looking at the blockchain and seeing what happens with the coins, there could be multiple reasons for the funds getting split.
[1] http://nashx.com/HowItWorks (requires trust in the website operator, as he holds the funds. at least it was like that last time I saw it)
[2] https://gist.github.com/etotheipi/2305966