I think it's a brilliant idea.
The technology is definitely there using flash it self to stream the video down, and flash does support capturing video well so pushing it upstream is also there, but the technical challenge that's making it hard is the latency issue and the bandwidth requirement server side (you need to stream live video down AND up in real time).
Justin.TV solve this to a certain extent, so your idea is extending this allowing every participant to upstream as well as downstream.
As much as Skype/Oovoo tries to make it easy, they still require registration and the need for a native client (which try to make direct connections between users as much as possible for said technical issue).
Make it happen!