This has been tried a dozen times. The chicken and egg problem is especially hard in real time. And in general, chat is dead. It's boring. The web has more to offer than it did when chat rooms were popular.
The implementation of rabblr also is really cool. But i think the part with the anonymous users might be a problem. I just logged in for a few minutes and it was full of trolling. Also that you can't carry the chat over multiple pages is anoying, but there might be a way around that. Plus i really don't want to talk to random people on webpages. I probably want to talk to someone who can give me an information that i want, which would be a site adminstrator or company staff or whatever. Like olark. Or i want to talk to my friends or people i know, but i can do that on any IM that i got open anyway.
So to sum it up. Nicely done, but i also doubt the usefulness of this tool.
If you run a team, campfire
If you're whiz kid running a team, hipchat
If you work, gchat
If you're a kid, facebook
If you're old, yahoo
If you're a nitwit, twitter