That's basically what the MuteTab Chrome extension does (although since extensions don't have access to OS functions, it doesn't tell you if anything is playing a sound or not.)

But with the new code they at the very least have it working with Flash. You can try it out right now by downloading a canary build and going to a Flash site (such as homestarrunner.com.)

The reason it works now while it didn't before is that they are passing extra information between Chrome and Flash so that Chrome can keep track of which Flash instances are making sound.

Edit: Thanks justinschuh for being more specific as to how this communication happens.