With Chromium based browsers you should be able to set every website to be muted at first. You'll only see a mute symbol on the tab when the tab tries to play sound.

If there is no sound at all, there shouldn't be an icon.