It's a interesting and strange format that I also have never seen. Anyway, some films/videos have subtitles with a description of all sounds. From https://en.wikipedia.org/wiki/Closed_captioning#Terminology
> In the United States and Canada, the terms subtitles and captions have different meanings. Subtitles assume the viewer can hear but cannot understand the language or accent, or the speech is not entirely clear, so they transcribe only dialogue and some on-screen text. Captions aim to describe to the deaf and hard of hearing all significant audio content—spoken dialogue and non-speech information such as the identity of speakers and, occasionally, their manner of speaking—along with any significant music or sound effects using words or symbols.
Subtitles are sometimes in different colours if there are multiple speakers, and a character will then usually have the same colour for the duration of the program. If it's not really ambiguous, they will all just be coloured white.
Captions can be useful for many reasons, including:
- Deaf or hard of hearing.
- Even if you can hear OK, but someone else is talking at the same time or if the TV is muted.
- If you want to know the spelling of the words (especially if it is something unusual such as a character's name).
- If they are speech in multiple languages.
- If some sound is too quiet to be noticeable.
- If you like to read it even though you can hear it OK (having it two ways might be what some people like, including myself).
- To make transcripts or caption scrollback.
What I think would be better is:
- Write all speech, regardless of the language. Foreign speech can be either written in that language, or translated to the language of the captions and adding a note about it.
- If text is on the screen and is being read (e.g. Jeopardy! clues), make hidden captions (I don't know if this can be done, but it ought to be possible to be done). This way, they will still be available in transcripts (or caption scrollback, but as far as I know nothing implements this).
- In some cases, even if text is being read on screen, you may wish to make non-hidden captions if the pronounciation is relevant to the plot and an unusual way to pronounce that word (or is relevant to the plot but is not in the dictionary, e.g. someone's name which is not so easy to understand how to pronounce when it is written down).
- Use closed captions instead of open captions, regardless of the language. Open captions cannot be turned off and the font cannot be adjusted (which might make them hard to read, or obscure a part of the picture).
- If you do not know what someone says because it is unclear, refer to the script, or if there is no script, ask the people in the show, if possible.
- Only significant sound effects (and tone of voice, etc) should be mentioned in the captions. Any sound effects, tone of voice, etc which are not significant should be omitted. (For example, "male announcer reads text on screen" is not significant for the messages between TV shows, but may sometimes be significant within a TV show, if it is a part of the plot, perhaps.)
- Ensure that you know the spelling of the names of the characters and other unusual things, when writing it. This way, any viewer who turns on the captions will know how to spell their name, too.
- If there is a long music that is prominent in the show, write the name of music in the captions. This way, if a viewer has both captions and sound on, and does not know what the music is called, then they can learn.
- In some cases, you may wish to include something even if it is not really relevant to the plot because the audience is not expected to know if it is or not, at that point in the story. Likewise, you can exclude something for similar reasons (e.g. a character whose name has not been mentioned yet, and is revealed later).
- Do not code the captions incorrectly. (Sometimes they are incorrect, causing problems with the display.)
I have seen most the above things sometimes, but not always; they are inconsistent and often do not do a good job, but sometimes it is good.
Another note: on DVD video, the way that subtitles and captions are different, is different than your description. On DVD video, captions are stored as text (and the user can adjust the fonts) and subtitles are stored as pictures with indexed colours (I have not seen any DVD player that you can adjust the colours or translucency of subtitles, even though it seems to me that it should be possible to do).
...But looking it up, it seems the term is actually a nonce only ever used to describe one particular fansub of the type (for a comedy audio drama from the Evangelion franchise, https://www.youtube.com/watch?v=HvLfQSoVhrM)
(indeed, the creator describes[1] why it was coined, and that the use of character portraits was inspired by video games — as sibling mentions, another place to find interesting colours and fonts in subtitles. And anime fansubs aside, I've also seen colour-coded official subs from several countries (must be a list of standards somewhere), e.g. this series on Dutch public TV that you can probably officially watch, because of lax DRM [2]
Now, to somehow tie all that together, a Russian Evangelion cosplay skit with English colour-coded subtitles— https://www.youtube.com/watch?v=8kn76Rc2aa0
[1] https://web.archive.org/web/20030320153637/http://www.absolu...
[2] https://www.npostart.nl/de-slet-van-6-vwo/05-07-2021/AT_2160...)