We have that in Estonia too, but we also have the extreme-surprise emoji Ö
But Wikipedia says it was a ligature of two letters ("Ь" and "I"), which later changed their values: https://en.wikipedia.org/wiki/Yery#Origin
Not sure what the 'Æ' could stand in for though
The other two are their own vowels, and they used to be rendered in English as "oe" and "ue". It's not clear to me why we decided that it was better to write "Gödel" as "Godel" than as "Goedel".