The dots are indeed called umlauts, at least in English:
In English, ¨ is a diacritic mark called diaeresis, which indicates that a vowel is distinct, and shouldn't be diphthongized or dropped: Coöperation, Noël, Brontë.
In German, ¨ is a diacritic mark called umlaut, which transforms the vowels A, O, and U into their umlauted versions: Ä, Ö, and Ü. These characters are not distinct letters of the German alphabet, but belong to a special weird in-between class.
In Swedish, ¨ is not a diacritic mark, does not have a name, and is simply an integral part of the letters Ä and Ö, which together with Å are distinct letters of the Swedish alphabet. The dots aren't modifiers, they're not optional, Ä is not sort-of-an-A, it's as distinct from A as any other vowel, and its pronunciation is closer to E than A.
Ä in Swedish originated as the AE ligature where the E moved upwards and above the A until it became stylized as two dots. In Danish and Norwegian, they instead promoted Æ to a distinct letter.
Wikipedia does a great job for example for Ü, distinguishing between U-umlaut, U-diaeresis, and letter-Ü: https://en.wikipedia.org/wiki/%C3%9C
Well you've just nerd-sniped me. I can't resist adding this bit of hyper-specific knowledge: it is called a tittle.
Conceptually umlaut dots are different from trema dots, but who cares?
But what the two dots are not: they are definitely not diaeresis.
You don't think of "y" being "u" with an Umlaut. For you this is a completely seperate letter. Ditto with ÄÖÅ im Swedish and ÆØÅ in Danish (Æ is never thought of as combined A and E).
No. The German alphabet contains 26 letters, A-Z, plus 4 special letters, ÄÖÜẞ.
The Swedish alphabet contains 29 letters, A-ZÅÄÖ.
Unicode also makes a difference, but generally recommends the merged character.
Otherwise, except when discussing German, I think "two dots" is much more common: https://en.wikipedia.org/wiki/Two_dots_(diacritic)