It depends on the language, and is ultimately down to the language authorities/users whether diacritical letters count separately or not.
Some examples (please correct me if I've mistaken anything):
(Modern) English: 26 letters; A-Z. ÆÐÞŒǷȜſꝛ are no longer letters, replaced with alternative letters or digraphs
French: 26 letters; A-Z. ÀÂÇÉÈÊËÎÏÔÙÛÜŸ are letters with diacritics. Æ and Œ are digraphs
German: 26 letters; A-Z. ÄÖÜ are letters with diacritics. ẞ is a ligature
Dutch: 26 letters; A-Z. IJ is a digraph
Spanish: 27 letters; A-Z + Ñ. LL is a digraph
Danish, Norwegian: 29 letters: A-Z + ÆØÅ
Swedish, Finnish: 29 letters; A-Z + ÅÄÖ. Finnish Š and Ž aren't letters and are replaced with digraphs
Icelandic: 32 letters; A-Z - CQWZ + ÁÉÍÓÚÝ + ÐÞÆÖ (all letters)
Hungarian: 44 letters; A-Z + ÁÉÍÓÖŐÚÜŰ + Cs, Dz, Dzs, Gy, Ly, Ny, Sz, Ty, Zs (all letters)