I think alliteration applies to pronunciation, not orthography. For example, "ceaseless sun" is an alliteration even though it is spelled with both C and S. I wonder if there is a word for the orthographic counterpart, which you describe here (and which I note in another comment, as the benefit of both starting with G).

Only if you pronounce it with a hard G, like in GIF.

Or have I been mispronouncing ‘Google’ all this time?