However, the closest thing I'm aware of to a word "starting out with a silent letter" is debt, which was originally spelled (and pronounced) without the 'b'. The 'b' was added by someone who felt the word should better reflect the latin 'debeo', and was never pronounced at any point.
There's a bit in Shakespeare's _Love's Labour's Lost_ where the pedant Holofernes is ranting about how the "b" should be pronounced in those two words, but as you say, it was always silent.
EDIT: Turns out "salmon" and "solder" were originally "saumon" and "soudor", with the "l" added in the 18th century (agsin to reflect the Latin spelling)
Of course French elevated this spelling conservativism to a fetish, proving that you can go to far.
But I find it sad that languages like German that reform their spelling to make it more phonetic saw away the history and meaning from the words and force you to memorise, rather than deduce, meanings. Likewise teaching English reading via "phonics" ignores the interconnection of the words and makes spelling harder rather than easier.
A funny way to respond to English inserting a spurious 'b' into the French dette / dete.
> I find it sad that languages like German that reform their spelling to make it more phonetic saw away the history and meaning from the words and force you to memorise, rather than deduce, meanings
(my emphasis)
This is overstated to the point of complete absurdity. You are always forced to memorize the meanings of words; that's the difference between words and sentences. To a great extent, if you don't have to memorize the meaning, it's not a word. (There are word-level syntax bits, such as arguably the re- prefix to verbs, but really the fact that the meaning is arbitrary is the essence of a word.)