Comparisons with language of the same family. If practically all modern Germanic languages use an /a/ in the word for "name", it will probably have been an /a/ as well in English. This is more or less the central procedure of comparative linguistics: finding the sound most consistent with other languages, with the least necessary assumptions. Vastly oversimplified of course, it's more complicated than just a majority rule, some sound changes are more likely than others for starters.

Comparisons with coexistent languages we know the pronunciation of. The letters used in English for vowels weren't chosen randomly, they used the letters for the corresponding vowels in (Ecclesiastical) Latin. So by knowing that in Latin the a stood for a vowel similar to /a/, you can say the same of the a in English. This is again only approximate, as not all vowels match perfectly, and English has many more vowels than Latin ever did.

Spelling errors. In graffiti in Pompeii, occasionally the -t at the end of 3rd person singular verb forms isn't written, or the -m in female accusative nouns, which means they weren't pronounced. A future historian could learn a lot about modern English from phrases like "cuz", "tho", "would of", "I red a book", etc..

Known sound changes. All Germanic languages went through a process of i-umlaut, where the presence of an /i/ changed the quality of the vowel in the previous syllable. In German its effects are most clear (hence the name), but it left its marks on English. For example, the plural of mouse being mice. The contemporary vowels for those words, /aʊ/ and /aɪ/, are not related by umlaut, but /u/ and /y/ (with the /y/ changing into /i/ later) would fit.

Loanwords into other languages. If we know something about the phonological history of another language, a centuries old loanword can tell us something about how a word was pronounced. The Finnish word for "ring" is "rengas", which gives a hint that it must have been pronounced with an /e/ 2500 years ago.

All these "hints" and more are combined by linguists, and for some languages this means we have a very high confidence of their earlier phonology. Especially for Middle English, with its vast documentation, many sister languages, well-understood origins, relative recency etc..

One of the ways can be through poetry. Since many poems tend to include rhymes, the pronunciation of a word can sometimes be gleaned by comparing it to its rhyme pair (there's probably an actual word for that). Of course, that presupposes you know the pronunciation of the other word.

Edit: Pretty much what edanm said. I really should do a page refresh before replying when the tab has been open for a while :)

I'm totally guessing, but maybe they use text that shows how something is pronounced?

Eg for one example, if you know how one word is pronounced, and you have a poem that includes that word, you can follow rhymes.

Another clue beyond those mentioned is misspellings: if you see a modern person write “should” as “shud” you know that the “l” is silent.