> Isn't 'th' just the aspirated version of 't'?
'th' is only an aspirated 't' for loan words in English. Most English words pronounce 'th' as the fricatives /θ/ or /ð/.
https://en.wikipedia.org/wiki/Th_(digraph)
"Thyme" is odd because it used to be spelled "tyme" in Middle English (1066-1400s). I assume it was changed to 'th' to be more similar to Latin and French. Something similar happened to "island" (iland) and "isle" (ile) where a silent 's' was added to make the words closer to the Latin "insula".