In English at least, the corresponding Greek letter is normally spelled "upsilon", which is obviously similar. The "psilon" in both "upsilon" and "epsilon" meant somethign like "simple" in Greek. So these letters were the "simple u" and "simple e". (Similarly with "omicron" and "omega", you can probably guess what size of "o" those were. :) Not sure if there are other cool facts about the Greek letters, for example those ending with "eta".

The pronunciation of the Greek name of the letter changed to “ipsilon” recently, less than a thousand years ago; perhaps the Balkan languages adopted the name after that. The original pronunciation was “hupsilon”, but that has been obsolete for over 2500 years. In between, it was pronounced in an in-between way involving the phoneme [y], which doesn't exist in English or Latin.