The word 'ye' in names such as Ye Olde Inn ('the old inn') was originally simply pronounced 'the'.
The pronunciation with y is due to a misinterpretation of the first letter.
'Ye' originated as a second-best way of writing 'þe'. The first letter, þ, was called thorn. It was a letter for the th sound that originated in Old English.
Where did þ come from? How did it disappear? And why was it substituted by y in the word 'ye'?
Click my new infographic to read all about it: