i love japan in general, had a lot of friends luckily that spoke great english, but the cost of living for me was 1700 usd per month (this was back in 2006, of course i lived in high flying shinjuku), and the english barrier was difficult to get over. probably if i was a bit younger and didn't need to seriously advance my career, i would have stuck it out and learned the language. you could probably find a decent apartment for 800-1200 usd, not sure what the rate is now.
i do think the high-tech scene is good in tokyo, the food awesome, the culture one of a kind, i just wasn't patient enough to learn the language.
From what I understand, part of it is just English being vastly different from Japanese, and the other part is the Japanese school system being really terrible at language teaching.