P.S. If you know of where I could find the full text of SICP in Japanese it'd be great if you could post it.
I worked in China for six months with an office full of guys who could discuss complicated security architecture, race conditions, closures and all kinds of advanced technical topic in English, but couldn't have a social conversation above the third-grade level. It made for a lot of silent lunches. You can see a similar phenomenon at international specialist conferences.
I think the suggestion to read young adult fiction is excellent. Just be prepared to feel humble.
There's a reason for this. The hardest vocabulary to memorise are the small words. The longer words are less common, and often share a root with the English (this is less true for Japanese, Thai, etc) Small words turn up more frequently in prose, so when reading a novel you get drilled on them more often.
Studying a technical work in a foreign language is something I've not tried - it's an interesting idea - but I personally would have trouble dealing with both issues at once - the language and the material.
Will you blog to let us know your progress?