If it makes you feel any better, you can have a successful business without being in the US, and should you later want to move the startup to the US having a successful business and all the fixings (such as a good immigration lawyer) will make the process quite a bit easier than it is when you're "just somebody with a gleam in his eye".
Immigration officials in both the US and Japan are not set up for tech companies, by the way. I used to work in a governmental unit of the prefecture which, ahem, zealously advocated the national government apply its immigration laws in the best interests of our prefectural high tech industry. If you're interested in trying the visa thing again in the near future, see if you can find yourself a stateside advocate like that. (I'd be kind of surprised if California doesn't have at least one state office which does something like that, probably sort of quietly for the obvious reason.)