It's honestly exhausting on all parties involved to try to commit to developing "true" bilingualism; my wife (Japanese) in particular seems to wonder if this level of commitment and effort is really necessary. After all, it's not like our daughters won't be able to speak Japanese if we stop, it's just that the "expected" level of literacy isn't going to be there (sans further effort on our daughters' part, anyway), and we're living in the US, so how important is that in the end?
I too know people who "lost" their chance to keep up with Japanese and regret it now, but there's a real price to pay in developing that, it feels. If nothing else, our daughters "lose" their Saturday, as well as other time to the homework. The saving grace is that our older daughter says she likes the people at the Japanese school better than the her regular American school (which is still a fine school to being with, and a Japanese/English bilingual program to boot).
I'm realizing it's really difficult to know what's best for your kids in the end, and there's some many potential tradeoffs...