The first Netflix streaming device from Roku was launched in May of 2008. Prior to that, Roku had a bunch of digital sign products called BrightSign, which is why the scripting language is called BrightScript (actually, I'm not sure if that's Roku, but if not I believe it's another company from the same owner). I understand why they chose BrightScript, they already had it working and had experience with it, I just think it ended up being a problem in the end.

Edit: To answer your original question, Mozilla had an open source engine, and I believe even had a version you could install as a stand alone JS interpreter, Rhino.