I'm not disagreeing that you can make a good product with old tech, but I'd contend that producing a good product and using some of the newer tools are not unrelated, on a number of levels.
I also don't think the argument that Java or C# is somehow a worse fit for agile development is valid. Sure, I might be able to get my app out more quickly with Python or Ruby, but long term it's not like I'm adding thousands of lines of code a week. A startup takes a long time to build, if you are still writing a ton of code half a year out and you aren't making revenue, I think you should take a step back from coding and focus on business development.
I think agile development is hindered or bolstered by your tools (using git instead of svn, for example) more than the language you write your program in.
There's so many ways for a startup to screw up - marketing, financing, hiring, pure bad luck - without throwing the whole unknown of an unfamiliar language or framework into the mix.
Unless you actually need new tech to solve a specific problem, use what you know.
If anything, though, I'm looking at a much wider definition of "startup" than just the technology kind... so I definitely don't think the "tech stack" is somehow more important...