Was this a workaround for frequent exception handling being too expensive on some platform? I like how Java lets some exception instances be reused without stack frames, so performance issues are less likely to compromise API design.
It's worth noting that nowadays Parse is just a wrapper around TryParse that throws (although this is an implementation detail)
A parsing failure isn't necessary an error. Sometimes it is, sometimes it's not. Even if exceptions weren't expensive, catching exceptions on all parsing (as you would in Java) is just ugly and, I would argue, not semantically correct. In my opinion, one should throw often but catch infrequently. Most of my large applications have a very small number of catches.
It's also perfectly reasonable to move this down a layer and have Parse return an Option type and choose to handle did-not-parse case or don't and let that trigger an exception.