Finally makes it much more easy, and much more readable, to do the right thing. Not having finally means duplicated code and harder to follow methods where resources have to be managed, and for most developers, this simply means resources are not managed properly at all.
C++ doesn't need finally because it has deterministic destructors, which are used to emulate finally clauses. In my opinion, destructors can cause less readable/predictable code than a proper finally block, but that might be matter of taste.
Comment from the language maintainer:
The only difference is the second example does rethrow the exception. Though this is still possible (however much more to type) it is wrong design. Since obviously you are using the exceptions as control flow.
This seems quite strange. Finally is a construct to manage control flow, and it's essential to control flow during error conditions. He seems to prefer just swallowing a (potentially critical) exception. Either I'm misreading him, or I think he fails to understand the point of either finally, or exception handling in general.